genie - Distributed Big Data Orchestration Service

  •        421

Genie is a federated job orchestration engine developed by Netflix. Genie provides REST-ful APIs to run a variety of big data jobs like Hadoop, Pig, Hive, Spark, Presto, Sqoop and more. It also provides APIs for managing the metadata of many distributed processing clusters and the commands and applications which run on them.See the official website to find documentation about Genie and specific documentation for various releases.

https://netflix.github.io/genie
https://github.com/Netflix/genie

Tags
Implementation
License
Platform

   




Related Projects

Alluxio - Data orchestration for analytics and machine learning in the cloud

  •    Java

Alluxio (formerly known as Tachyon) is a virtual distributed storage system. It bridges the gap between computation frameworks and storage systems, enabling computation applications to connect to numerous storage systems through a common interface.

ClickHouse - Column-oriented database management system that allows generating analytical data reports in real time

  •    C++

ClickHouse is a fast open-source OLAP database management system. It is column-oriented and allows to generate analytical reports using SQL queries in real-time. Large queries are parallelized using multiple cores, taking all the necessary resources available on the current server. It supports distributed processing, data can reside on different shards. Each shard can be a group of replicas used for fault tolerance. All shards are used to run a query in parallel, transparently for the user.

spring-xd - Spring XD makes it easy to solve common big data problems such as data ingestion and export, real-time analytics, and batch workflow orchestration

  •    Java

While it is possible today to build such solutions using Spring (see the Spring Data Book for details and examples), Spring XD will move well beyond the framework API level by providing an out-of-the-box executable server, a pluggable module system, a high level configuration DSL, a simple model for distributing data processing instances on or off the Hadoop cluster, and more.You can fork the repository and/or monitor JIRA to see what is going on. As always, we consider the feedback from our broad and passionate community to be one of our greatest assets.


spring-boot-security-saml-sample - Sample SAML 2.0 Service Provider with Spring Boot.

  •    Java

Currently Spring Security SAML module doesn't provide a starter for Spring Boot. Moreover, its configuration is XML-based as of this writing. The aim of this project is to explain how to develop a Service Provider (SP) which uses Spring Boot and Spring Security SAML Extension, by defining an annotation-based configuration (Java Configuration). Thymeleaf is also used as template engine.

microservices-basics-spring-boot - Basic architecture framework to create complete microservices using Spring boot and Spring cloud

  •    Java

This repository is an example of how to get Microservices going using Spring Boot, Spring Cloud, Spring OAuth 2 and Netflix OSS frameworks. It also builds on distributed system concepts and tries to provide solutions for common distributed system problem using implementations for circuit breakers, consumer driven contracts etc.

user-guide-springcloud - [Cloudframeworks]Microservices Architecture with Spring Cloud - user guide / [云框架]基于Spring Cloud的微服务架构-用户指南

  •    

Compared with the traditional architecture model, microservices enjoy advantages on the following aspects:language independence, independent process communication, high decoupling, fixed task boundary and on-demand expansion. It's a perfect tool for internet business which requires fast delivery, quick response and constantly testing. Tech gaints such as Twitter, Netflix, Amazon and eBay are all loyal customers of the mircoservices architecture model. At present, three mainstream microservice frameworks are named Spring Cloud, Dubbo and API Gateway. Spring Cloud is a set of tools for rapid construction of distributed systems. Thanks to Spring Boot's development convenience, Spring Cloud provides JVM cloud application which could provide a simple way forconfiguration, service discovery, circuit breakers, intelligent routing, micro-agents, control buses, global locks, decision-making campaigns, distributed sessions, cluster state management and many other functions.

mgmt - Next generation distributed, event-driven, parallel config management!

  •    Go

Mgmt is a next generation automation tool. It has similarities to other tools in the configuration management space, but has a fast, modern, distributed systems approach. The project contains an engine and a language. Please have a look at an introductory video or blog post. Mgmt is a fairly new project. It is usable today, but not yet feature complete. With your help you'll be able to influence our design and get us to 1.0 sooner! Interested developers should read the quick start guide.

spring-boot-security-oauth2-minimal - Minimal implementation of Authorization Server, Resource Server and OAuth2 Client in Spring Boot with Spring Security and JWT

  •    Java

Spring Boot can be pretty magical, especially with regard to auto-configuration. While I'm a fan of convention-over-configuration development (I've used Grails for years), that doesn't mean it always makes sense. At least not at first, to newcomers like me. I found that some of the documentation and examples tend to gloss over "minor details" which makes it hard to grasp why you need to do something, or when the framework takes care of it for you. It took me quite a few hours of trial-and-error, tracing execution paths, reading auto-configuration classes, failing with out-of-date examples, etc.

spring-petclinic - A sample Spring-based application

  •    Java

In its default configuration, Petclinic uses an in-memory database (HSQLDB) which gets populated at startup with data. A similar setup is provided for MySql in case a persistent database configuration is needed. Note that whenever the database type is changed, the data-access.properties file needs to be updated and the mysql-connector-java artifact from the pom.xml needs to be uncommented.The Spring Petclinic master branch in the main spring-projects GitHub org is the "canonical" implementation, currently based on Spring Boot and Thymeleaf. There are quite a few forks in a special GitHub org spring-petclinic. If you have a special interest in a different technology stack that could be used to implement the Pet Clinic then please join the community there.

light-4j - A fast, lightweight and more productive microservices framework

  •    Java

Light 4j is a fast, lightweight and cloud native microservices framework. Light means lightweight, lighting fast and shed light on how to program with modern Java SE. It is 44 times faster than the most popular microservices platform Spring Boot embedded Tomcat and use only 1/5 of memory.

gs-spring-boot - Building an Application with Spring Boot :: Learn how to build an application with minimal configuration

  •    Java

This guide provides a sampling of how Spring Boot helps you accelerate and facilitate application development. As you read more Spring Getting Started guides, you will see more use cases for Spring Boot. It is meant to give you a quick taste of Spring Boot. If you want to create your own Spring Boot-based project, visit Spring Initializr, fill in your project details, pick your options, and you can download either a Maven build file, or a bundled up project as a zip file. You’ll build a simple web application with Spring Boot and add some useful services to it.

nacos-spring-project - Nacos ECO Project for Spring Framework

  •    Java

Nacos Spring Project, which is based on Nacos, fully embraces the Spring ecosystem and is designed to help you build Spring applications rapidly. These features strongly depend on Spring Framework 3.2+ API, and can be seamlessly integrated with any Spring Stack, such as Spring Boot and Spring Cloud.

tut-spring-boot-oauth2 - Spring Boot and OAuth2:: A tutorial on "social" login and single sign on with Facebook and Github

  •    Java

This guide shows you how to build a sample app doing various things with "social login" using OAuth2 and Spring Boot. It starts with a simple, single-provider single-sign on, and works up to a self-hosted OAuth2 Authorization Server with a choice of authentication providers (Facebook or Github). The samples are all single-page apps using Spring Boot and Spring OAuth on the back end. They also all use plain jQuery on the front end, but the changes needed to convert to a different JavaScript framework or to use server side rendering would be minimal. Because one of the samples is a full OAuth2 Authorization Server we have used the shim JAR which supports bridging from Spring Boot 2.0 to the old Spring Security OAuth2 library. The simpler samples could also be implemented using the native OAuth2 support in Spring Boot security features. The configuration is very similar.

spring-fu - Spring Fu is an experimental Kotlin micro-framework based on functional configuration intended to test new ideas for future Spring Boot releases

  •    Kotlin

Spring Fu (for functional APIs) is an experimental Kotlin micro-framework based on functional configuration intended to test ideas for upcoming Spring Boot releases. It makes it easy to create fast and lightweight Spring-powered applications with functional APIs instead of annotations. See the reference documentation for more details.

QConf - Qihoo Distributed Configuration Management System

  •    C++

We have build another interesting proect pika. Pika is a nosql compatible with redis protocol with huge storage space. You can have a try.QConf is a Distributed Configuration Management System! A better replacement of the traditional configuration file. As designed, configuration items which is constantly accessed and modified should be completely separated with application code, and QConf is where it should be.

Micro - Distributed Cloud OS

  •    Go

Micro is an operating system built for the cloud. It addresses the key requirements for building services in the cloud. It leverages the microservices architecture pattern and provides a set of services which act as the building blocks of a platform. Micro deals with the complexity of distributed systems and provides simpler programmable abstractions to build on.






We have large collection of open source products. Follow the tags from Tag Cloud >>


Open source products are scattered around the web. Please provide information about the open source projects you own / you use. Add Projects.