Cadence - Distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way

  •        23

Cadence is a distributed, scalable, durable, and highly available orchestration engine we developed at Uber Engineering to execute asynchronous long-running business logic in a scalable and resilient way.

Business logic is modeled as workflows and activities. Workflows are the implementation of coordination logic. Its sole purpose is to orchestrate activity executions. Activities are the implementation of a particular task in the business logic. The workflow and activity implementation are hosted and executed in worker processes. These workers long-poll the Cadence server for tasks, execute the tasks by invoking either a workflow or activity implementation, and return the results of the task back to the Cadence server. Furthermore, the workers can be implemented as completely stateless services which in turn allows for unlimited horizontal scaling.

https://github.com/uber/cadence
https://cadenceworkflow.io/

Tags
Implementation
License
Platform

   




Related Projects

zeebe - Distributed Workflow Engine for Microservices Orchestration

  •    Java

Zeebe provides visibility into and control over business processes that span multiple microservices. Starting with Zeebe 0.20.0, the "developer preview" label was removed from Zeebe and the first production-ready version was released .

containerops - DevOps Orchestration Platform

  •    Go

There are many tools, projects, plugins, services adopted in the DevOps workflow. However, no one can cover all DevOps tasks. When developers move from one tool to another, they are facing the huge risk of reinvestment. How should we promote the DevOps process and make iteration more speedy? The principle of DevOps orchestration is to keep your original process working without any changes and just assemble tools, projects, plugins, services into orchestration engine. Improvements to the DevOps process can be done by gradually adding or replacing tools, projects, plugins or services for a smooth migration, and overall, DevOps needs to be promoted step by step, invasive changes are dangerous. Combo the different DevOps services, tools and plugins to implementing DevOps orchestration is very complex, and it should resolve many challenges like deliver data between jobs, or resolve environment consistency for tools or plugins.

genie - Distributed Big Data Orchestration Service

  •    Java

Genie is a federated job orchestration engine developed by Netflix. Genie provides REST-ful APIs to run a variety of big data jobs like Hadoop, Pig, Hive, Spark, Presto, Sqoop and more. It also provides APIs for managing the metadata of many distributed processing clusters and the commands and applications which run on them.See the official website to find documentation about Genie and specific documentation for various releases.

infrakit - A toolkit for creating and managing declarative, self-healing infrastructure.

  •    Go

InfraKit is a toolkit for infrastructure orchestration. With an emphasis on immutable infrastructure, it breaks down infrastructure automation and management processes into small, pluggable components. These components work together to actively ensure the infrastructure state matches the user's specifications. InfraKit therefore provides infrastructure support for higher-level container orchestration systems and can make your infrastructure self-managing and self-healing. In this video, InfraKit was used to build a custom linux operating system (based on linuxkit). We then deployed a cluster of virtual machine instances on a local Mac laptop using the Mac Xhyve hypervisor (HyperKit). A cluster of 3 servers booted up in seconds. Later, after the custom OS image has been updated with a new public key, InfraKit detects the change and orchestrates a rolling update of the nodes. We then deploy the same OS image to a bare-metal ARM server running on Packet.net, where the server uses custom ipxe boot directly from the localhost. It demonstrates some of the key concepts and components in InfraKit and shows how InfraKit can be used to implement an integrated workflow from custom OS image creation to cluster deployment and Day N management. The entire demo is published as a playbook, and you can create your own playbooks too.

deploykit - A toolkit for creating and managing declarative, self-healing infrastructure.

  •    Go

InfraKit is a toolkit for infrastructure orchestration. With an emphasis on immutable infrastructure, it breaks down infrastructure automation and management processes into small, pluggable components. These components work together to actively ensure the infrastructure state matches the user's specifications. InfraKit therefore provides infrastructure support for higher-level container orchestration systems and can make your infrastructure self-managing and self-healing. In this video, InfraKit was used to build a custom linux operating system (based on linuxkit). We then deployed a cluster of virtual machine instances on a local Mac laptop using the Mac Xhyve hypervisor (HyperKit). A cluster of 3 servers booted up in seconds. Later, after the custom OS image has been updated with a new public key, InfraKit detects the change and orchestrates a rolling update of the nodes. We then deploy the same OS image to a bare-metal ARM server running on Packet.net, where the server uses custom ipxe boot directly from the localhost. It demonstrates some of the key concepts and components in InfraKit and shows how InfraKit can be used to implement an integrated workflow from custom OS image creation to cluster deployment and Day N management. The entire demo is published as a playbook, and you can create your own playbooks too.


rexray - REX-Ray is a container storage orchestration engine enabling persistence for cloud native workloads

  •    Go

REX-Ray provides a vendor agnostic storage orchestration engine. The primary design goal is to provide persistent storage for Docker, Kubernetes, and Mesos.It is additionally available as a Go package, CLI tool, and Linux service which enables it to be used for additional use cases.

django-DefectDojo - DefectDojo is an open-source application vulnerability correlation and security orchestration tool

  •    Python

DefectDojo is a security program and vulnerability management tool. DefectDojo allows you to manage your application security program, maintain product and application information, schedule scans, triage vulnerabilities and push findings into defect trackers. Consolidate your findings into one source of truth with DefectDojo. Try out DefectDojo in our testing environment.

Vision Automation Services

  •    

Workflow/Orchestration Automation System

Apache ODE - WS-BPEL implementation that supports web services orchestration using flexible process

  •    Java

Apache ODE is a WS-BPEL implementation that supports web services orchestration using flexible process definitions.

Tendermint - Tendermint Core (BFT Consensus) in Go

  •    Go

Tendermint Core is Byzantine Fault Tolerant (BFT) middleware that takes a state transition machine - written in any programming language - and securely replicates it on many machines.

Freefluo - A Workflow Enactment Engine

  •    

A workflow orchestration tool for web services. It can handle WSDL based web service invocations.

conductor - Conductor is a microservices orchestration engine - https://netflix

  •    Java

Conductor is an orchestration engine that runs in the cloud.Getting Started guide.

orca - orchestration engine

  •    Groovy

Orca is the orchestration engine for Spinnaker. It is responsible for taking a pipeline or task definition and managing the stages and tasks, coordinating the other Spinnaker services. Orca pipelines are composed of stages which in turn are composed of tasks. The tasks of a stage share a common context and can publish to a global context shared across the entire pipeline allowing multiple stages to co-ordinate. For example a bake stage publishes details of the image it creates which is then used by a deploy stage.

gru - Orchestration made easy with Go and Lua

  •    Go

Gru is a fast and concurrent orchestration framework powered by Go and Lua, which allows you to manage your UNIX/Linux systems with ease.You can find the latest documentation here.

Helios - Docker container orchestration platform

  •    Java

Helios is a Docker orchestration platform for deploying and managing containers across an entire fleet of servers. Helios provides a HTTP API as well as a command-line client to interact with servers running your containers. It also keeps a history of events in your cluster including information such as deploys, restarts and version changes.

RackHD - RackHD™ provides hardware management and orchestration services

  •    Python

RackHD is a technology stack created for enabling hardware management and orchestration, to provide cohesive APIs to enable automated infrastructure. In a Converged Infrastructure Platform (CIP) architecture, RackHD software provides hardware management and orchestration (M&O). It serves as an abstraction layer between other M&O layers and the underlying physical hardware. Developers can use the RackHD API to create a user interface that serves as single point of access for managing hardware services regardless of the specific hardware in place. This repository acts as a single source location to help you get or build all the pieces to learn about, take advantage of, and contribute to RackHD.

Joget - Workflow + Low-Code Application Development + Open Source

  •    Java

Joget Workflow is an open source platform to easily build enterprise web apps. It helps to build full-fledged apps e.g. CRM, HR, Healthcare, etc. More than just a workflow or BPM platform, Joget offers full-fledged agile application development capabilities (consisting of processes, forms, lists, CRUD and UI), not just back-end EAI/orchestration/integration or the task-based interface for traditional workflow/BPM products.

cattle - Infrastructure orchestration engine

  •    Java

You probably just want to run Rancher and not Cattle directly.Copyright (c) 2014-2015 Rancher Labs, Inc.

rexray - REX-Ray is a container storage orchestration engine enabling persistence for cloud native workloads

  •    Go

Note: All hosted unstable and staged binaries older than 0.11.2-rc1 have been pruned due to quota restrictions. The long-term goal of the REX-Ray project is to enable collaboration between organizations focused on creating enterprise-grade storage plugins for the Container Storage Interface (CSI). As a rapidly changing specification, CSI support within REX-Ray will be planned when CSI reaches version 1.0, currently projected for a late 2018 release. In the interim, there remains active engagement with the project to support the community.

Softix.Orchestration

  •    WPF

A simple workflow engine written in .NET that will allow non-coders to create and run workflows (like automatic parsing of emails, call center scripts etc)