flowgraph - Ready-send coordination layer on top of goroutines.

  •        16

Go (Golang) offers direct support for concurrent programming with goroutines, channels, and the select statement. Used together they offer all the building blocks necessary for programming across many cores and many Unix boxes. But so much is possible with goroutines that constructing scaleable and reliable systems (that won't deadlock or be throttled by bottlenecks) requires the application or invention of additional concepts.Flowgraphs are a distinct model of concurrent programming that augment channels with ready-send handshake mechanisms to ensure that no data is sent before the receiver is ready. MPI (a framework for supercomputer computation) directly supports flowgraph computation, but doesn't address flow-based computation within a single Unix process. Go with its goroutines (more efficient than threads according to Rob Pike) facilitates taking the MPI model down to whatever granularity the concurrent programmer wants.

https://github.com/vectaport/flowgraph

Tags
Implementation
License
Platform

   




Related Projects

Flowgraph Viewer

  •    

The flowgraph viewer enables users to view, build and share flowgraphs for the Crysis-franchise. It's built on Silverlight4, using MEF and Mvvmlight.

cpp-taskflow - Fast C++ Parallel Programming with Task Dependency Graphs

  •    C++

A fast C++ header-only library to help you quickly build parallel programs with complex task dependencies. Cpp-Taskflow lets you quickly build parallel dependency graphs using modern C++17. It supports both static and dynamic tasking, and is by far faster, more expressive, and easier for drop-in integration than existing libraries.

tbb - Official Intel(R) Threading Building Blocks (Intel TBB) GitHub repository.

  •    C++

Intel(R) Threading Building Blocks (Intel(R) TBB) lets you easily write parallel C++ programs that take full advantage of multicore performance, that are portable, composable and have future-proof scalability. Here are the latest Changes and Release Notes (contains system requirements and known issues).

Jepsen - A framework for distributed systems verification, with fault injection

  •    Clojure

Jepsen is a Clojure library. A test is a Clojure program which uses the Jepsen library to set up a distributed system, run a bunch of operations against that system, and verify that the history of those operations makes sense. Jepsen has been used to verify everything from eventually-consistent commutative databases to linearizable coordination systems to distributed task schedulers. It can also generate graphs of performance and availability, helping you characterize how a system responds to different faults.

haret - A strongly consistent distributed coordination system, built using proven protocols & implemented in Rust

  •    Rust

haret provides a distributed service built on proven protocols that exposes strongly consistent coordination primitives via an easy to use API, in order to satisfy the essential dependencies of many distributed applications. You can read more about why we decided to build haret here.This README contains quickstart instructions for both developers and users. More information about using haret can be found in the rough and incomplete User Guide.


Channel Manager framework for distributed applications

  •    

The framework implements data flow programming approach ( http://en.wikipedia.org/wiki/Dataflow_programming ) The general idea of the paradigm is building distributed application from components using channels and ports. Each component is an executable (running as separate pro...

Redisson - Redis based In-Memory Data Grid for Java

  •    Java

Redisson - distributed Java objects and services (Set, Multimap, SortedSet, Map, List, Queue, BlockingQueue, Deque, BlockingDeque, Semaphore, Lock, AtomicLong, Map Reduce, Publish / Subscribe, Bloom filter, Spring Cache, Executor service, Tomcat Session Manager, Scheduler service, JCache API) on top of Redis server. Rich Redis client.

TWiki - Wiki and Web 2.0 Application Platform

  •    Perl

TWiki is a flexible, powerful, and easy to use enterprise wiki, enterprise collaboration platform, and web application platform. It is a Structured Wiki, typically used to run a project development space, a document management system, a knowledge base, or any other groupware tool, on an intranet, extranet or the Internet. TWiki is a cgi-bin script written in Perl. It reads a text file, hyperlinks it and converts it to HTML on the fly.

Go kit - A standard library for microservices.

  •    Go

Go kit is a programming toolkit for building microservices (or elegant monoliths) in Go. We solve common problems in distributed systems and application architecture so you can focus on delivering business value. Go is a great general-purpose language, but microservices require a certain amount of specialized support. RPC safety, system observability, infrastructure integration, even program design — Go kit fills in the gaps left by the standard library, and makes Go a first-class language for writing microservices in any organization.

Fossil

  •    C

Fossil is a distributed version control like Git and Mercurial. Fossil also supports distributed bug tracking and distributed wiki all in a single integrated package. It is simple, high-reliability, distributed software configuration management.

ZooKeeper - Centralized service for maintaining configuration information

  •    Java

ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. All of these kinds of services are used in some form or another by distributed applications.

etcd - Distributed reliable key-value store for the most critical data of a distributed system

  •    Go

etcd is a distributed, consistent key-value store for shared configuration and service discovery. It is simple, secure, fast and reliable. it uses the Raft consensus algorithm to manage a highly-available replicated log.

ringpop-go - Scalable, fault-tolerant application-layer sharding for Go applications

  •    Go

Ringpop is a library that brings cooperation and coordination to distributed applications. It maintains a consistent hash ring on top of a membership protocol and provides request forwarding as a routing convenience. It can be used to shard your application in a way that's scalable and fault tolerant.to install remaining golang dependencies and install the pre-commit hook.

Apache Tajo - A big data warehouse system on Hadoop

  •    Java

Apache Tajo is a robust big data relational and distributed data warehouse system for Apache Hadoop. Tajo is designed for low-latency and scalable ad-hoc queries, online aggregation, and ETL (extract-transform-load process) on large-data sets stored on HDFS (Hadoop Distributed File System) and other data sources.

Jaeger - A Distributed Tracing System

  •    Go

Jaeger is a Distributed Tracing System. It is inspired by Dapper and OpenZipkin, is a distributed tracing system released as open source by Uber Technologies. It can be used for monitoring microservice-based architectures.

Hypertable - A high performance, scalable, distributed storage and processing system for structured

  •    C++

Hypertable is based on Google's Bigtable Design, which is a proven scalable design that powers hundreds of Google services. Many of the current scalable NoSQL database offerings are based on a hash table design which means that the data they manage is not kept physically ordered. Hypertable keeps data physically sorted by a primary key and it is well suited for Analytics.

Luxun - A high-throughput, persistent, distributed, publish-subscribe messaging system based on memo

  •    Java

A high-throughput, persistent, distributed, publish-subscribe messaging system based on memory mapped file and Thrift RPC.

Apache Accumulo - Key Value Store based on Google BigTable

  •    Java

The Apache Accumulo sorted, distributed key/value store is a robust, scalable, high performance data storage and retrieval system. Apache Accumulo is based on Google's BigTable design and is built on top of Apache Hadoop, Zookeeper, and Thrift. Apache Accumulo features a few novel improvements on the BigTable design in the form of cell-based access control and a server-side programming mechanism that can modify key/value pairs at various points in the data management process.

Ganglia - scalable distributed monitoring system

  •    C

Ganglia is a scalable distributed monitoring system for high-performance computing systems such as clusters and Grids. It is based on a hierarchical design targeted at federations of clusters. It leverages widely used technologies such as XML for data representation, XDR for compact, portable data transport, and RRDtool for data storage and visualization.

Bagri - XML/Document DB on top of distributed cache

  •    Java

Bagri is a Document Database built on top of distributed cache solution like Hazelcast or Coherence. The system allows to process semi-structured schema-less documents and perform distributed queries on them in real-time. It scales horizontally very well with use of data sharding, when all documents are distributed evenly between distributed cache partitions.