FlockDB - A distributed, fault-tolerant graph database from Twitter

  •        991

FlockDB is much simpler than other graph databases such as neo4j because it tries to solve fewer problems. It scales horizontally and is designed for on-line, low-latency, high throughput environments such as web-sites. Twitter uses FlockDB to store social graphs (who follows whom, who blocks whom) and secondary indices. As of April 2010, the Twitter FlockDB cluster stores 13+ billion edges and sustains peak traffic of 20k writes/second and 100k reads/second.

https://github.com/twitter/flockdb

Tags
Implementation
License
Platform

   




Related Projects

Eclipse JNoSQL - Framework which has the goal to help Java developers to create Jakarta EE applications with NoSQL

  •    Java

Eclipse JNoSQL is a Java framework that streamlines the integration of Java applications with NoSQL databases. It defines a set of APIs and provides a standard implementation for most NoSQL databases. This clearly helps to achieve very low coupling with the underlying NoSQL technologies used in applications.

Titan - Scalable Graph Database

  •    Java

Titan is a scalable graph database optimized for storing and querying graphs containing hundreds of billions of vertices and edges distributed across a multi-machine cluster. Titan is a transactional database that can support thousands of concurrent users executing complex graph traversals. It is a native Blueprints enabled graph database and as such, it supports the full TinkerPop stack of technologies.

Aerospike Database Server – Flash-optimized, in-memory, nosql database

  •    C

Aerospike is a distributed, scalable NoSQL database. It provides support to create a high-performance, scalable platform that would meet the needs of today's web-scale applications. It supports operational efficiency, robustness and reliability expected from traditional databases.

Cassandra - Scalable Distributed Database

  •    Java

The Apache Cassandra Project develops a highly scalable second-generation distributed database, bringing together Dynamo's fully distributed design and Bigtable's ColumnFamily-based data model. Cassandra is suitable for applications that can't afford to lose data. Data is automatically replicated to multiple nodes for fault-tolerance.

Graphlite - Embedded graph databases for Python

  •    Python

Graphlite is an embedded graph databases for Python. Graphlite aims to change that by building a simple and fast graph layer over SQLite. Similar to FlockDB, Graphlite only stores adjacency lists, but they can be queried in the style of normal graph databases, e.g. with traversals.


OrientDB - The NoSQL Graph-Document DBMS

  •    Java

OrientDB has the flexibility of the Document databases and the power of the Graph databases to manage relationships. It can work in schema-less mode, schema-full or a mix of both. It can store up to 150,000 records per second on common hardware. OrientDB has been designed to be very fast. It inherits the best features and concepts from the Object Databases, Graph DBMS and the modern NoSQL engines.

agensgraph - AgensGraph, a transactional graph database based on PostgreSQL

  •    C

AgensGraph is a new generation multi-model graph database for the modern complex data environment. AgensGraph is a multi-model database, which supports the relational and graph data model at the same time that enables developers to integrate the legacy relational data model and the flexible graph data model in one database. AgensGraph supports ANSI-SQL and openCypher (http://www.opencypher.org). SQL queries and Cypher queries can be integrated into a single query in AgensGraph. AgensGraph is based on the powerful PostgreSQL RDBMS, and is very robust, fully-featured and ready for enterprise use. AgensGraph is optimized for handling complex connected graph data and provides plenty of powerful database features essential to the enterprise database environment including ACID transactions, multi-version concurrency control, stored procedure, triggers, constraints, sophisticated monitoring and a flexible data model (JSON). Moreover, AgensGraph leverages the rich eco-systems of PostgreSQL and can be extended with many outstanding external modules, like PostGIS.

Neo4j - Graph Database

  •    Java

Neo4j is a high-performance graph engine with all the features of a mature and robust database. It is a graph database, storing data in the nodes and relationships of a graph. It includes the usual database features like ACID transactions, durable persistence, concurrency control, transaction recovery, high availability.

BigchainDB - The Scalable Blockchain Database

  •    Python

BigchainDB allows developers and enterprise to deploy blockchain proof-of-concepts, platforms and applications with a scalable blockchain database, supporting a wide range of industries and use cases. It is a decentralization ecosystem: a decentralized database, at scale. It can perform 1 million writes per second throughput, store petabytes of data, and sub-second latency.

Nebula Graph - A distributed, scalable, lightning-fast graph database

  •    C++

Nebula Graph is an open-source graph database capable of hosting super large-scale graphs with billions of vertices (nodes) and trillions of edges, with milliseconds of latency. It delivers enterprise-grade high performance to simplify the most complex data sets imaginable into meaningful and useful information.

MongoDB - NoSQL Document Store Database

  •    C++

MongoDB (from "humongous") is a scalable, high-performance, open source, dynamic-schema, document-oriented database. MongoDB bridges the gap between key-value stores (which are fast and highly scalable) and traditional RDBMS systems.

RethinkDB - Distributed JSON database

  •    C++

RethinkDB is built to store JSON documents, and scale to multiple machines with very little effort. It has a pleasant query language that supports really useful queries like table joins and group by, and is easy to setup and learn. It supports JSON data model, Distributed joins, subqueries, aggregation, atomic updates, Hadoop-style map/reduce.

Hypertable - A high performance, scalable, distributed storage and processing system for structured

  •    C++

Hypertable is based on Google's Bigtable Design, which is a proven scalable design that powers hundreds of Google services. Many of the current scalable NoSQL database offerings are based on a hash table design which means that the data they manage is not kept physically ordered. Hypertable keeps data physically sorted by a primary key and it is well suited for Analytics.

tidis - Distributed transactional NoSQL database, Redis protocol compatible using tikv as backend

  •    Go

Tidis is a Distributed NoSQL database, providing a Redis protocol API (string, list, hash, set, sorted set), written in Go. Tidis is like TiDB layer, providing protocol transform and data structure compute, powered by TiKV backend distributed storage which use Raft for data replication and 2PC for distributed transaction.

Dgraph - Fast, Transactional, Distributed Graph Database

  •    Go

Dgraph is a horizontally scalable and distributed graph database, providing ACID transactions, consistent replication and linearizable reads. It's built from ground up to perform for a rich set of queries. Being a native graph database, it tightly controls how the data is arranged on disk to optimize for query performance and throughput, reducing disk seeks and network calls in a cluster.

flockdb - A distributed, fault-tolerant graph database

  •    Scala

Twitter is no longer maintaining this project or responding to issues or PRs. FlockDB is much simpler than other graph databases such as neo4j because it tries to solve fewer problems. It scales horizontally and is designed for on-line, low-latency, high throughput environments such as web-sites.

JanusGraph - Distributed graph database

  •    Java

JanusGraph is a highly scalable graph database optimized for storing and querying large graphs with billions of vertices and edges distributed across a multi-machine cluster. JanusGraph is a transactional database that can support thousands of concurrent users, complex traversals, and analytic graph queries.

BangDB - NoSQL for Real Time Performance

  •    C++

Bangdb is pure vanilla key value nosql data store. The goal of bangdb is to be fast, reliable, robust, scalable and easy to use data store for various data management services required by applications. Bangdb comes in flavors like Embedded In memory, Network, Distributed data grid/ Elastic Cache. The bangdb is highly concurrent and runs parallel operations as much as possible.

Skytable - Your next NoSQL database

  •    Rust

Skytable is an effort to provide the best of key/value stores, document stores and columnar databases, that is, simplicity, flexibility and queryability at scale. The name 'Skytable' exemplifies our vision to create a database that has limitless possibilities. It is natively multithreaded and scales to millions of queries per second per node with no optimizations left off the table. The database server doesn't need more than 1MB to run.






We have large collection of open source products. Follow the tags from Tag Cloud >>


Open source products are scattered around the web. Please provide information about the open source projects you own / you use. Add Projects.