bigben - BigBen - a generic, multi-tenant, time-based event scheduler and cron scheduling framework

  •        68

Request and Response channels can be mixed. For example, the event requests can be sent through HTTP APIs but the event triggers (response) can be received through a Kafka Topic.

https://github.com/walmartlabs/bigben
http://walmartlabs.com

Dependencies:

com.walmartlabs.bigben:bigben-commons:1.0.2-SNAPSHOT
com.walmartlabs.bigben:bigben-lib:1.0.2-SNAPSHOT
com.walmartlabs.bigben:bigben-cassandra:1.0.2-SNAPSHOT
com.walmartlabs.bigben:bigben-kafka:1.0.2-SNAPSHOT
com.walmartlabs.bigben:cron:1.0.2-SNAPSHOT
org.slf4j:slf4j-api:1.7.25
com.google.guava:guava:24.0-jre
com.fasterxml.jackson.core:jackson-databind:2.9.4
com.fasterxml.jackson.module:jackson-module-kotlin:2.9.4.1
com.hazelcast:hazelcast:3.8.6
com.ning:async-http-client:1.9.31
javax.ws.rs:javax.ws.rs-api:2.1
org.apache.commons:commons-lang3:3.7
org.apache.commons:commons-text:1.2
org.jetbrains.kotlin:kotlin-reflect:1.2.20

Tags
Implementation
License
Platform

   




Related Projects

Hazelcast Jet - A general purpose distributed data processing engine, built on top of Hazelcast.

  •    Java

Hazelcast Jet is a distributed computing platform built for high-performance stream processing and fast batch processing. It embeds Hazelcast In-Memory Data Grid (IMDG) to provide a lightweight, simple-to-deploy package that includes scalable in-memory storage. Hazelcast Jet performs parallel execution to enable data-intensive applications to operate in near real-time.

Hazelcast Jet - Distributed data processing engine, built on top of Hazelcast

  •    Java

Hazelcast Jet is a distributed computing platform built for high-performance stream processing and fast batch processing. It embeds Hazelcast In Memory Data Grid (IMDG) to provide a lightweight package of a processor and a scalable in-memory storage. It supports distributed java.util.stream API support for Hazelcast data structures such as IMap and IList, Distributed implementations of java.util.{Queue, Set, List, Map} data structures highly optimized to be used for the processing

Cassandra - Scalable Distributed Database

  •    Java

The Apache Cassandra Project develops a highly scalable second-generation distributed database, bringing together Dynamo's fully distributed design and Bigtable's ColumnFamily-based data model. Cassandra is suitable for applications that can't afford to lose data. Data is automatically replicated to multiple nodes for fault-tolerance.

akka-analytics - Large-scale event processing with Akka Persistence and Apache Spark

  •    Scala

Events for a given persistenceId are partitioned across nodes in the Cassandra cluster where the partition is represented by the partition field in the key. The eventTable() method returns an RDD in which events with the same persistenceId - partition combination (= cluster partition) are ordered by increasing sequenceNr but the ordering across cluster partitions is not defined. If needed the RDD can be sorted with sortByKey() by persistenceId, partition and sequenceNr in that order of significance. Btw, the default size of a cluster partition in the Cassandra journal is 5000000 events (see akka-persistence-cassandra). The stream of events (written by all persistent actors) is partially ordered i.e. events with the same persistenceId are ordered by sequenceNr whereas the ordering of events with different persistenceId is not defined. Details about Kafka consumer params are described here.

Hazelcast - In-Memory Data Grid for Java

  •    Java

Hazelcast is a clustering and highly scalable data distribution platform for Java. It supports Distributed implementations of java.util.{Queue, Set, List, Map}, java.util.concurrency.locks.Lock, java.util.concurrent.ExecutorService, Distributed Indexing and Query support, Dynamic scaling, partitioning with backups, fail-over, Web-based cluster monitoring tool and lot more.


cassieq - A distributed queue built off cassandra

  •    Javascript

CassieQ is a distributed queue built on cassandra. CassieQ leverages bucketing of messages to avoid queues modeled off deletes.

YugaByte Database - Transactional, high-performance database for building internet-scale, globally-distributed applications

  •    C++

A cloud-native database for building mission-critical applications. This repository contains the Community Edition of the YugaByte Database.YugaByte offers both SQL and NoSQL in a single, unified db. It is meant to be a system-of-record/authoritative database that applications can rely on for correctness and availability. It allows applications to easily scale up and scale down in the cloud, on-premises or across hybrid environments without creating operational complexity or increasing the risk of outages.

Kairosdb - Fast distributed scalable time series database written on top of Cassandra

  •    Java

KairosDB is a fast distributed scalable time series database written on top of Cassandra. Data can be pushed in KairosDB via multiple protocols : Telnet, Rest, Graphite. KairosDB stores time series in Cassandra, the popular and performant NoSQL datastore. It supports aggregators which can perform an operation on data points and down samples. Standard functions like min, max, sum, count, mean etc.

Bagri - XML/Document DB on top of distributed cache

  •    Java

Bagri is a Document Database built on top of distributed cache solution like Hazelcast or Coherence. The system allows to process semi-structured schema-less documents and perform distributed queries on them in real-time. It scales horizontally very well with use of data sharding, when all documents are distributed evenly between distributed cache partitions.

cronsun - A Distributed, Fault-Tolerant Cron-Style Job System.

  •    Go

cronsun is a distributed cron-style job system. It's similar with crontab on stand-alone *nix. The goal of this project is to make it much easier to manage jobs on lots of machines and provides high availability. cronsun is different from Azkaban, Chronos, Airflow.

dkron - Dkron - Distributed, fault tolerant job scheduling system http://dkron.io

  •    Go

Dkron is written in Go and leverage the power of distributed key-value stores and serf for providing fault tolerance, reliability and scalability while keeping simple and easily instalable. Dkron is inspired by the google whitepaper Reliable Cron across the Planet and by Airbnb Chronos borrowing the same features from it.

Solandra - Solandra = Solr + Cassandra

  •    Java

Solandra is a real-time distributed search engine built on Apache Solr and Apache Cassandra. At its core, Solandra is a tight integration of Solr and Cassandra, meaning within a single JVM both Solr and Cassandra are running, and documents are stored and disributed using Cassandra's data model.

Hawkular metrics - Time Series Metrics Engine based on Cassandra

  •    Java

Hawkular Metrics is a metrics collection, aggregation, visualization framework. Hawkular is a set of Open Source projects designed to be a generic solution for common monitoring problems. The Hawkular projects provide REST services that can be used for all kinds of monitoring needs. The aim is to provide a generic solution that can be used for common monitoring problems.

stream-reactor - Streaming reference architecture for ETL with Kafka and Kafka-Connect

  •    Scala

Lenses offers SQL (for data browsing and Kafka Streams), Kafka Connect connector management, cluster monitoring and more. A collection of components to build a real time ingestion pipeline.

BigBen - Monitoring utility

  •    Perl

BigBen is a client/server application, written in Perl, and built to monitor the status of several systems from a central point. It monitors processes, disk space and system load in a configurable way.

chronos - Fault tolerant job scheduler for Mesos which handles dependencies and ISO8601 based schedules

  •    Scala

Chronos is a replacement for cron. It is a distributed and fault-tolerant scheduler that runs on top of Apache Mesos that can be used for job orchestration. It supports custom Mesos executors as well as the default command executor. Thus by default, Chronos executes sh (on most systems bash) scripts. Chronos can be used to interact with systems such as Hadoop (incl. EMR), even if the Mesos agents on which execution happens do not have Hadoop installed. Included wrapper scripts allow transfering files and executing them on a remote machine in the background and using asynchronous callbacks to notify Chronos of job completion or failures. Chronos is also natively able to schedule jobs that run inside Docker containers.

GeoMesa - Suite of tools for working with big geo-spatial data in a distributed fashion

  •    Scala

GeoMesa is an open-source, distributed, spatio-temporal database built on a number of distributed cloud data storage systems, including Accumulo, HBase, Cassandra, and Kafka. Leveraging a highly parallelized indexing strategy, GeoMesa aims to provide as much of the spatial querying and data manipulation to Accumulo as PostGIS does to Postgres.

JanusGraph - Distributed graph database

  •    Java

JanusGraph is a highly scalable graph database optimized for storing and querying large graphs with billions of vertices and edges distributed across a multi-machine cluster. JanusGraph is a transactional database that can support thousands of concurrent users, complex traversals, and analytic graph queries.

twitter-cassandra

  •    Ruby

A Ruby client for the Cassandra distributed database

facebook-cassandra

  •    

Cassandra is a distributed storage system for managing structured data while providing reliability at a massive scale.