Scribe is a server for aggregating log data that's streamed in real time from clients. It is designed to be scalable and reliable. It is developed and maintained by Facebook. It is designed to scale to a very large number of nodes and be robust to network and node failures. There is a scribe server running on every node in the system, configured to aggregate messages and send them to a central scribe server (or servers) in larger groups.https://github.com/facebook/scribe
|Tags||log-analysis log-aggregation messaging-system publish-subscribe facebook|
Kafka provides a publish-subscribe solution that can handle all activity stream data and processing on a consumer-scale web site. This kind of activity (page views, searches, and other user actions) are a key ingredient in many of the social feature on the modern web. This data is typically handled by "logging" and ad hoc log aggregation solutions due to the throughput requirements. This kind of ad hoc solution is a viable solution to providing logging data to Hadoop.log-analysis log-aggregation messaging-system publish-subscribe messaging log-management
Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. It is designed to be very cost effective and easy to operate. It does not index the contents of the logs, but rather a set of labels for each log stream. It is especially good fit for storing Kubernetes Pod logs. Metadata such as Pod labels is automatically scraped and indexed. By storing compressed, unstructured logs and only indexing metadata, Loki is simpler to operate and cheaper to run.grafana prometheus logging cloudnative log-collector log-aggregation logs log-shipper ingress-logs
Liftbridge provides lightweight, fault-tolerant message streams by implementing a durable stream augmentation for the NATS messaging system. It extends NATS with a Kafka-like publish-subscribe log API that is highly available and horizontally scalable. Use Liftbridge as a simpler and lighter alternative to systems like Kafka and Pulsar or use it to add streaming semantics to an existing NATS deployment. See the introduction post on Liftbridge and this post for more context and some of the inspiration behind it.nats streaming messaging cloud-native grpc distributed-systems liftbridge message-queue pubsub
Jafka mq is a distributed publish-subscribe messaging system cloned from Apache Kafka.publish-subscribe message-queue pub-sub distributed kafka-alternative kafka-clone
A high-throughput, persistent, distributed, publish-subscribe messaging system based on memory mapped file and Thrift RPC.publish-subscribe message-queue pub-sub distributed thrift-rpc
Sentry is a realtime event logging and aggregation platform. It specializes in monitoring errors and extracting all the information needed to do a proper post-mortem without any of the hassle of the standard user feedback loop.log log-analysis log-search log-management aggregation event-logging
a fast distributed publish-subscribe messaging system (mq)
White Elephant is a Hadoop log aggregator and dashboard which enables visualization of Hadoop cluster utilization across users. The server is a JRuby web application. In a production environment it can be deployed to tomcat and reads aggregated usage data directly from Hadoop. This data is stored in an in-memory database provided by HyperSQL. Charting is provided by Rickshaw. This project is developed by LinkedIn.log-analysis log-aggregation hadoop-logs visualization log-management
Emitter is a free open source real-time messaging service that connects all devices. This publish-subscribe messaging API is built for speed and security. Emitter is a real-time communication service for connecting online devices. Infrastructure and APIs for IoT, gaming, apps and real-time web. At its core, emitter.io is a distributed, scalable and fault-tolerant publish-subscribe messaging platform based on MQTT protocol and featuring message storage.pubsub networking emitter mqtt mqtt-broker realtime high-performance low-latency
Kids is a log aggregation system. It aggregates messages like Scribe and its pub/sub pattern is ported from Redis.
Tremor is an event processing system. It was originally designed as a replacement for software such as Logstash or Telegraf. However tremor has outgrown this singular use case by supporting more complex workflows such as aggregation, rollups, an ETL language, and a query language.event-processing log-shipper ingress ingress-logs log-aggregation tremor-script tremor-languages tremor-query tremor-runtime hacktoberfest2021 kafka
GoAccess is an open source real-time web log analyzer and interactive viewer that runs in a terminal on *nix systems or through your browser. It provides fast and valuable HTTP statistics for system administrators that require a visual server report on the fly. It supports nearly all web log formats (Apache, Nginx, Amazon S3, Elastic Load Balancing, CloudFront, etc)data-analysis analytics nginx apache webserver web-analytics monitoring dashboard command-line log-management log-analysis log-search
NATS Server is a simple, high performance open source messaging system for cloud native applications, IoT messaging, and microservices architectures. It implements a highly scalable and elegant publish-subscribe (pub/sub) distribution model. The performant nature of NATS make it an ideal base for building modern, reliable, scalable cloud native distributed systems.messaging message-bus message-queue cloud-native microservices-architecture nats-server distributed-systems cloud cloud-computing
Scribe is a server for aggregating log data streamed in real time from a large number of servers.
Faye is a set of tools for simple publish-subscribe messaging between web clients. It ships with easy-to-use message routing servers for Node.js and Rack applications, and clients that can be used on the server and in the browser. It is based on the Bayeux protocol.pub-sub messaging message-queue publish-subscribe bayeux
A Node.js client for the NATS messaging system. The json connect property makes it easier to exchange JSON data with other clients.nats messaging pubsub publish subscribe queue distributed queueing
Apache kafka is yet another precious gem from Apache Software Foundation. Kafka was originally developed at Linkedin and later on became a member of Apache project. Apache Kafka is a distributed publish-subscribe messaging system. Kafka differs from traditional messaging system as it is designed as distributed system, persists messages on disk and supports multiple subscribers. Kafka-Message-Server is an sample application for demonstrating kafka usage as message-server. Please follow the below instructions for productive use of the sample application.
MBassador is a light-weight, high-performance event bus implementing the publish subscribe pattern. It is designed for ease of use and aims to be feature rich and extensible while preserving resource efficiency and performance. The core of MBassador is built around a custom data structure that provides non-blocking reads and minimized lock contention for writes such that performance degradation of concurrent read/write access is minimal.event-bus messaging message-queue pub-sub publish-subscribe
We have large collection of open source products. Follow the tags from
Tag Cloud >>
Open source products are scattered around the web. Please provide information
about the open source projects you own / you use.