We have collection of more than 1 Million open source products ranging from Enterprise product to small libraries in all platforms. We aggregate information from all open source repositories. Search and find the best for your needs. Check out projects section.
Scribe - Real time log aggregation used in Facebook
- 14899
Scribe is a server for aggregating log data that's streamed in real time from clients. It is designed to be scalable and reliable. It is developed and maintained by Facebook. It is designed to scale to a very large number of nodes and be robust to network and node failures. There is a scribe server running on every node in the system, configured to aggregate messages and send them to a central scribe server (or servers) in larger groups.
https://github.com/facebook/scribe| Tags | log-analysis log-aggregation messaging-system publish-subscribe facebook |
| Implementation | C++ |
| License | Apache |
| Platform | Linux |
Related Projects
Kafka - A high-throughput distributed messaging system
Kafka provides a publish-subscribe solution that can handle all activity stream data and processing on a consumer-scale web site. This kind of activity (page views, searches, and other user actions) are a key ingredient in many of the social feature on the modern web. This data is typically handled by "logging" and ad hoc log aggregation solutions due to the throughput requirements. This kind of ad hoc solution is a viable solution to providing logging data to Hadoop.
log-analysis log-aggregation messaging-system publish-subscribe messaging log-managementLoki - Multi-tenant log aggregation system inspired by Prometheus
Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. It is designed to be very cost effective and easy to operate. It does not index the contents of the logs, but rather a set of labels for each log stream. It is especially good fit for storing Kubernetes Pod logs. Metadata such as Pod labels is automatically scraped and indexed. By storing compressed, unstructured logs and only indexing metadata, Loki is simpler to operate and cheaper to run.
grafana prometheus logging cloudnative log-collector log-aggregation logs log-shipper ingress-logsliftbridge - Lightweight, fault-tolerant message streams.
Liftbridge provides lightweight, fault-tolerant message streams by implementing a durable stream augmentation for the NATS messaging system. It extends NATS with a Kafka-like publish-subscribe log API that is highly available and horizontally scalable. Use Liftbridge as a simpler and lighter alternative to systems like Kafka and Pulsar or use it to add streaming semantics to an existing NATS deployment. See the introduction post on Liftbridge and this post for more context and some of the inspiration behind it.
nats streaming messaging cloud-native grpc distributed-systems liftbridge message-queue pubsubJafka - A fast and simple distributed publish-subscribe messaging system (mq)
Jafka mq is a distributed publish-subscribe messaging system cloned from Apache Kafka.
publish-subscribe message-queue pub-sub distributed kafka-alternative kafka-cloneLuxun - A high-throughput, persistent, distributed, publish-subscribe messaging system based on memo
A high-throughput, persistent, distributed, publish-subscribe messaging system based on memory mapped file and Thrift RPC.
publish-subscribe message-queue pub-sub distributed thrift-rpcSentry - Realtime Platform-Agnostic Error Logging and Aggregation platform
Sentry is a realtime event logging and aggregation platform. It specializes in monitoring errors and extracting all the information needed to do a proper post-mortem without any of the hassle of the standard user feedback loop.
log log-analysis log-search log-management aggregation event-loggingjafka - a fast distributed publish-subscribe messaging system (mq)
a fast distributed publish-subscribe messaging system (mq)
White-elephant - Hadoop log aggregator and dashboard
White Elephant is a Hadoop log aggregator and dashboard which enables visualization of Hadoop cluster utilization across users. The server is a JRuby web application. In a production environment it can be deployed to tomcat and reads aggregated usage data directly from Hadoop. This data is stored in an in-memory database provided by HyperSQL. Charting is provided by Rickshaw. This project is developed by LinkedIn.
log-analysis log-aggregation hadoop-logs visualization log-managementemitter - High performance, distributed and low latency publish-subscribe platform.
Emitter is a free open source real-time messaging service that connects all devices. This publish-subscribe messaging API is built for speed and security. Emitter is a real-time communication service for connecting online devices. Infrastructure and APIs for IoT, gaming, apps and real-time web. At its core, emitter.io is a distributed, scalable and fault-tolerant publish-subscribe messaging platform based on MQTT protocol and featuring message storage.
pubsub networking emitter mqtt mqtt-broker realtime high-performance low-latencykids - Kids Is Data Stream
Kids is a log aggregation system. It aggregates messages like Scribe and its pub/sub pattern is ported from Redis.
Tremor - An early-stage event processing system for unstructured data with rich support for structural pattern-matching, filtering and transformation
Tremor is an event processing system. It was originally designed as a replacement for software such as Logstash or Telegraf. However tremor has outgrown this singular use case by supporting more complex workflows such as aggregation, rollups, an ETL language, and a query language.
event-processing log-shipper ingress ingress-logs log-aggregation tremor-script tremor-languages tremor-query tremor-runtime hacktoberfest2021 kafkaMinPubSub - 0.1kb publish/subscribe messaging micro-framework
A micro publish/subscribe messaging framework, weighing in at only 198 bytes gzipped. Created by rewriting Peter Higgins' jQuery plugin, MinPubSub is completely self contained with no external dependencies. Keep your projects loosely coupled with this powerful design pattern. Also available on NPM and Bower. If you don't want these methods added to the window object simply modify the immediate function to pass in the JavaScript object of your choosing.
publish subscribe pubsub observerGoAccess - Real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser
GoAccess is an open source real-time web log analyzer and interactive viewer that runs in a terminal on *nix systems or through your browser. It provides fast and valuable HTTP statistics for system administrators that require a visual server report on the fly. It supports nearly all web log formats (Apache, Nginx, Amazon S3, Elastic Load Balancing, CloudFront, etc)
data-analysis analytics nginx apache webserver web-analytics monitoring dashboard command-line log-management log-analysis log-searchgnatsd - High-Performance server for NATS, the cloud native messaging system.
NATS Server is a simple, high performance open source messaging system for cloud native applications, IoT messaging, and microservices architectures. It implements a highly scalable and elegant publish-subscribe (pub/sub) distribution model. The performant nature of NATS make it an ideal base for building modern, reliable, scalable cloud native distributed systems.
messaging message-bus message-queue cloud-native microservices-architecture nats-server distributed-systems cloud cloud-computingscribe - Scribe is a server for aggregating log data streamed in real time from a large number of servers
Scribe is a server for aggregating log data streamed in real time from a large number of servers.
Faye - Simple pub/sub messaging for the web
Faye is a set of tools for simple publish-subscribe messaging between web clients. It ships with easy-to-use message routing servers for Node.js and Rack applications, and clients that can be used on the server and in the browser. It is based on the Bayeux protocol.
pub-sub messaging message-queue publish-subscribe bayeuxnats.js - Node.js client for NATS, the cloud native messaging system.
A Node.js client for the NATS messaging system. The json connect property makes it easier to exchange JSON data with other clients.
nats messaging pubsub publish subscribe queue distributed queueingKafka-Message-Server - Example application based on Apache Kafka framework to show it usage as distributed message server
Apache kafka is yet another precious gem from Apache Software Foundation. Kafka was originally developed at Linkedin and later on became a member of Apache project. Apache Kafka is a distributed publish-subscribe messaging system. Kafka differs from traditional messaging system as it is designed as distributed system, persists messages on disk and supports multiple subscribers. Kafka-Message-Server is an sample application for demonstrating kafka usage as message-server. Please follow the below instructions for productive use of the sample application.
MBassador - Light-weight, high-performance event bus implementing the publish subscribe pattern
MBassador is a light-weight, high-performance event bus implementing the publish subscribe pattern. It is designed for ease of use and aims to be feature rich and extensible while preserving resource efficiency and performance. The core of MBassador is built around a custom data structure that provides non-blocking reads and minimized lock contention for writes such that performance degradation of concurrent read/write access is minimal.
event-bus messaging message-queue pub-sub publish-subscribe
We have large collection of open source products. Follow the tags from
Tag Cloud >>
Open source products are scattered around the web. Please provide information
about the open source projects you own / you use.
Add Projects.
