streamparse - Run Python in Apache Storm topologies. Pythonic API, CLI tooling, and a topology DSL.

•    Python

Streamparse lets you run Python code against real-time streams of data via Apache Storm. With streamparse you can create Storm bolts and spouts in Python without having to write a single line of Java. It also provides handy CLI utilities for managing Storm clusters and projects.The Storm/streamparse combo can be viewed as a more robust alternative to Python worker-and-queue systems, as might be built atop frameworks like Celery and RQ. It offers a way to do "real-time map/reduce style computation" against live streams of data. It can also be a powerful way to scale long-running, highly parallel Python processes in production.

kafka-storm-starter - Code examples that show to integrate Apache Kafka 0

•    Scala

Code examples that show how to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark 1.1+ while using Apache Avro as the data serialization format. Take a look at the Kafka Streams code examples at https://github.com/confluentinc/examples.

storm - Simple and powerful toolkit for BoltDB

•    Go

Storm is a simple and powerful toolkit for BoltDB. Basically, Storm provides indexes, a wide range of methods to store and fetch data, an advanced query system, and much more.In addition to the examples below, see also the examples in the GoDoc.

wirbelsturm - Wirbelsturm is a Vagrant and Puppet based tool to perform 1-click local and remote deployments, with a focus on big data related infrastructure

•    Shell

Wirbelsturm is a Vagrant and Puppet based tool to perform 1-click local and remote deployments, with a focus on big data related infrastructure. Wirbelsturm's goal is to make tasks such as "I want to deploy a multi-node Storm cluster" simple, easy, and fun.

Storm

STORM is a free and open source tool for testing web services. It is written mostly in F#. (I love this language!) STORM allows you to 1. Test web services written using any technology (.NET , Java, etc.) 2. Dynamically invoke web service methods even those that h...

bullet-storm - The Apache Storm implementation of the Bullet backend

•    Java

This project implements Bullet on Storm. It also includes the PubSub implementation that uses Storm DRPC as the PubSub.All documentation has moved to Github Pages here.

registry - Schema Registry

•    Java

Registry is a versioned entity framework that allows to build various registry services such as Schema Registry, ML Model Registry etc..

streamline - StreamLine - Streaming Analytics

•    Java

Develop and deploy Streaming Analytics applications visually with bindings for streaming engine and multiple source/sinks, rich set of streaming operators and operational lifecycle management. Streaming Analytics Manager makes it easy to develop, monitor streaming applications and also provides analytics of data thats being processed by streaming application.

SimpleStorm - Simple Storm-like distributed application implementation

•    Javascript

You have Spouts (message sources) and Bolts (message processor). An Spout should have a start function. An spout emit message via its controller, in any of its methods.

storm-cookbook - Chef cookbook for storm

•    Ruby

When you run this cookbook on debian platform, you should run also apt::default recipe before storm recipes. All storm.yaml options are supported through the node['storm']['storm_yaml'] not object. See the attributes/storm_yaml.rb for more details.

storm - Simple ORM for Tornado

•    Python

Storm stands for S imple T ornado O bject R elational M apping. It uses Mongo DB for the backend, but it should be swappable for any storage by implementing the storm.db.Database interface.

node-storm - Node.js multilang protocol implementation for Storm

•    Javascript

Build and run Storm topologies with Node.js. Your topology's node modules are installed before being packaged and submitted to the cluster. If you include node modules that build native code, this can cause problems if the machine submitting the topology is a different platform or architecture than the cluster nodes. In this case it is recommended you build and submit your topology from the same platform as your remote Storm cluster.

crawling-framework - Framework for crawling news articles

•    Java

Framework to simplify news crawling

stormkafkamon - Dumps state of Storm Kafka consumers

•    Python

This tool could be used to perform simple monitoring of spout throughput. Tested against Kafka 0.72 and Storm 0.82 (along with associated Kafka spout from storm-contrib), running on Ubuntu 12.04.

bigdata-notebook

•    Scala

A repository to hold all my Hadoop and Machine Learning related codes.

streaming-readings - Streaming System 相关的论文读物

Streaming System 相关的论文读物

storm-dynamic-spout - A framework for building spouts for Apache Storm and a Kafka based spout for dynamically skipping messages to be processed later

•    Java

A framework for building spouts for Apache Storm and a Kafka based spout for dynamically skipping messages to be processed later.

straw - A platform for real-time streaming search

•    Python

I completed this project as a Fellow in the 2015C Insight Data Engineering Silicon Valley program. The core of the platform is an Apache Storm cluster which parallelizes the work of real-time streaming search. Internally, the Storm cluster consumes messages from a Kafka cluster and these messages are distributed to bolts which each contain a Lucene-Luwak index. The project contains a demo flask UI which handles subscriptions with a Redis PUBSUB system.

shield - 基于Strom的日志实时流量分析主动防御(CCFirewall)系统

基于Strom的日志实时流量分析主动防御(CCFirewall)系统

calories - Calories Tracker for the Commandline

•    Go

Calories is a commandline tool for tracking calories and weight using the Harris Benedict formula for calculating your BMR (Basal Metabolic Rate). When you start calories, it will ask you where to put the calories.db file, which will store all of your data.