Displaying 1 to 20 from 197 results

broccoli - Browser compilation library – an asset pipeline for applications that run in the browser

  •    Javascript

A fast, reliable asset pipeline, supporting constant-time rebuilds and compact build definitions. Comparable to the Rails asset pipeline in scope, though it runs on Node and is backend-agnostic. For background and architecture, see the introductory blog post.For the command line interface, see broccoli-cli.

streamjs - Lazy Object Streaming Pipeline for JavaScript

  •    Javascript

Stream.js is a lightweight (2.6 KB minified, gzipped), intensely tested (700+ assertions, 97% coverage) functional programming library for operating upon collections of in-memory data. It requires EcmaScript 5+, has built-in support for ES6 features and works in all current browsers, Node.js and Java 8 Nashorn. Before explaining how Stream.js works in detail, here's a few real world code samples.

go_spider - [ηˆ¬θ™«ζ‘†ζžΆ (golang)] An awesome Go concurrent Crawler(spider) framework

  •    Go

A crawler of vertical communities achieved by GOLANG. Latest stable Release: Version 1.2 (Sep 23, 2014).

data-integration - A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow

  •    Python

Data integration pipelines as code: pipelines, tasks and commands are created using declarative Python code. PostgreSQL as a data processing engine.

gulp - The streaming build system

  •    Javascript

This file will give you a taste of what gulp does. Node already supports a lot of ES2015, to avoid compatibility problem we suggest to install Babel and rename your gulpfile.js as gulpfile.babel.js.

pipeline - PipelineAI: Real-Time Enterprise AI Platform

  •    HTML

Each model is built into a separate Docker image with the appropriate Python, C++, and Java/Scala Runtime Libraries for training or prediction. Use the same Docker Image from Local Laptop to Production to avoid dependency surprises.

papermill - πŸ“š Parameterize, execute, and analyze notebooks

  •    Python

Papermill is a tool for parameterizing, executing, and analyzing Jupyter Notebooks. To parameterize your notebook designate a cell with the tag parameters.

gaia - Build powerful pipelines in any programming language.

  •    Go

Develop powerful pipelines with the help of SDKs and simply check-in your code into a git repository. Gaia automatically clones your code repository, compiles your code to a binary and executes it on-demand. All results are streamed back and formatted to a user-friendly graphical output. Automation Engineer, DevOps, SRE, Cloud Engineer, Platform Engineer - they all have one in common: The majority of tech people are not motivated to take up this work and they are hard to recruit.


  •    Javascript

A fault-tolerant, multi-worker, multi-stage job pipeline built on Firebase

gollum - An n:m message multiplexer written in Go

  •    Go

Gollum is an n:m multiplexer that gathers messages from different sources and broadcasts them to a set of destinations.Gollum originally started as a tool to MUL-tiplex LOG-files (read it backwards to get the name). It quickly evolved to a one-way router for all kinds of messages, not limited to just logs. Gollum is written in Go to make it scalable and easy to extend without the need to use a scripting language.

datakit - Connect processes into powerful data pipelines with a simple git-like filesystem interface

  •    OCaml

DataKit is a tool to orchestrate applications using a Git-like dataflow. It revisits the UNIX pipeline concept, with a modern twist: streams of tree-structured data instead of raw text. DataKit allows you to define complex build pipelines over version-controlled data.DataKit is currently used as the coordination layer for HyperKit, the hypervisor component of Docker for Mac and Windows, and for the DataKitCI continuous integration system.

syntax_sugar_python - A library adding some anti-Pythonic syntatic sugar to Python

  •    Python

This lib adds some anti-Pythonic "syntactic sugar" to Python. NOTE: This is merely an experimental prototype to show some potential of operator overloading in Python. Only tested under Python 3.6.0. Anything may evolve without announcement in advance.

lambdacd - a library to define a continuous delivery pipeline in code

  •    Clojure

I'd love to hear from you! If you have a question, a bug report or feature request please reach out. LambdaCD is built in Clojure and ClojureScript with Leiningen as a build-tool. The ./go script is your main entry-point that wraps all important development tasks. Call it without arguments to see all the options.

nextflow - A DSL for data-driven computational pipelines

  •    Groovy

With the rise of big data, techniques to analyse and run experiments on large datasets are increasingly necessary. Parallelization and distributed computing are the best ways to tackle this kind of problem, but the tools commonly available to the bioinformatics community traditionally lack good support for these techniques, or provide a model that fits badly with the specific requirements in the bioinformatics domain and, most of the time, require the knowledge of complex tools or low-level APIs.

galaxy - Data intensive science for everyone.

  •    Python

You may wish to make changes from the default configuration. This can be done in the config/galaxy.ini file. Note that not all dependencies for the tools provided in the tool_conf.xml.sample are included. To install them please visit "Manage dependencies" in the admin interface.

toil - A scalable, efficient, cross-platform and easy-to-use workflow engine in pure Python

  •    Python

ATTENTION: Toil has moved from https://github.com/BD2KGenomics/toil to https://github.com/DataBiosphere/toil as of July 5th, 2018. Toil is a scalable, efficient, cross-platform pipeline management system, written entirely in Python, and designed around the principles of functional programming.

mu - A full-stack DevOps on AWS framework

  •    Go

Amazon ECS (EC2 Container Service) provides an excellent platform for deploying microservices as containers. The challenge however is that there is a significant learning curve for microservice developers to deploy their applications in an efficient manner. Specifically, they must learn to use CloudFormation to orchestrate the management of ECS, ECR, EC2, ELB, VPC, and IAM resources. Additionally, tools like CodeBuild and CodePipeline must be mastered to create a continuous delivery pipeline for their microservices. To address these challenges, this tool was created to simplify the declaration and administration of the AWS resources necessary to support microservices. Similar to how the Serverless Framework improved the developer experience of Lambda and API Gateway, this tool makes it easier for developers to use ECS as a microservices platform.

Pipeline Style Programming in C#. Inspired by F#.

  •    CSharp

Add F# pipelining like capabilities to your C# code and increase your functional programming experience. Can be ported to be used in Java too.

Commerce Server 2009 Orders using Pipelines in a Console Application

  •    CSharp

Using Commerce Server 2009 outside of a Web Application this project show how to create dummy Orders using Pipelines in a NON WEB CONTEXT, an example is a Console Application but you could make unit/integration/load testing, and winform applications for management purpose.


  •    DotNet

Rendering.NET is an abstraction for any visualization device and over several APIs like OpenGL, DirectX, XNA, WebGL, WPF, Silverlight, Mobile DirectX, etc.