SaltStack - Infrastructure automation and management system

•    Python

Salt is a new approach to infrastructure management. Easy enough to get running in minutes, scalable enough to manage tens of thousands of servers, and fast enough to communicate with them in seconds. Salt delivers a dynamic communication bus for infrastructures that can be used for orchestration, remote execution, configuration management and much more.

paasta - An open, distributed platform as a service

•    Python

Want to know more about the opinions behind what makes PaaSTA special? Check out the PaaSTA Principles.Note: PaaSTA has been running in production at Yelp for more than a year, and has a number of "Yelpisms" still lingering in the codebase. We have made efforts to excise them, but there are bound to be lingering issues. Please help us by opening an issue or better yet a pull request.

kubicorn - Simple. Cloud Native. Kubernetes. Infrastructure.

•    Go

Create, manage, snapshot, and scale Kubernetes infrastructure in the public cloud.kubicorn is an unofficial project that solves the Kubernetes infrastructure problem and gives users a rich golang library to work with infrastructure.

flynn - A next generation open source platform as a service (PaaS)

•    Go

Flynn is designed to run anything that can run on Linux, not just stateless web apps. Flynn comes with highly available database appliances, including PostgreSQL, MySQL, and MongoDB. Eventually we want Flynn to be the only tool developers and ops teams need to develop, deploy, and manage running software.

minicron - 👨🏻‍💻 Monitor your cron jobs

•    Ruby

minicron makes it simple to monitor your cron jobs and ensure they are running both correctly and on schedule. Latest stable release in 0.9.7 tag but 0.9.x is not being actively developed/supported. This branch is under active development for 1.0.0.

infrakit - A toolkit for creating and managing declarative, self-healing infrastructure.

•    Go

InfraKit is a toolkit for infrastructure orchestration. With an emphasis on immutable infrastructure, it breaks down infrastructure automation and management processes into small, pluggable components. These components work together to actively ensure the infrastructure state matches the user's specifications. InfraKit therefore provides infrastructure support for higher-level container orchestration systems and can make your infrastructure self-managing and self-healing. In this video, InfraKit was used to build a custom linux operating system (based on linuxkit). We then deployed a cluster of virtual machine instances on a local Mac laptop using the Mac Xhyve hypervisor (HyperKit). A cluster of 3 servers booted up in seconds. Later, after the custom OS image has been updated with a new public key, InfraKit detects the change and orchestrates a rolling update of the nodes. We then deploy the same OS image to a bare-metal ARM server running on Packet.net, where the server uses custom ipxe boot directly from the localhost. It demonstrates some of the key concepts and components in InfraKit and shows how InfraKit can be used to implement an integrated workflow from custom OS image creation to cluster deployment and Day N management. The entire demo is published as a playbook, and you can create your own playbooks too.

doctl - A command line tool for DigitalOcean services

•    Go

There are four ways to install doctl: using a package manager, downloading a GitHub release, building a development version from source, or building it with Docker. A package manager allows you to install and keep up with new doctl versions using only a few commands. Currently, doctl is available as part of Homebrew for macOS users and Snap for GNU/Linux users.

kubicorn - Simple, cloud native infrastructure for Kubernetes.

•    Go

Create, manage, snapshot, and scale Kubernetes infrastructure in the public cloud. kubicorn is an free and open source project that solves the Kubernetes infrastructure problem and gives users a rich golang library to work with infrastructure.

sacred - Sacred is a tool to help you configure, organize, log and reproduce experiments developed at IDSIA

•    Python

The documentation is hosted at ReadTheDocs.

itamae - Configuration management tool inspired by Chef, but simpler and lightweight

•    Ruby

Simple and lightweight configuration management tool inspired by Chef. And then excute itamae command to apply a recipe to a local machine.

deploykit - A toolkit for creating and managing declarative, self-healing infrastructure.

•    Go

InfraKit is a toolkit for infrastructure orchestration. With an emphasis on immutable infrastructure, it breaks down infrastructure automation and management processes into small, pluggable components. These components work together to actively ensure the infrastructure state matches the user's specifications. InfraKit therefore provides infrastructure support for higher-level container orchestration systems and can make your infrastructure self-managing and self-healing. In this video, InfraKit was used to build a custom linux operating system (based on linuxkit). We then deployed a cluster of virtual machine instances on a local Mac laptop using the Mac Xhyve hypervisor (HyperKit). A cluster of 3 servers booted up in seconds. Later, after the custom OS image has been updated with a new public key, InfraKit detects the change and orchestrates a rolling update of the nodes. We then deploy the same OS image to a bare-metal ARM server running on Packet.net, where the server uses custom ipxe boot directly from the localhost. It demonstrates some of the key concepts and components in InfraKit and shows how InfraKit can be used to implement an integrated workflow from custom OS image creation to cluster deployment and Day N management. The entire demo is published as a playbook, and you can create your own playbooks too.

jsdelivr - A free, fast, and reliable Open Source CDN for npm and GitHub

We are looking for contributors. Please check open issues in the above repos if you think you could help, or open a new one if you have an idea you'd like to discuss. jsDelivr is a free CDN for open source files. We are tightly integrated with Github and npm allowing us to automatically provide a reliable CDN service to almost every open source project out there.

Awesome-System-for-Machine-Learning - A curated list of research in machine learning system

A curated list of research in machine learning system. Link to the code if available is also present. Now we have a team to maintain this project. You are very welcome to pull request by using our template.

sketchy - A task based API for taking screenshots and scraping text from websites.

•    Javascript

Sketchy is a task based API for taking screenshots and scraping text from websites. Sketchy's capture model contains all of the information associated with screenshotting, scraping, and storing html files from a provided URL. Screenshots (sketches), text scrapes, and html files can either be stored locally or on an S3 bucket. Optionally, token auth can be configured for creating and retrieving captures. Sketchy can also perform callbacks if required.

prefect - The easiest way to automate your data

•    Python

We've rebuilt data engineering for the data science era. Prefect is a new workflow management system, designed for modern infrastructure and powered by the open-source Prefect Core workflow engine. Users organize Tasks into Flows, and Prefect takes care of the rest.

AutoSpotting - Saves up to 90% of AWS EC2 costs by automating the use of spot instances on existing AutoScaling groups

•    Go

AutoSpotting is the leading open source spot market automation tool, optimized towards quick/easy/frictionless adoption of the EC2 spot market at any scale. It is usually set up to monitor existing long-running AutoScaling groups with minimal configuration changes(often just tagging them, but even that can be avoided by using their existing tags), yielding the usual 70%-90% Spot cost savings but in a better integrated, more cost effective and easier to adopt way than the alternative tools and solutions.

sceptre - Build better AWS infrastructure

•    Python

Sceptre is a tool to drive AWS CloudFormation. It automates the mundane, repetitive and error-prone tasks, enabling you to concentrate on building better infrastructure. View our Docker repository. Images available from version 2.0.0 onward.

catalyst - Accelerated deep learning R&D

•    Python

Accelerated deep learning R&D

autospotting - Automatically replace AWS EC2 instances in AutoScaling groups with identically configured but up to 90% cheaper spot instances

•    Go

A simple and easy to use tool designed to significantly lower your Amazon AWS costs by automating the use of spot instances. When installed and enabled on an existing on-demand AutoScaling group, AutoSpotting clones one of your on-demand instances from the group with a spot instance that is cheaper, at least as large (automatically considering memory, CPU cores and disk volumes) and configured identically to it. Once the new spot instance is ready, it is attached to the group and an on-demand instance is detached and terminated to keep the group at constant capacity.

Puppet - Server Automation Framework

•    Ruby

Puppet Open Source is a flexible, customizable designed to help system administrators automate the many repetitive tasks they regularly perform. As a declarative, model-based approach to IT automation, it lets you define the desired state of your infrastructure using the Puppet configuration language. Once these configurations are deployed, Puppet automatically installs the necessary packages and starts the related services, and then regularly enforces the desired state.