PiCluster - Manage Docker Containers

  •        80

PiCluster is a simple way to manage Docker containers on multiple hosts. Docker Swarm not that good and Kubernetes was too difficult to install currently on ARM. PiCluster will only build and run images from Dockerfile's on the host specified in the config file. This software will work on regular x86 hardware also and is not tied to ARM.

  • Move containers to different hosts in the cluster
  • Run commands in parallel across Nodes
  • Heartbeat for services
  • Easily build and orchestrate Docker images across nodes
  • Web interface
  • Monitor host metrics (Disk, CPU, Memory)
  • HTTP interface
  • Virtual IP Manager
  • Rsyslog Analytics
  • Built-in web terminal to easily run commands on nodes
  • Integrate the Kibana dashboard into PiCluster
  • Integrates with Elasticsearch to store the PiCluster logs.
  • Automatic container failover to different nodes
  • Pull container images from a registry
  • Upload Dockerfile archives to the entire cluster
  • Functions-as-a-Service (FaaS)

https://github.com/picluster/picluster

Tags
Implementation
License
Platform

   




Related Projects

containership - A simple container management platform


Absolutely! First install Vagrant, then install our cloud plugin by running cs plugin add cloud. Once installed, run cs cloud create-cluster vagrant --leaders 1 --followers 2 to bring up a local cluster to play around with.Containership will run on any linux host with node & docker (see our installation guide for more details). If you're using open source Containership, use our official setting up your first cluster guide. If you'd like to manage your Containership cluster using our Containership Cloud management product, sign up for an account and follow the official getting started guide.

Helios - Docker container orchestration platform


Helios is a Docker orchestration platform for deploying and managing containers across an entire fleet of servers. Helios provides a HTTP API as well as a command-line client to interact with servers running your containers. It also keeps a history of events in your cluster including information such as deploys, restarts and version changes.

ACS-Deployment-Tutorial - A tutorial on how to deploy a Dockerised deep learning application on Azure Container Services


Deploying machine learning models can often be tricky due to their numerous dependencies, deep learning models often even more so. One of the ways to overcome this is to use Docker containers. Unfortunately, it is rarely straight-forward. In this tutorial, we will demonstrate how to deploy a pre-trained deep learning model using Azure Container Services, which allows us to orchestrate a number of containers using DC/OS. By using Azure Container Services, we can ensure that it is performant, scalable and flexible enough to accommodate any deep learning framework. The Docker image we will be deploying can be found here. It contains a simple Flask web application with Nginx web server. The deep learning framework we will use is the Microsoft Cognitive Toolkit (CNTK) and we will be using a pre-trained model; specifically the ResNet 152 model.Azure Container Services enables you to configure, construct and manage a cluster of virtual machines pre-configured to run containerized applications. Once the cluster is set up you can use a number of open-source scheduling and orchestration tools, such as Kubernetes and DC/OS. This is ideal for machine learning application since we can use Docker containers which enable us to have ultimate flexibility in the libraries we use and allows us to easily scale up based on demand. While always ensuring that our application remains performant. You can create an ACS through the Azure portal but in this tutorial we will be constructing it using the Azure CLI.



ecs-cid-sample - In this code provided with the blog, we will demonstrate how to use the draining state to update the AMI used by EC2 instances in your cluster by updating the launch configuration of your auto-scaling group


Amazon EC2 Container Service (Amazon ECS) is a container management service that makes it easy to run, stop, and manage Docker containers on a cluster of Amazon EC2 instances. When you run tasks using Amazon ECS, you place them on a cluster, which is a logical grouping of EC2 instances. Amazon ECS downloads your container images from a registry that you specify, and runs those images on the container instances within your cluster.There are times when EC2 instances need to be removed from the cluster, for example cluster scale-down or updating an AMI. Today we have delivered Container Instance Draining to simplify these scenarios. The draining state prevents new tasks from being started on the container instance, notifies the service scheduler to move tasks that are running on the instance to other instances in the cluster, and enables you to wait until tasks have successfully moved before terminating the instance.

acs-engine - Azure Container Service Engine - a place for community to collaborate and build the best open Docker container infrastructure for Azure


The Azure Container Service Engine (acs-engine) generates ARM (Azure Resource Manager) templates for Docker enabled clusters on Microsoft Azure with your choice of DC/OS, Kubernetes, Swarm Mode, or Swarm orchestrators. The input to the tool is a cluster definition. The cluster definition is very similar to (in many cases the same as) the ARM template syntax used to deploy a Microsoft Azure Container Service cluster.Execute make ci to run the checkin validation tests.

Empire - A PaaS built on top of Amazon EC2 Container Service (ECS)


Empire is a control layer on top of Amazon EC2 Container Service (ECS) that provides a Heroku like workflow. It conforms to a subset of the Heroku Platform API, which means you can use the same tools and processes that you use with Heroku, but with all the power of EC2 and Docker. Empire is targeted at small to medium sized startups that are running a large number of microservices and need more flexibility than what Heroku provides.

Kubernetes - Container Cluster Manager


Kubernetes is an open source orchestration system for Docker containers. It handles scheduling onto nodes in a compute cluster and actively manages workloads to ensure that their state matches the users declared intentions. Using the concepts of "labels" and "pods", it groups the containers which make up an application into logical units for easy management and discovery.

Moby Project - An open framework to assemble specialized container systems


Moby is an open-source project created by Docker to advance the software containerization movement. It provides a “Lego set” of dozens of components, the framework for assembling them into custom container-based systems, and a place for all container enthusiasts to experiment and exchange ideas.

Deis - Your PaaS. Your Rules.


Deis (pronounced DAY-iss) is an open source PaaS that makes it easy to deploy and manage applications on your own servers. Deis builds upon Docker and CoreOS to provide a lightweight PaaS with a Heroku-inspired workflow. Deis can deploy any application or service that can run inside a Docker container. In order to be scaled horizontally, applications must follow Heroku's 12-factor methodology and store state in external backing services.

Flocker - Container data volume manager for your Dockerized application


Flocker is an open-source Container Data Volume Manager for your Dockerized applications. By providing tools for data migrations, Flocker gives ops teams the tools they need to run containerized stateful services like databases in production. Unlike a Docker data volume which is tied to a single server, a Flocker data volume, called a dataset, is portable and can be used with any container, no matter where that container is running.

Fn - The container native, cloud agnostic serverless platform.


The Fn project is a container native serverless platform that you can run anywhere -- any cloud or on-premise. It’s easy to use, supports every programming language, and is extensible and performant. Fn packages your functions as containers and runs on any platform supporting Docker. Push your functions to a repository and deploy anywhere--your laptop or the cloud.

powerstrip-flocker - Portable volumes using just the Docker CLI


When you want to run Docker in production, you want to run it across multiple machines and you probably want to use some orchestration tools. However, when you attach a volume to a Docker container, the machine it's running on becomes a pet when it should be cattle.You should be able to run a stateful container with a given volume on any host in your cluster, and the platform should handle moving data around as necessary.

tack - Terraform module for creating Kubernetes cluster running on Container Linux by CoreOS in an AWS VPC


Opinionated Terraform module for creating a Highly Available Kubernetes cluster running on Container Linux by CoreOS (any channel) in an AWS Virtual Private Cloud VPC. With prerequisites installed make all will simply spin up a default cluster; and, since it is based on Terraform, customization is much easier than CloudFormation.The default configuration includes Kubernetes add-ons: DNS, Dashboard and UI.

Kontena - The Developer Friendly Container & Microservices Platform


Kontena is a new developer friendly, open source platform for orchestrating applications that are run on Docker containers. It simplifies deploying and running containerized applications on any infrastructure. By leveraging technologies such as Docker, CoreOS and Weave, it provides complete solution for organizations of any size.

Docker - The Linux container engine


Docker is an open-source engine that automates the deployment of any application as a lightweight, portable, self-sufficient container that will run virtually anywhere. Common use cases for Docker include Automating the packaging and deployment of applications, Creation of lightweight, private PAAS environments, Automated testing and continuous integration/deployment, Deploying and scaling web apps, databases and backend services

faas-netes - Enable Kubernetes as a backend for OpenFaaS (Functions as a Service)


This is a plugin to enable Kubernetes as an OpenFaaS backend. The existing CLI and UI are fully compatible. It also opens up the possibility for other plugins to be built for orchestation frameworks such as Nomad, Mesos/Marathon or even a cloud-managed back-end such as Hyper.sh or Azure ACI.OpenFaaS is an event-driven serverless framework for containers. Any container for Windows or Linux can be leveraged as a serverless function. OpenFaaS is quick and easy to deploy (less than 60 secs) and lets you avoid writing boiler-plate code.

OpenEBS - Containerized Storage for Containers


OpenEBS is containerized block storage written in Go for cloud native and other environments w/ per container (or pod) QoS SLAs, tiering and replica policies across AZs and environments, and predictable and scalable performance.

ecs-machine-learning - An EC2 Container Service Architecture which provides the infrastructure for Deep Learning


AWS Solutions Architects are seeing an emerging type of application for ECS: GPU-accelerated workloads, or, more specifically, workloads that need to leverage large amounts of GPUs across many nodes. For example, at Amazon.com, the Amazon Personalization Team runs significant Machine Learning workloads that leverage many GPUs on Amazon ECS. Let’s take a look at how ECS enables GPU workloads.In order to run GPU-enabled work on an ECS cluster, a Docker image configured with Nvidia CUDA drivers, which allow the container to communicate with the GPU hardware, is built and stored in EC2 Container Registry. An ECS Task Definition is used to point to the container image in ECR and specify configuration for the container at runtime, like how much CPU and memory each container should use, the command to run inside the container, if a data volume should be mounted in the container, where the source dataset lives in S3, and so on.