Nomad - Easily Deploy Applications at Any Scale

  •        768

Nomad is a cluster manager, designed for both long lived services and short lived batch processing workloads. Developers use a declarative job specification to submit work, and Nomad ensures constraints are satisfied and resource utilization is optimized by efficient task packing. Nomad supports all major operating systems and virtualized, containerized, or standalone applications.
Jobs can specify tasks which are Docker containers. Nomad will automatically run the containers on clients which have Docker installed, scale up and down based on the number of instances requested, and automatically recover from failures. Nomad is designed to be a global-scale scheduler. Multiple datacenters can be managed as part of a larger region, and jobs can be scheduled across datacenters if requested. Multiple regions join together and federate jobs making it easy to run jobs anywhere.

https://www.nomadproject.io/
https://github.com/hashicorp/nomad

Tags
Implementation
License
Platform

   




Related Projects

cluster-scheduler-simulator - Automatically exported from code


This simulator can be used to prototype and compare different cluster scheduling strategies and policies. It generates synthetic cluster workloads from empirical parameter distributions (thus generating unique workloads even from a small amount of input data), simulates their scheduling and execution using a discrete event simulator, and finally permits analysis of scheduling performance metrics.While the simulator will simulate job arrival, scheduler decision making and task placement, it does not simulate the actual execution of the tasks or variation in their runtime due to shared resources.

amazon-ecs-java-microservices - This is a reference architecture for java microservice on Amazon ECS


Containers are a method of operating system virtualization that allow you to run an application and its dependencies in resource-isolated processes. Containers allow you to easily package an application's code, configurations, and dependencies into easy to use building blocks that deliver environmental consistency, operational efficiency, developer productivity, and version control. Containers can help ensure that applications deploy quickly, reliably, and consistently regardless of deployment environment. Containers also give you more granular control over resources giving your infrastructure improved efficiency.Running a single container on a single server is easy. ECS is a cluster management service that helps you manage a group of clusters through a graphical user interface or by accessing a command line. With ECS you can install, operate, and scale your own cluster management infrastructure. With simple API calls, you can launch and stop Docker-enabled applications, query the complete state of your cluster, and access many familiar features like security groups, Elastic Load Balancing, EBS volumes, and IAM roles. You can use Amazon ECS to schedule the placement of containers across your cluster based on your resource needs and availability requirements. You can also integrate your own scheduler or third-party schedulers to meet business or application specific requirements.

hashi-ui - A modern user interface for @hashicorp Consul & Nomad


For Nomad, it was quite simple, no mobile-optimized, (somewhat) feature-complete and live-updating interface existed.Today the Consul and Nomad UI exist in the same binary, but do not "cross-talk" to each other, but long term goal is to integrate them even closer, so from Nomad Job UI you can see Consul health check status for the job tasks, and vice versa be able to cross-link between two otherwise distinct systems.

spark-cluster-deployment - Automates Spark standalone cluster tasks with Puppet and Fabric.


Apache Spark is a research project for distributed computing which interacts with HDFS and heavily utilizes in-memory caching. Spark 1.0.0 can be deployed to traditional cloud and job management services such as EC2, Mesos, or Yarn. Further, Spark's standalone cluster mode enables Spark to run on other servers without installing other job management services.However, configuring and submitting applications to a Spark 1.0.0 standalone cluster currently requires files to be synchronized across the entire cluster, including the Spark installation directory. This project utilizes Fabric and Puppet to further automate the Spark standalone cluster. The Puppet scripts are MIT-licensed from stefanvanwouw/puppet-spark and wikimedia/puppet-cdh4.

drmaa - Compute cluster (HPC) job submission library for Go (#golang) based on the open DRMAA standard


This is a job submission library for Go (#golang) which is compatible to the DRMAA standard. The Go library is a wrapper around the DRMAA C library implementation provided by many distributed resource managers (cluster schedulers).The library was developed using Univa Grid Engine's libdrmaa.so. It was tested with Grid Engine and Torque, but it should work also other resource manager / cluster scheduler.



Kubernetes - Container Cluster Manager


Kubernetes is an open source orchestration system for Docker containers. It handles scheduling onto nodes in a compute cluster and actively manages workloads to ensure that their state matches the users declared intentions. Using the concepts of "labels" and "pods", it groups the containers which make up an application into logical units for easy management and discovery.

containership - A simple container management platform


Absolutely! First install Vagrant, then install our cloud plugin by running cs plugin add cloud. Once installed, run cs cloud create-cluster vagrant --leaders 1 --followers 2 to bring up a local cluster to play around with.Containership will run on any linux host with node & docker (see our installation guide for more details). If you're using open source Containership, use our official setting up your first cluster guide. If you'd like to manage your Containership cluster using our Containership Cloud management product, sign up for an account and follow the official getting started guide.

ecs-cid-sample - In this code provided with the blog, we will demonstrate how to use the draining state to update the AMI used by EC2 instances in your cluster by updating the launch configuration of your auto-scaling group


Amazon EC2 Container Service (Amazon ECS) is a container management service that makes it easy to run, stop, and manage Docker containers on a cluster of Amazon EC2 instances. When you run tasks using Amazon ECS, you place them on a cluster, which is a logical grouping of EC2 instances. Amazon ECS downloads your container images from a registry that you specify, and runs those images on the container instances within your cluster.There are times when EC2 instances need to be removed from the cluster, for example cluster scale-down or updating an AMI. Today we have delivered Container Instance Draining to simplify these scenarios. The draining state prevents new tasks from being started on the container instance, notifies the service scheduler to move tasks that are running on the instance to other instances in the cluster, and enables you to wait until tasks have successfully moved before terminating the instance.

evQueue - Job scheduler and queueing engine


evQueue is an open source job scheduler and queueing engine. It features an event-driven C++ engine and a PHP / MySQL web control interface which provides tasks monitoring and creation. It provides both simple task execution and complex task chaining (workflow) using an easy to use drag & drop web interface. Workflow description includes output linking to input, conditions, loops... Queues management provides an easy way for task parallelization and resource control.

nomad-helper - Helper command to help ease working with HashiCorp's Nomad scheduler


nomad-helper is a tool meant to enable teams to quickly onboard themselves with nomad, by exposing scaling functionality in a simple to use and share yaml format.The project got build artifacts for linux, darwin and windows in the GitHub releases tab.

kel-cluster - Kel cluster management Python library


Kel cluster management Python library. It is a part of the Kel Project.Kel is an open source Platform as a Service (PaaS) from Eldarion, Inc. that makes it easy to manage web application deployment and hosting through the entire lifecycle from development through testing to production. It adds components and tools on top of Kubernetes that help developers manage their application infrastructure. Kel builds on Eldarion's 7+ years experience running one of the leading Python and Django PaaSes.

nomad-firehose - Firehose all nomad job, allocation, nodes and evaluations changes to rabbitmq, kinesis or stdout


nomad-firehose is a tool meant to enable teams to quickly build logic around nomad task events without hooking into Nomad API.The project got build artifacts for linux, darwin and windows in the GitHub releases tab.

Helix - Cluster Management Framework


Helix is a generic cluster management framework used for the automatic management of partitioned, replicated and distributed resources hosted on a cluster of nodes. It helps to perform scheduling of maintenance tasks, such as backups, garbage collection, file consolidation, index rebuilds, repartitioning of data or resources across the cluster, informing dependent systems of changes so they can react appropriately to cluster changes, throttling system tasks and changes and so on.

PiCluster - Manage Docker Containers


PiCluster is a simple way to manage Docker containers on multiple hosts. Docker Swarm not that good and Kubernetes was too difficult to install currently on ARM. PiCluster will only build and run images from Dockerfile's on the host specified in the config file. This software will work on regular x86 hardware also and is not tied to ARM.

kubernetes-ovn-heterogeneous-cluster - Bootstrap a Kubernetes cluster that can run both Linux and Windows workloads on Google Cloud


At the time of this writing, the instructions are meant to be run on Google Compute Engine, but apart from gcloud calls and a few networking details, everything detailed below should work regardless of the adopted cloud-provider.Go Here for instructions on an automated deployment of the same cluster described in the manual steps above on Google Cloud Platform.

Redisson - Redis based In-Memory Data Grid for Java


Redisson - distributed Java objects and services (Set, Multimap, SortedSet, Map, List, Queue, BlockingQueue, Deque, BlockingDeque, Semaphore, Lock, AtomicLong, Map Reduce, Publish / Subscribe, Bloom filter, Spring Cache, Executor service, Tomcat Session Manager, Scheduler service, JCache API) on top of Redis server. Rich Redis client.

OpenEBS - Containerized Storage for Containers


OpenEBS is containerized block storage written in Go for cloud native and other environments w/ per container (or pod) QoS SLAs, tiering and replica policies across AZs and environments, and predictable and scalable performance.

NSQ - A realtime distributed messaging platform in Go


NSQ is a realtime distributed messaging platform designed to operate at scale, handling billions of messages per day. It promotes distributed and decentralized topologies without single points of failure, enabling fault tolerance and high availability coupled with a reliable message delivery guarantee. It scales horizontally, without any centralized brokers. Built-in discovery simplifies the addition of nodes to the cluster.

pycluster - python server client cluster job scheduler


python server client cluster job scheduler

nomad-on-kubernetes - Tutorial on running Nomad on Kubernetes.


The Nomad on Kubernetes tutorial documents how to deploy Nomad on Kubernetes. The following diagram provides a high level overview of the Nomad on Kubernetes architecture.Nomad and Kubernetes have many differences in terms of managing applications but have also been found to complement each other. Nomad is workload agnostic and supports running non-containerized applications -- broadening the type of workloads you can run across your infrastructure. Other components of the Nomad stack such as Consul and Vault can be leveraged directly within Kubernetes. Consul can provide federated service discovery across multiple Kubernetes clusters and existing platforms such as virtual machines. Vault can provide robust secrets management to Kubernetes workloads including dynamic secret generation.