replicator - Automated Cluster and Job Scaling For HashiCorp Nomad

  •        192

Replicator is a fast and highly concurrent Go daemon that provides dynamic scaling of Nomad jobs and worker nodes. Replicator job scaling policies are configured as meta parameters within the job specification. A job scaling policy allows scaling constraints to be defined per task-group. Currently supported scaling metrics are CPU and Memory; there are plans for additional metrics as well as different metric backends in the future. Details of configuring job scaling and other important information can be found on the Replicator Job Scaling wiki page.

https://github.com/elsevier-core-engineering/replicator

Tags
Implementation
License
Platform

   




Related Projects

hashi-ui - A modern user interface for @hashicorp Consul & Nomad

  •    Javascript

For Nomad, it was quite simple, no mobile-optimized, (somewhat) feature-complete and live-updating interface existed.Today the Consul and Nomad UI exist in the same binary, but do not "cross-talk" to each other, but long term goal is to integrate them even closer, so from Nomad Job UI you can see Consul health check status for the job tasks, and vice versa be able to cross-link between two otherwise distinct systems.

nomad-autoscaler - Nomad Autoscaler brings autoscaling to your Nomad workloads.

  •    Go

The Nomad Autoscaler is an autoscaling daemon for Nomad, architectured around plug-ins to allow for easy extensibility in terms of supported metrics sources, scaling targets and scaling algorithms. Horizontal Application Autoscaling: The process of automatically controlling the number of instances of an application to have sufficient work throughput to meet service-level agreements (SLA). In Nomad, horizontal application autoscaling can be achieved by modifying the number of allocations in a task group based on the value of a relevant metric, such as CPU and memory utilization or number of open connections.

Kong - The Microservice API Gateway

  •    Lua

Kong is a cloud-native, fast, scalable, and distributed Microservice Abstraction Layer (also known as an API Gateway, API Middleware or in some cases Service Mesh). Backed by the battle-tested NGINX with a focus on high performance, Kong was made available as an open-source platform in 2015. Under active development, Kong is used in production at thousands of organizations from startups, Global 5000 and Government organizations.

kubernetes-ec2-autoscaler - A batch-optimized scaling manager for Kubernetes

  •    Python

kubernetes-ec2-autoscaler is a node-level autoscaler for Kubernetes on AWS EC2 that is designed for batch jobs. Kubernetes is a container orchestration framework that schedules Docker containers on a cluster, and kubernetes-ec2-autoscaler can scale AWS Auto Scaling Groups based on the pending job queue. The autoscaler can be run anywhere as long as it can access the AWS and Kubernetes APIs, but the recommended way is to set it up as a Kubernetes Pod.

levant - An open source templating and deployment tool for HashiCorp Nomad jobs

  •    Go

Levant is an open source templating and deployment tool for HashiCorp Nomad jobs that provides realtime feedback and detailed failure messages upon deployment issues. Realtime Feedback: Using watchers, Levant provides realtime feedback on Nomad job deployments allowing for greater insight and knowledge about application deployments.


react-native-size-matters - A React-Native utility belt for scaling the size your apps UI across different sized devices

  •    Javascript

A React-Native utility belt for scaling the size your apps UI across different sized devices. When developing with react-native, you need to manually adjust your app to look great on variety of different screen sizes. That's a tedious job. react-native-size-matters provides some simple tooling to make your scaling a whole lot easier. The idea is to develop once on a standard ~5" screen mobile device and then simply apply the provided utils. 📖 You can read more about what led to this library on my blog post, that can be found in this repo or at Soluto Engineering Blog.

aws-autoscaling - Tools and Documentation about using Auto Scaling

  •    Shell

Tools and Documentation about using Auto Scaling

NGINX-Demos - NGINX and NGINX Plus demos

  •    HTML

autoscaling-demo: This demo uses one NGINX Plus instance as a load balancer with two upstream groups, one for NGINX Plus web servers and one for Elasticsearch nodes. All of the instances run in Docker containers. The demo uses both the upstream_conf and status api's. If shows creating a new NGINX Plus environment and adding and removing containers manually and with autoscaling. consul-api-demo: This demo spins up a bunch of docker containers and shows NGINX Plus being used in conjuction with Consul, a service discovery platform. It uses the upstream_conf API in NGINX Plus to add the servers registered with Consul and remove the ones which get deregistered without the need for reloading NGINX Plus. This automates the process of upstream reconfiguration in NGINX Plus based on Consul data using a simple bash script and Consul watches.

gomplate - A flexible commandline tool for template rendering

  •    Go

Read the docs at gomplate.hairyhenderson.ca. gomplate is a template renderer which supports a growing list of datasources, such as: JSON (including EJSON - encrypted JSON), YAML, AWS EC2 metadata, BoltDB, Hashicorp Consul and Hashicorp Vault secrets.

KEDA - Kubernetes-based Event Driven Autoscaling component. It provides event driven scale for any container running in Kubernetes

  •    Go

KEDA is a Kubernetes-based Event Driven Autoscaler. With KEDA, you can drive the scaling of any container in Kubernetes based on the number of events needing to be processed. KEDA works alongside standard Kubernetes components like the Horizontal Pod Autoscaler and can extend functionality without overwriting or duplication. With KEDA you can explicitly map the apps you want to use event-driven scale, with other apps continuing to function. This makes KEDA a flexible and safe option to run alongside any number of any other Kubernetes applications or frameworks.

Nomad - Easily Deploy Applications at Any Scale

  •    Go

Nomad is a cluster manager, designed for both long lived services and short lived batch processing workloads. Developers use a declarative job specification to submit work, and Nomad ensures constraints are satisfied and resource utilization is optimized by efficient task packing. Nomad supports all major operating systems and virtualized, containerized, or standalone applications.

envconsul - Launch a subprocess with environment variables using data from @HashiCorp Consul and Vault

  •    Go

Envconsul provides a convenient way to launch a subprocess with environment variables populated from HashiCorp Consul and Vault. The tool is inspired by envdir and envchain, but works on many major operating systems with no runtime requirements. It is also available via a Docker container for scheduled environments.Envconsul supports 12-factor applications which get their configuration via the environment. Environment variables are dynamically populated from Consul or Vault, but the application is unaware; applications just read environment variables. This enables extreme flexibility and portability for applications across systems.

ecs-deploy - CLI tool to simplify Amazon ECS deployments

  •    Python

ecs-deploy simplifies deployments on Amazon ECS by providing a convinience CLI tool for complex actions, which are executed pretty often. Alternatively you can pass the AWS credentials (via --access-key-id and --secret-access-key) or the AWS configuration profile (via --profile) as options when you run ecs.

Vitess - Database clustering system for horizontal scaling of MySQL

  •    Go

Vitess is a database clustering system for horizontal scaling of MySQL through generalized sharding. By encapsulating shard-routing logic, Vitess allows application code and database queries to remain agnostic to the distribution of data onto multiple shards. With Vitess, you can even split and merge shards as your needs grow, with an atomic cutover step that takes only a few seconds. Vitess has been a core component of YouTube's database infrastructure since 2011, and has grown to encompass tens of thousands of MySQL nodes.

Apollo - :rocket: An open-source platform for cloud native applications based on Apache Mesos and Docker

  •    Python

Apollo is an open source project to aid with building and deploying IAAS and PAAS services. It is particularly geared towards managing containerized applications across multiple hosts, and big data type workloads. Apollo leverages other open source components to provide basic mechanisms for deployment, maintenance, and scaling of infrastructure and applications.However, the majority of initial development is happening on AWS so most of our instructions and scripts are built around that. Stayed tuned for more cloud provider support! If you make it work on other infrastructure please let us know and contribute instructions/code. For more info on cloud support see our roadmap.

vapor-core

  •    PHP

Laravel Vapor is an auto-scaling, serverless deployment platform for Laravel, powered by AWS Lambda. Manage your Laravel infrastructure on Vapor and fall in love with the scalability and simplicity of serverless. Vapor abstracts the complexity of managing Laravel applications on AWS Lambda, as well as interfacing those applications with SQS queues, databases, Redis clusters, networks, CloudFront CDN, and more.

autospotting - Automatically replace AWS EC2 instances in AutoScaling groups with identically configured but up to 90% cheaper spot instances

  •    Go

A simple and easy to use tool designed to significantly lower your Amazon AWS costs by automating the use of spot instances. When installed and enabled on an existing on-demand AutoScaling group, AutoSpotting clones one of your on-demand instances from the group with a spot instance that is cheaper, at least as large (automatically considering memory, CPU cores and disk volumes) and configured identically to it. Once the new spot instance is ready, it is attached to the group and an on-demand instance is detached and terminated to keep the group at constant capacity.

Bee Queue - A simple, fast, robust job/task queue for Node.js, backed by Redis

  •    Javascript

A simple, fast, robust job/task queue for Node.js, backed by Redis.Bee-Queue is meant to power a distributed worker pool and was built with short, real-time jobs in mind. A web server can enqueue a job, wait for a worker process to complete it, and return its results within an HTTP request. Scaling is as simple as running more workers.

Clouddiscovery - The tool to help you discover resources in the cloud environment

  •    Python

Cloudiscovery helps you to analyze resources in your cloud (AWS/GCP/Azure/Alibaba/IBM) account. Now this tool only can check resources in AWS, but they are working to expand to other providers. The tool consists of various commands to help you understand the cloud infrastructure. It provides a CLI to easily perform desired actions.

AutoSpotting - Saves up to 90% of AWS EC2 costs by automating the use of spot instances on existing AutoScaling groups

  •    Go

AutoSpotting is the leading open source spot market automation tool, optimized towards quick/easy/frictionless adoption of the EC2 spot market at any scale. It is usually set up to monitor existing long-running AutoScaling groups with minimal configuration changes(often just tagging them, but even that can be avoided by using their existing tags), yielding the usual 70%-90% Spot cost savings but in a better integrated, more cost effective and easier to adopt way than the alternative tools and solutions.






We have large collection of open source products. Follow the tags from Tag Cloud >>


Open source products are scattered around the web. Please provide information about the open source projects you own / you use. Add Projects.