Kubernetes-GPU-Guide - This guide should help fellow researchers and hobbyists to easily automate and accelerate there deep leaning training with their own Kubernetes GPU cluster

  •        30

This guide should help fellow researchers and hobbyists to easily automate and accelerate there deep leaning training with their own Kubernetes GPU cluster. Therefore I will explain how to easily setup a GPU cluster on multiple Ubuntu 16.04 bare metal servers and provide some useful scripts and .yaml files that do the entire setup for you. By the way: If you need a Kubernetes GPU-cluster for other reasons, this guide might be helpful to you as well.

https://github.com/Langhalsdino/Kubernetes-GPU-Guide

Tags
Implementation
License
Platform

   




Related Projects

guide - Kubernetes clusters for the hobbyist.

  •    

The tinkerers of today are the leaders of tomorrow. This guide answers the question of how to setup and operate a fully functional, secure Kubernetes cluster on a cloud provider such as Hetzner Cloud, DigitalOcean or Scaleway. It explains how to overcome the lack of external ingress controllers, fully isolated secure private networking and persistent distributed block storage.

kubernetes-vagrant-centos-cluster - Setting up a distributed Kubernetes cluster along with Istio service mesh locally with Vagrant and VirtualBox

  •    Shell

Setting up a Kubernetes cluster and istio service mesh with vagrantfile which consists of 1 master(also as node) and 3 nodes. You don't have to create complicated CA files or configuration. Because I want to setup the etcd, apiserver, controller and scheduler without docker container.

kube-cluster-osx - Local development multi node Kubernetes Cluster for macOS made very simple

  •    Shell

Kube-Cluster for macOS is a status bar app which allows in an easy way to bootstrap and control multi-node (master+ two nodes) Kubernetes cluster on three CoreOS VMs. It leverages macOS native Hypervisor virtualisation framework of using corectl command line tool, so there are no needs to use VirtualBox or any other virtualisation software anymore.

kubernetes-cluster-federation - Kubernetes cluster federation tutorial

  •    

This tutorial will walk you through setting up a Kubernetes cluster federation composed of four Kubernetes clusters across multiple GCP regions.This guide is not for people looking for a fully automated command to bring up a Kubernetes cluster federation. If that's you then check out Setting up Cluster Federation with Kubefed.

kubeadm-ha - Kubernetes high availiability deploy based on kubeadm (for v1

  •    Smarty

kube-apiserver: exposes the Kubernetes API. It is the front-end for the Kubernetes control plane. It is designed to scale horizontally – that is, it scales by deploying more instances. etcd: is used as Kubernetes’ backing store. All cluster data is stored here. Always have a backup plan for etcd’s data for your Kubernetes cluster. kube-scheduler: watches newly created pods that have no node assigned, and selects a node for them to run on. kube-controller-manager: runs controllers, which are the background threads that handle routine tasks in the cluster. Logically, each controller is a separate process, but to reduce complexity, they are all compiled into a single binary and run in a single process. kubelet: is the primary node agent. It watches for pods that have been assigned to its node (either by apiserver or via local configuration file) kube-proxy: enables the Kubernetes service abstraction by maintaining network rules on the host and performing connection forwarding. keepalived cluster config a virtual IP address (192.168.20.10), this virtual IP address point to k8s-master01, k8s-master02, k8s-master03. nginx service as the load balancer of k8s-master01, k8s-master02, k8s-master03's apiserver. The other nodes kubernetes services connect the keepalived virtual ip address (192.168.20.10) and nginx exposed port (16443) to communicate with the master cluster's apiservers.


sonobuoy - Heptio Sonobuoy is a diagnostic tool that makes it easier to understand the state of a Kubernetes cluster by running a set of Kubernetes conformance tests in an accessible and non-destructive manner

  •    Go

Heptio Sonobuoy is a diagnostic tool that makes it easier to understand the state of a Kubernetes cluster by running a set of Kubernetes conformance tests in an accessible and non-destructive manner. It is a customizable, extendable, and cluster-agnostic way to generate clear, informative reports about your cluster. Sonobuoy supports Kubernetes versions 1.9 and later.

kubernetes-the-hard-way - Bootstrap Kubernetes the hard way on Google Cloud Platform. No scripts.

  •    

This tutorial walks you through setting up Kubernetes the hard way. This guide is not for people looking for a fully automated command to bring up a Kubernetes cluster. If that's you then check out Google Container Engine, or the Getting Started Guides.Kubernetes The Hard Way is optimized for learning, which means taking the long route to ensure you understand each task required to bootstrap a Kubernetes cluster.

pipeline - A step by step guide on creating build and deployment pipelines for Kubernetes.

  •    

The pipeline tutorial walks you through creating an end-to-end deployment pipeline using Cloud Container Builder, GitHub, and multiple Kubernetes clusters. This tutorial will demonstrate how to propagate a Kubernetes deployment through multiple environments, each backed by a dedicated Kubernetes cluster, using a collection of Kubernetes manifest files across a set of GitHub repositories representing each environment.

k3os - Purpose built OS for Kubernetes, fully managed by Kubernetes.

  •    Go

k3OS is a Linux distribution designed to remove as much as possible OS maintenance in a Kubernetes cluster. It is specifically designed to only have what is need to run k3s. Additionally the OS is designed to be managed by kubectl once a cluster is bootstrapped. Nodes only need to join a cluster and then all aspects of the OS can be managed from Kubernetes. Both k3OS and k3s upgrades are handled by the k3OS operator. Download the ISO from the latest release and run in VMware, VirtualBox, or KVM. The server will automatically start a single node Kubernetes cluster. Log in with the user rancher and run kubectl. This is a "live install" running from the ISO media and changes will not persist after reboot.

gardener - Kubernetes API server extension and controller manager managing the full lifecycle of conformant Kubernetes clusters (Shoots) as a service on AWS, Azure, GCP, and OpenStack

  •    Go

The Gardener implements the automated management and operation of Kubernetes clusters as a service and aims to support that service on multiple Cloud providers (AWS, GCP, Azure, OpenStack). Its main principle is to use Kubernetes itself as base for its tasks. In essence, the Gardener is an extension API server along with a bundle of Kubernetes controllers which introduces new API objects in an existing Kubernetes cluster (which is called Garden cluster) in order to use them for the management of further Kubernetes clusters (which are called Shoot clusters). To do that reliably and to offer a certain quality of service, it requires to control the main components of a Kubernetes cluster (etcd, API server, controller manager, scheduler). These so-called control plane components are hosted in Kubernetes clusters themselves (which are called Seed clusters).

katlas - A distributed graph-based platform to automatically collect, discover, explore and relate multi-cluster kubernetes resources and metadata

  •    Go

K-Atlas (pronounced Cutlass), is a distributed graph based platform to automatically collect, discover, explore and relate multi-cluster Kubernetes resources and metadata. K-Atlas's rich query language allows for simple and efficient exploration and extensibility. It addresses following problems in a large scale enterprise environment of Kubernetes.

Kops - The easiest way to get a production grade Kubernetes cluster up and running

  •    Go

The easiest way to get a production grade Kubernetes cluster up and running. It helps you create, destroy, upgrade and maintain production-grade, highly available, Kubernetes clusters from the command line. AWS (Amazon Web Services) is currently officially supported, with GCE and VMware vSphere in alpha and other platforms planned.

tack - Terraform module for creating Kubernetes cluster running on Container Linux by CoreOS in an AWS VPC

  •    HCL

Opinionated Terraform module for creating a Highly Available Kubernetes cluster running on Container Linux by CoreOS (any channel) in an AWS Virtual Private Cloud VPC. With prerequisites installed make all will simply spin up a default cluster; and, since it is based on Terraform, customization is much easier than CloudFormation.The default configuration includes Kubernetes add-ons: DNS, Dashboard and UI.

Telepresence - Local development against a remote Kubernetes or OpenShift cluster

  •    Python

Telepresence substitutes a two-way network proxy for your normal pod running in the Kubernetes cluster. This pod proxies data from your Kubernetes environment (e.g., TCP connections, environment variables, volumes) to the local process. The local process has its networking transparently overridden so that DNS calls and TCP connections are routed through the proxy to the remote Kubernetes cluster.

zero-to-jupyterhub-k8s - Resources for deploying JupyterHub to a Kubernetes Cluster

  •    Python

This is under active development and subject to change. This repo contains resources, such as Helm charts and the Zero to JupyterHub Guide, which help you to deploy JupyterHub on Kubernetes.

kubernetes-vagrant-coreos-cluster - Kubernetes cluster (for testing purposes) made easy with Vagrant and CoreOS

  •    Shell

Turnkey Kubernetes cluster setup with Vagrant 2.1.1+ and CoreOS. If you're lazy, or in a hurry, jump to the TL;DR section.

Skuber - A Scala Kubernetes client library

  •    Scala

Skuber is a Scala client library for Kubernetes. It provides a fully featured, high-level and strongly typed Scala API for managing Kubernetes cluster resources (such as Pods, Services, Deployments, ReplicaSets, Ingresses etc.) via the Kubernetes REST API server. See the programming guide for more details.

aws-iam-authenticator - A tool to use AWS IAM credentials to authenticate to a Kubernetes cluster

  •    Go

A tool to use AWS IAM credentials to authenticate to a Kubernetes cluster. The initial work on this tool was driven by Heptio. The project recieves contributions from multiple community engineers and is currently maintained by Heptio and Amazon EKS OSS Engineers. If you are an administrator running a Kubernetes cluster on AWS, you already need to manage AWS IAM credentials to provision and update the cluster. By using AWS IAM Authenticator for Kubernetes, you avoid having to manage a separate credential for Kubernetes access. AWS IAM also provides a number of nice properties such as an out of band audit trail (via CloudTrail) and 2FA/MFA enforcement.

kubeadm-dind-cluster - A Kubernetes multi-node test cluster based on kubeadm

  •    Shell

A Kubernetes multi-node cluster for developer of Kubernetes and projects that extend Kubernetes. Based on kubeadm and DIND (Docker in Docker). Supports both local workflows and workflows utilizing powerful remote machines/cloud instances for building Kubernetes, starting test clusters and running e2e tests.

kubernetes-elasticsearch-cluster - Elasticsearch cluster on top of Kubernetes made easy.

  •    

Elasticsearch (6.2.4) cluster on top of Kubernetes made easy. Given this, I'm going to demonstrate how to provision a production grade scenario consisting of 3 master, 2 client and 2 data nodes.