Minio - Open source object storage server compatible with Amazon S3 APIs

We have collection of more than 1 Million open source products ranging from Enterprise product to small libraries in all platforms. We aggregate information from all open source repositories. Search and find the best for your needs. Check out projects section.

Minio - Open source object storage server compatible with Amazon S3 APIs

Minio is an object storage server, It is compatible with Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups and container / VM images. Size of an object can range from a few KBs to a maximum of 5TB. Minio server is light enough to be bundled with the application stack, similar to NodeJS, Redis and MySQL.

MinIO is different in that it was designed from its inception to be the standard in private cloud object storage. Because MinIO is purpose-built to serve only objects, a single-layer architecture achieves all of the necessary functionality without compromise. The result is a cloud-native object server that is simultaneously performant, scalable and lightweight.

While MinIO excels at traditional object storage use cases like secondary storage, disaster recovery and archiving, it is unique at overcoming the private cloud challenges associated with machine learning, analytics and cloud-native application workloads.

https://min.io/
https://github.com/minio/minio

Tags	aws-s3 storage cloud s3-storage cloud-storage object-storage cloud-native object-store file-storage blob
Implementation	Go
License	Apache
Platform	Windows MacOS Linux

Related Projects

S3 - Node.js implementation of a server handling the Amazon S3 protocol

Javascript

CloudServer (formerly S3 Server) is an open-source Amazon S3-compatible object storage server that is part of Zenko, Scality’s Open Source Multi-Cloud Data Controller.CloudServer provides a single AWS S3 API interface to access multiple backend data storage both on-premise or public in the cloud.

aws-s3 docker storage cloud s3-storage cloud-storage cloud-native object-storage nodejs aws-s3-alernative

mc - Minio Client is a replacement for ls, cp, mkdir, diff and rsync commands for filesystems and object storage

Minio Client (mc) provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff, find etc. It supports filesystems and Amazon S3 compatible cloud storage service (AWS Signature v2 and v4).then use the mc config command.

storage google-cloud-storage aws-s3 filesystem s3 tools coreutils

cloudserver - Zenko CloudServer, an open-source Node

Javascript

CloudServer (formerly S3 Server) is an open-source Amazon S3-compatible object storage server that is part of Zenko, Scality’s Open Source Multi-Cloud Data Controller. CloudServer provides a single AWS S3 API interface to access multiple backend data storage both on-premise or public in the cloud.

aws-s3 docker multiple-backends storage cloud s3-storage cloud-storage cloud-native object-storage nodejs s3 server

chartmuseum - Helm Chart Repository with support for Amazon S3, Google Cloud Storage, Microsoft Azure Blob Storage, Alibaba Cloud OSS Storage, and Openstack Object Storage

ChartMuseum is an open-source Helm Chart Repository written in Go (Golang), with support for cloud storage backends, including Google Cloud Storage, Amazon S3, Microsoft Azure Blob Storage, Alibaba Cloud OSS Storage and Openstack Object Storage. Works as a valid Helm Chart Repository, and also provides an API for uploading new chart packages to storage etc.

helm kubernetes chartmuseum

Rclone - rsync for cloud storage like Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Yandex Files

Rclone ("rsync for cloud storage") is a command line program to sync files and directories to and from different cloud storage providers like Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Yandex.

golang sync encryption dropbox amazon-drive ftp s3 google-cloud-storage onedrive google-drive sftp webdav cloud-storage openstack-swift rclone hubic fuse-filesystem azure-blob backblaze-b2 file-synchronization file-sync

Quickwit - Fast and cost-efficient distributed search engine for immutable data

Rust

Quickwit is a distributed search engine built from the ground up to offer cost-efficiency and high reliability. By mere mortals for mere mortals, Quickwit's architecture is as simple as possible. Quickwit is written in Rust and built on top of the mighty tantivy library. It designed to index big datasets straight from object storage like AWS S3 in a stateless manner.

search-engine distributed-systems searchengine full-text-search

OpenEBS - Containerized Storage for Containers

OpenEBS is containerized block storage written in Go for cloud native and other environments w/ per container (or pod) QoS SLAs, tiering and replica policies across AZs and environments, and predictable and scalable performance.

storage storage-container nomad persistent-storage docker pod devops k8s kubernetes ebs ebs-volumes block-storage containerization persistent-workloads cloud-native object-store

lakeFS - Git-like capabilities for your object storage

lakeFS is an open source layer that delivers resilience and manageability to object-storage based data lakes. With lakeFS you can build repeatable, atomic and versioned data lake operations - from complex ETL jobs to data science and analytics.

apache-spark aws-s3 google-cloud-storage data-engineering data-lake object-storage datalake hadoop-filesystem data-quality data-versioning azure-blob-storage apache-sparksql git-for-data lakefs datalakes

meteor-slingshot - Upload files directly to AWS S3, Google Cloud Storage and others in meteor

Javascript

Direct and secure file-uploads to AWS S3, Google Cloud Storage and others. There are many many packages out there that allow file uploads to S3, Google Cloud and other cloud storage services, but they usually rely on the meteor apps' server to relay the files to the cloud service, which puts the server under unnecessary load.

rclone - "rsync for cloud storage" - Google Drive, Amazon Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Cloudfiles, Google Cloud Storage, Yandex Files

See the home page for installation, usage, documentation, changelog and configuration walkthroughs. This is free software under the terms of MIT the license (check the COPYING file included in this package).

cloud-storage s3 openstack-swift google-drive azure-blob amazon-drive backblaze-b2 sftp ftp rclone hubic dropbox google-cloud-storage onedrive sync fuse-filesystem encryption webdav

Neon - Serverless Postgres, alternative to AWS Aurora Postgres

Rust

Neon is a serverless open source alternative to AWS Aurora Postgres. It separates storage and compute and substitutes PostgreSQL storage layer by redistributing data across a cluster of nodes. Neon separates storage and compute to make on demand scalability possible. Compute activates on an incoming connection and shuts down to save resources on inactivity.

database-as-a-service postgres-as-a-service postgres serverless cloud

Cloud Sync Service

This Windows Service lets you sync your files across other machines by using Cloud File Storage as gateway. Amazon S3 and Windows Azure supported.

azure azure-storage blob-storage cloud cloud-storage cloud-sync sql-server-backup

goofys - a high-performance, POSIX-ish Amazon S3 file system written in Go

Goofys allows you to mount an S3 bucket as a filey system. It's a Filey System instead of a File System because goofys strives for performance first and POSIX second. Particularly things that are difficult to support on S3 or would translate into more than one round-trip would either fail (random writes) or faked (no per-file permission). Goofys does not have a on disk data cache (checkout catfs), and consistency model is close-to-open.

s3-bucket s3 posix filesystem posix-ish aws-s3 cloud-storage fuse gcs google-cloud-storage openstack-swift fuse-filesystem

thanos - Highly available Prometheus setup with long term storage capabilities.

Thanos is a set of components that can be composed into a highly available metric system with unlimited storage capacity. It can be added seamlessly on top of existing Prometheus deployments and leverages the Prometheus 2.0 storage format to cost-efficiently store historical metric data in any object storage while retaining fast query latencies. Additionally, it provides a global query view across all Prometheus installations and can merge data from Prometheus HA pairs on the fly.

prometheus google-cloud-storage high-availability prometheus-ha-pairs thanos s3 prometheus-setup metric-data storage

operator - MinIO Operator creates/configures/manages MinIO clusters on Kubernetes

MinIO is a Kubernetes-native high performance object store with an S3-compatible API. The MinIO Kubernetes Operator supports deploying MinIO Tenants onto private and public cloud infrastructures ("Hybrid" Cloud). The MinIO Console provides a graphical user interface (GUI) for interacting with MinIO Tenants.

Perkeep - Personal storage system for life: a way of storing, syncing, sharing, modelling and backing up content

Perkeep (née Camlistore) is a set of open source formats, protocols, and software for modeling, storing, searching, sharing and synchronizing data in the post-PC era. Data may be files or objects, tweets or 5TB videos, and you can access it via a phone, browser or FUSE filesystem. Perkeep can store files like a traditional filesystem (think: “directories”, “files”, “filenames”), it’s specialized in storing higher-level objects, which can represent anything.

blob object-store cloud-storage cloud backup file-storage

Alluxio - Data orchestration for analytics and machine learning in the cloud

Java

Alluxio (formerly known as Tachyon) is a virtual distributed storage system. It bridges the gap between computation frameworks and storage systems, enabling computation applications to connect to numerous storage systems through a common interface.

distributed-storage big-data memory-speed hadoop spark virtual-file-system presto tensorflow storage object-store

OpenStack - Software for building Private and Public Clouds

Python

OpenStack is a cloud operating system that controls large pools of compute, storage, and networking resources throughout a datacenter, all managed through a dashboard that gives administrators control while empowering their users to provision resources through a web interface.

cloud cloud-computing datacenter iaas infrastructure-as-a-service cloud-management

Zenko - Multi-Cloud Data Controller

Javascript

Zenko is Scality’s Open Source Multi-Cloud Data Controller.Zenko's goal is to provide a unifed namespace, access API and search capabilities to data stored locally using Docker volumes or Scality RING or in public cloud storage services like Amazon S3, or Microsoft Azure Blob storage.

data-controller data-storage mult-cloud data-management

minio-go - Minio Client SDK for Go

The Minio Go Client SDK provides simple APIs to access any Amazon S3 compatible object storage.This quickstart guide will show you how to install the Minio client SDK, connect to Minio, and provide a walkthrough for a simple file uploader. For a complete list of APIs and examples, please take a look at the Go Client API Reference.

libraries minio client-sdk s3 cloud fileuploader aws-signature