Rclone - rsync for cloud storage like Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Yandex Files

•    Go

---

Rclone ("rsync for cloud storage") is a command line program to sync files and directories to and from different cloud storage providers like Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Yandex.

golang sync encryption dropbox amazon-drive ftp s3 google-cloud-storage onedrive google-drive sftp webdav cloud-storage openstack-swift rclone hubic fuse-filesystem azure-blob backblaze-b2 file-synchronization file-sync

diskover - File system crawler, disk space usage, file search engine and file system analytics powered by Elasticsearch

•    Python

---

diskover is an open source file system crawler and disk space usage software that uses Elasticsearch to index and manage data across heterogeneous storage systems. Using diskover, you are able to more effectively search and organize files and system administrators are able to manage storage infrastructure, efficiently provision storage, monitor and report on storage use, and effectively make decisions about new infrastructure purchases. As the amount of file data generated by business' continues to expand, the stress on expensive storage infrastructure, users and system administrators, and IT budgets continues to grow.

elasticsearch crawler filesystem-visualization filesystem-analysis filesystem-indexer disk-space disk-usage storage-analytics storage filesystem file-indexing duplicatefilefinder metadata duplicate-files botnet file-tagging analytics aws-s3 tree-walker

elFinder - Open-source file manager for web, written in JavaScript using jQuery and jQuery UI

•    Javascript

---

elFinder is an open-source file manager for web, written in JavaScript using jQuery UI.

html5 filemanager integration jquery-ui ajax 3rd-party-connectors file-manager file-dialog

goofys - a high-performance, POSIX-ish Amazon S3 file system written in Go

•    Go

---

Goofys allows you to mount an S3 bucket as a filey system. It's a Filey System instead of a File System because goofys strives for performance first and POSIX second. Particularly things that are difficult to support on S3 or would translate into more than one round-trip would either fail (random writes) or faked (no per-file permission). Goofys does not have a on disk data cache (checkout catfs), and consistency model is close-to-open.

s3-bucket s3 posix filesystem posix-ish aws-s3 cloud-storage fuse gcs google-cloud-storage openstack-swift fuse-filesystem

rclone - "rsync for cloud storage" - Google Drive, Amazon Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Cloudfiles, Google Cloud Storage, Yandex Files

•    Go

---

See the home page for installation, usage, documentation, changelog and configuration walkthroughs. This is free software under the terms of MIT the license (check the COPYING file included in this package).

cloud-storage s3 openstack-swift google-drive azure-blob amazon-drive backblaze-b2 sftp ftp rclone hubic dropbox google-cloud-storage onedrive sync fuse-filesystem encryption webdav

Minio - Open source object storage server compatible with Amazon S3 APIs

•    Go

---

Minio is an object storage server, It is compatible with Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups and container / VM images. Size of an object can range from a few KBs to a maximum of 5TB. Minio server is light enough to be bundled with the application stack, similar to NodeJS, Redis and MySQL.

aws-s3 storage cloud s3-storage cloud-storage object-storage cloud-native object-store file-storage blob

multer-s3 - multer storage engine for amazon s3

•    Javascript

---

Streaming multer storage engine for AWS S3. This project is mostly an integration piece for existing code samples from Multer's storage engine documentation with s3fs as the substitution piece for file system. Existing solutions I found required buffering the multipart uploads into the actual filesystem which is difficult to scale.

multer s3 amazon aws

Meteor-CollectionFS - Reactive file manager for Meteor

•    Javascript

---

CollectionFS is a suite of Meteor packages that together provide a complete file management solution including uploading, downloading, storage, synchronization, manipulation, and copying. It supports several storage adapters for saving to the local filesystem, GridFS, or S3, and additional storage adapters can be created. Victor Leung wrote a great quick start guide for the most basic image uploading and displaying task.

Bebop - Simple discussion board / forum web application.

•    Go

---

Bebop is a simple discussion board / forum web application.Create a new empty database (MySQL оr PostgreSQL) that will be used as a data store and a database user with all privileges granted on this database.

forum discussion-board web-app rest-api vuejs

Flajaxian S3 Amazon Service Uploader

•    CSharp

---

Flajaxian S3 Amazon Service Uploader is adapter for the Flajaxian FileUploader .NET web control designed for asynchronous file upload to an Amazon Service of multiple files at the same time, without a page post back and with a progress bar indicating the current upload progres...

amazon s3 file-upload ajax dime flash

angular-filemanager - JavaScript File Manager Material Design Folder Explorer Navigator Browser Manager in AngularJS with CSS3 Responsive (with FTP in PHP / Java / Node)

•    Javascript

---

JavaScript File Manager Material Design Folder Explorer Navigator Browser Manager in AngularJS with CSS3 Responsive (with FTP in PHP / Java / Node)

angular file-explorer filemanager file-manager file-browser file-upload material-design angularjs angular-component

S3 tools

•    PHP

---

OpenSource tools to access Amazon S3 file storage. Subprojects: s3cmd - unix-like tools to manipulate stored files from the command line, s3browser - PHP interface for viewing stored files in a browser, s3fuse - driver to mount the S3 storage locally

megacmd - A command-line client for mega.co.nz storage service

•    Go

---

I'm (@ncw) doing light maintenance on megacmd, but mainly on the underlying go-mega library which I'm using to provide a mega backend for rclone. Mega (mega.co.nz) is an excellent free storage service which provides 50 GB of free storage space. It has a web based user interface to upload and download files. Megacmd is a command-line tool for performing file and directory transfer between local directories and mega service. Features of megacmd are much similar to s3cmd utility, which is used to perform file transfer to Amazon S3.

SnowFS - SnowFS - a fast, scalable version control file storage for graphic files :art:

•    TypeScript

---

SnowFS is a high-performance command-line application and node library for Windows, macOS and Linux with a focus on binary file versioning. It is made for the graphics industry and was initially developed for Snowtrack. Disclaimer: This project is in alpha state and actively developed. Do not use this yet in a production environment or without backups.

storage version-control version-manager version-control-system apfs binary-manager version-tracker graphic-files

Ceph - Distributed Object Store

•    C++

---

Ceph provides seamless access to objects using native language bindings or radosgw, a REST interface that’s compatible with applications written for S3 and Swift. Ceph’s RADOS Block Device (RBD) provides access to block device images that are striped and replicated across the entire storage cluster. Ceph provides a POSIX-compliant network file system that aims for high performance, large data storage, and maximum compatibility with legacy applications.

object-store block-device file-system distributed block filesystem distributed-filesystem distributed-storage

Google Desktop File Storage

•    VBNET

---

This is a gadget for the Google desktop utility on Windows that allows you to drag and drop files onto it for storage and retrieval. Files are stored in various backing store implementations, including Local File System, Amazon S3 etc.

meteor-slingshot - Upload files directly to AWS S3, Google Cloud Storage and others in meteor

•    Javascript

---

Direct and secure file-uploads to AWS S3, Google Cloud Storage and others. There are many many packages out there that allow file uploads to S3, Google Cloud and other cloud storage services, but they usually rely on the meteor apps' server to relay the files to the cloud service, which puts the server under unnecessary load.

lcfs - LCFS Graph driver for Docker

•    C

---

tl;dr: Every time you build, pull or destroy a Docker container, you are using a storage driver. Current storage drivers like Device Mapper, AUFS, and Overlay2 implement container behavior using file systems designed to run a full OS. We are open-sourcing a file system that is purpose-built for the container lifecycle. We call this new file system Layer Cloning File System (LCFS). Because it is designed only for containers, it is up to 2.5x faster to build an image and up to almost 2x faster to pull an image. We're looking forward to working with the container community to improve and expand this new tool. Layer Cloning FileSystem (LCFS) is a new filesystem purpose-built to be a Docker storage driver. All Docker images are constructed of layers using storage drivers (graph drivers) like AUFS, OverlayFS, and Device Mapper. As a design principle, LCFS focuses on layers as the first-class citizen. The LCFS filesystem operates directly on top of block devices, as opposed to merging separate filesystems. Thereby, LCFS aims to directly manage at the container image’s layer level, eliminate the overhead of having a second filesystem that then is merged, and to optimize for density.

S3 - Node.js implementation of a server handling the Amazon S3 protocol

•    Javascript

---

CloudServer (formerly S3 Server) is an open-source Amazon S3-compatible object storage server that is part of Zenko, Scality’s Open Source Multi-Cloud Data Controller.CloudServer provides a single AWS S3 API interface to access multiple backend data storage both on-premise or public in the cloud.

aws-s3 docker storage cloud s3-storage cloud-storage cloud-native object-storage nodejs aws-s3-alernative

Cloud Sync Service

•    

---

This Windows Service lets you sync your files across other machines by using Cloud File Storage as gateway. Amazon S3 and Windows Azure supported.

azure azure-storage blob-storage cloud cloud-storage cloud-sync sql-server-backup