398 best open source s3 projects.

We have collection of more than 1 Million open source products ranging from Enterprise product to small libraries in all platforms. We aggregate information from all open source repositories. Search and find the best for your needs. Check out projects section.

Displaying 1 to 20 from 398 results

Foundatio - Pluggable foundation blocks for building distributed apps.

CSharp

Pluggable foundation blocks for building loosely coupled distributed apps.Includes implementations in Redis, Azure, AWS and in memory (for development).

azure c-sharp lock message-bus storage redis metrics aws distributed-systems foundatio s3 jobs job-scheduler queue messaging statsd distributed-locks distributed logging abstractions

node-s3-client - high level amazon s3 client for node.js

Javascript

See also the companion CLI tool which is meant to be a drop-in replacement for s3cmd: s3-cli.This contains a reference to the aws-sdk module. It is a valid use case to use both this module and the lower level aws-sdk module in tandem.

amazon s3 sync folder directory retry limit stream async parallel multipart size

js-sdk - 基于七牛 API 开发的前端 JavaScript SDK

Javascript

Javascript SDK for Qiniu Resource (Cloud) Storage API

cloud storage s3 qiniu web-service

knox - S3 Lib

Javascript

Node Amazon S3 Client.More options are documented below for features like other endpoints or regions.

aws amazon s3

aws-mobile-react-native-starter - AWS Mobile React Native Starter App https://aws

Javascript

Bootstrap a React Native application on AWS. This starter automatically provisions a Serverless infrastructure with authentication, authorization, image storage, API access and database operations. It also includes user registration and MFA support. The sample use case is a "Pet Tracker" where after a user registers and logs in they can upload pictures of their pet to the system along with information like the birthday or breed.A companion blog post for this repository can be found in the AWS Mobile Blog: Announcing: React Native Starter Project with One-Click AWS Deployment and Serverless Infrastructure.

react-native aws serverless dynamodb cognito api-gateway lambda mobile-hub s3 aws-amplify aws-amplify-react-native

awesome-aws - A curated list of awesome Amazon Web Services (AWS) libraries, open source repos, guides, blogs, and other resources

Python

A curated list of awesome AWS libraries, open source repos, guides, blogs, and other resources.Inspired by the awesome list.

aws cloud aws-sdk aws-cli cloudformation cloudwatch dynamodb elasticsearch kinesis lambda serverless machine-learning redshift s3 ec2 iam rds route53 ecs cloud-management

S3-Uploads - The WordPress Plugin to Store Uploads on Amazon S3

S3 is a WordPress plugin to store uploads on S3. S3-Uploads aims to be a lightweight "drop-in" for storing uploads on Amazon S3 instead of the local filesystem. It's focused on providing a highly robust S3 interface with no "bells and whistles", WP-Admin UI or much otherwise. It comes with some helpful WP-CLI commands for generating IAM users, listing files on S3 and Migrating your existing library to S3.

wordpress wordpress-plugin s3

smart_open - Utils for streaming large files (S3, HDFS, gzip, bz2...)

Python

There are a few optional keyword arguments that are useful only for S3 access. These are both passed to boto.s3_connect() as keyword arguments. The S3 reader supports gzipped content, as long as the key is obviously a gzipped file (e.g. ends with ".gz").

s3 hdfs webhdfs boto streaming file streaming-data gzip-stream bz2

goofys - a high-performance, POSIX-ish Amazon S3 file system written in Go

Goofys allows you to mount an S3 bucket as a filey system. It's a Filey System instead of a File System because goofys strives for performance first and POSIX second. Particularly things that are difficult to support on S3 or would translate into more than one round-trip would either fail (random writes) or faked (no per-file permission). Goofys does not have a on disk data cache (checkout catfs), and consistency model is close-to-open.

s3-bucket s3 posix filesystem posix-ish aws-s3 cloud-storage fuse gcs google-cloud-storage openstack-swift fuse-filesystem

arc - :paperclip: Flexible file upload and attachment library for Elixir

Elixir

Arc is a flexible file upload library for Elixir with straightforward integrations for Amazon S3 and ImageMagick. Browse the readme below, or jump to a full example.

storage elixir attachment s3 s3-storage

rclone - "rsync for cloud storage" - Google Drive, Amazon Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Cloudfiles, Google Cloud Storage, Yandex Files

See the home page for installation, usage, documentation, changelog and configuration walkthroughs. This is free software under the terms of MIT the license (check the COPYING file included in this package).

cloud-storage s3 openstack-swift google-drive azure-blob amazon-drive backblaze-b2 sftp ftp rclone hubic dropbox google-cloud-storage onedrive sync fuse-filesystem encryption webdav

thanos - Highly available Prometheus setup with long term storage capabilities.

Thanos is a set of components that can be composed into a highly available metric system with unlimited storage capacity. It can be added seamlessly on top of existing Prometheus deployments and leverages the Prometheus 2.0 storage format to cost-efficiently store historical metric data in any object storage while retaining fast query latencies. Additionally, it provides a global query view across all Prometheus installations and can merge data from Prometheus HA pairs on the fly.

prometheus google-cloud-storage high-availability prometheus-ha-pairs thanos s3 prometheus-setup metric-data storage

aws-mobile-react-native-starter - AWS Mobile React Native Starter App https://aws

Javascript

Please submit issues to the aws-amplify repository. Bootstrap a React Native application on AWS. This starter automatically provisions a Serverless infrastructure with authentication, authorization, image storage, API access and database operations. It also includes user registration and MFA support. The sample use case is a "Pet Tracker" where after a user registers and logs in they can upload pictures of their pet to the system along with information like the birthday or breed.

react-native aws serverless dynamodb cognito api-gateway lambda mobile-hub s3 aws-amplify aws-amplify-react-native

wal-g - Archival and Restoration for Postgres

WAL-G is an archival restoration tool for Postgres. WAL-G is the successor of WAL-E with a number of key differences. WAL-G uses LZ4, LZMA or Brotli compression, multiple processors and non-exclusive base backups for Postgres. More information on the design and implementation of WAL-G can be found on the Citus Data blog post "Introducing WAL-G by Citus: Faster Disaster Recovery for Postgres".

postgresql postgres backup archiving s3

s3-lambda - Lambda functions over S3 objects with concurrency control (each, map, reduce, filter)

Javascript

s3-lambda enables you to run lambda functions over a context of S3 objects. It has a stateless architecture with concurrency control, allowing you to process a large number of files very quickly. This is useful for quickly prototyping complex data jobs without an infrastructure like Hadoop or Spark. At Littlstar, we use s3-lambda for all sorts of data pipelining and analytics.

s3 lambda-functions concurrency node

claudia - Deploy Node.js projects to AWS Lambda and API Gateway easily

Javascript

Claudia makes it easy to deploy Node.js projects to AWS Lambda and API Gateway. It automates all the error-prone deployment and configuration tasks, and sets everything up the way JavaScript developers expect out of the box. This means that you can get started with Lambda and API Gateway easily, and focus on solving important business problems instead of dealing with AWS deployment workflows. Check out this video to see how to create and deploy a microservice in under 5 minutes. With the help of Claudia builder projects, you can also use API Gateway as if it were a lightweight javascript web server, or create and deploy chat bots for various platforms easily.

aws-lambda aws serverless api-gateway nodejs claudia lambda apigateway s3 api microservices

fine-uploader - Multiple file upload plugin with image previews, drag and drop, progress bars

Javascript

FineUploader is also simple to use. In the simplest case, you only need to include one JavaScript file. There are absolutely no other required external dependencies. For more information, please see the documentation. If you'd like to help and keep this project strong and relevant, you have several options.

fineuploader web-development files vanilla-javascript vanilla-js standalone api file-upload javascript-library dropzone amazon aws azure chunk chunking cross-domain cross-site drag drop file file-input file-uploader input jquery-plugin multiple preview progress resume s3 selection upload widget

backup - Easy full stack backup operations on UNIX-like systems.

Ruby

This project is now in maintenance. No new features are planned. Backup is a system utility for Linux and Mac OS X, distributed as a RubyGem, that allows you to easily perform backup operations. It provides an elegant DSL in Ruby for modeling your backups. Backup has built-in support for various databases, storage protocols/services, syncers, compressors, encryptors and notifiers which you can mix and match. It was built with modularity, extensibility and simplicity in mind.

backup database s3 encryption slack mysql postgresql mongodb

wal-e - Continuous Archiving for Postgres

Python

WAL-E is a program designed to perform continuous archiving of PostgreSQL WAL files and base backups. To correspond on using WAL-E or to collaborate on its development, do not hesitate to send mail to the mailing list at wal-e@googlegroups.com (archives and subscription settings). Github issues are also currently being used to track known problems, so please feel free to submit those.

backup backups postgresql postgres archiving s3 google-cloud-storage openstack-swift blob-store azure-blob replication recovery pitr

cloudserver - Zenko CloudServer, an open-source Node

Javascript

CloudServer (formerly S3 Server) is an open-source Amazon S3-compatible object storage server that is part of Zenko, Scality’s Open Source Multi-Cloud Data Controller. CloudServer provides a single AWS S3 API interface to access multiple backend data storage both on-premise or public in the cloud.