Pluggable foundation blocks for building loosely coupled distributed apps.Includes implementations in Redis, Azure, AWS and in memory (for development).
azure c-sharp lock message-bus storage redis metrics aws distributed-systems foundatio s3 jobs job-scheduler queue messaging statsd distributed-locks distributed logging abstractionsSee also the companion CLI tool which is meant to be a drop-in replacement for s3cmd: s3-cli.This contains a reference to the aws-sdk module. It is a valid use case to use both this module and the lower level aws-sdk module in tandem.
amazon s3 sync folder directory retry limit stream async parallel multipart sizeJavascript SDK for Qiniu Resource (Cloud) Storage API
cloud storage s3 qiniu web-serviceNode Amazon S3 Client.More options are documented below for features like other endpoints or regions.
aws amazon s3Bootstrap a React Native application on AWS. This starter automatically provisions a Serverless infrastructure with authentication, authorization, image storage, API access and database operations. It also includes user registration and MFA support. The sample use case is a "Pet Tracker" where after a user registers and logs in they can upload pictures of their pet to the system along with information like the birthday or breed.A companion blog post for this repository can be found in the AWS Mobile Blog: Announcing: React Native Starter Project with One-Click AWS Deployment and Serverless Infrastructure.
react-native aws serverless dynamodb cognito api-gateway lambda mobile-hub s3 aws-amplify aws-amplify-react-nativeA curated list of awesome AWS libraries, open source repos, guides, blogs, and other resources.Inspired by the awesome list.
aws cloud aws-sdk aws-cli cloudformation cloudwatch dynamodb elasticsearch kinesis lambda serverless machine-learning redshift s3 ec2 iam rds route53 ecs cloud-managementS3 is a WordPress plugin to store uploads on S3. S3-Uploads aims to be a lightweight "drop-in" for storing uploads on Amazon S3 instead of the local filesystem. It's focused on providing a highly robust S3 interface with no "bells and whistles", WP-Admin UI or much otherwise. It comes with some helpful WP-CLI commands for generating IAM users, listing files on S3 and Migrating your existing library to S3.
wordpress wordpress-plugin s3There are a few optional keyword arguments that are useful only for S3 access. These are both passed to boto.s3_connect() as keyword arguments. The S3 reader supports gzipped content, as long as the key is obviously a gzipped file (e.g. ends with ".gz").
s3 hdfs webhdfs boto streaming file streaming-data gzip-stream bz2Goofys allows you to mount an S3 bucket as a filey system. It's a Filey System instead of a File System because goofys strives for performance first and POSIX second. Particularly things that are difficult to support on S3 or would translate into more than one round-trip would either fail (random writes) or faked (no per-file permission). Goofys does not have a on disk data cache (checkout catfs), and consistency model is close-to-open.
s3-bucket s3 posix filesystem posix-ish aws-s3 cloud-storage fuse gcs google-cloud-storage openstack-swift fuse-filesystemArc is a flexible file upload library for Elixir with straightforward integrations for Amazon S3 and ImageMagick. Browse the readme below, or jump to a full example.
storage elixir attachment s3 s3-storageSee the home page for installation, usage, documentation, changelog and configuration walkthroughs. This is free software under the terms of MIT the license (check the COPYING file included in this package).
cloud-storage s3 openstack-swift google-drive azure-blob amazon-drive backblaze-b2 sftp ftp rclone hubic dropbox google-cloud-storage onedrive sync fuse-filesystem encryption webdavThanos is a set of components that can be composed into a highly available metric system with unlimited storage capacity. It can be added seamlessly on top of existing Prometheus deployments and leverages the Prometheus 2.0 storage format to cost-efficiently store historical metric data in any object storage while retaining fast query latencies. Additionally, it provides a global query view across all Prometheus installations and can merge data from Prometheus HA pairs on the fly.
prometheus google-cloud-storage high-availability prometheus-ha-pairs thanos s3 prometheus-setup metric-data storagePlease submit issues to the aws-amplify repository. Bootstrap a React Native application on AWS. This starter automatically provisions a Serverless infrastructure with authentication, authorization, image storage, API access and database operations. It also includes user registration and MFA support. The sample use case is a "Pet Tracker" where after a user registers and logs in they can upload pictures of their pet to the system along with information like the birthday or breed.
react-native aws serverless dynamodb cognito api-gateway lambda mobile-hub s3 aws-amplify aws-amplify-react-nativeWAL-G is an archival restoration tool for Postgres. WAL-G is the successor of WAL-E with a number of key differences. WAL-G uses LZ4, LZMA or Brotli compression, multiple processors and non-exclusive base backups for Postgres. More information on the design and implementation of WAL-G can be found on the Citus Data blog post "Introducing WAL-G by Citus: Faster Disaster Recovery for Postgres".
postgresql postgres backup archiving s3s3-lambda enables you to run lambda functions over a context of S3 objects. It has a stateless architecture with concurrency control, allowing you to process a large number of files very quickly. This is useful for quickly prototyping complex data jobs without an infrastructure like Hadoop or Spark. At Littlstar, we use s3-lambda for all sorts of data pipelining and analytics.
s3 lambda-functions concurrency nodeClaudia makes it easy to deploy Node.js projects to AWS Lambda and API Gateway. It automates all the error-prone deployment and configuration tasks, and sets everything up the way JavaScript developers expect out of the box. This means that you can get started with Lambda and API Gateway easily, and focus on solving important business problems instead of dealing with AWS deployment workflows. Check out this video to see how to create and deploy a microservice in under 5 minutes. With the help of Claudia builder projects, you can also use API Gateway as if it were a lightweight javascript web server, or create and deploy chat bots for various platforms easily.
aws-lambda aws serverless api-gateway nodejs claudia lambda apigateway s3 api microservicesFineUploader is also simple to use. In the simplest case, you only need to include one JavaScript file. There are absolutely no other required external dependencies. For more information, please see the documentation. If you'd like to help and keep this project strong and relevant, you have several options.
fineuploader web-development files vanilla-javascript vanilla-js standalone api file-upload javascript-library dropzone amazon aws azure chunk chunking cross-domain cross-site drag drop file file-input file-uploader input jquery-plugin multiple preview progress resume s3 selection upload widgetThis project is now in maintenance. No new features are planned. Backup is a system utility for Linux and Mac OS X, distributed as a RubyGem, that allows you to easily perform backup operations. It provides an elegant DSL in Ruby for modeling your backups. Backup has built-in support for various databases, storage protocols/services, syncers, compressors, encryptors and notifiers which you can mix and match. It was built with modularity, extensibility and simplicity in mind.
backup database s3 encryption slack mysql postgresql mongodbWAL-E is a program designed to perform continuous archiving of PostgreSQL WAL files and base backups. To correspond on using WAL-E or to collaborate on its development, do not hesitate to send mail to the mailing list at wal-e@googlegroups.com (archives and subscription settings). Github issues are also currently being used to track known problems, so please feel free to submit those.
backup backups postgresql postgres archiving s3 google-cloud-storage openstack-swift blob-store azure-blob replication recovery pitrCloudServer (formerly S3 Server) is an open-source Amazon S3-compatible object storage server that is part of Zenko, Scality’s Open Source Multi-Cloud Data Controller. CloudServer provides a single AWS S3 API interface to access multiple backend data storage both on-premise or public in the cloud.
aws-s3 docker multiple-backends storage cloud s3-storage cloud-storage cloud-native object-storage nodejs s3 server
We have large collection of open source products. Follow the tags from
Tag Cloud >>
Open source products are scattered around the web. Please provide information
about the open source projects you own / you use.
Add Projects.