multer-s3 - multer storage engine for amazon s3

  •        212

Streaming multer storage engine for AWS S3. This project is mostly an integration piece for existing code samples from Multer's storage engine documentation with s3fs as the substitution piece for file system. Existing solutions I found required buffering the multipart uploads into the actual filesystem which is difficult to scale.

https://github.com/badunk/multer-s3#readme

Dependencies:

file-type : ^3.3.0
run-parallel : ^1.1.6

Tags
Implementation
License
Platform

   




Related Projects

S3 - Node.js implementation of a server handling the Amazon S3 protocol

  •    Javascript

CloudServer (formerly S3 Server) is an open-source Amazon S3-compatible object storage server that is part of Zenko, Scality’s Open Source Multi-Cloud Data Controller.CloudServer provides a single AWS S3 API interface to access multiple backend data storage both on-premise or public in the cloud.

serverless-image-resizing - CloudFormation template to resize images on-the-fly using Amazon API Gateway, AWS Lambda, and Amazon S3

  •    Javascript

Resizes images on the fly using Amazon S3, AWS Lambda, and Amazon API Gateway. Using a conventional URL structure and S3 static website hosting with redirection rules, requests for resized images are redirected to a Lambda function via API Gateway which will resize the image, upload it to S3, and redirect the requestor to the resized image. The next request for the resized image will be served from S3 directly.Use the Amazon Linux Docker container image to build the package using your local system. This repo includes Makefile that will download Amazon Linux, install Node.js and developer tools, and build the extensions using Docker. Run make all.

lambda-refarch-imagerecognition - The Image Recognition and Processing Backend reference architecture demonstrates how to use AWS Step Functions to orchestrate a serverless processing workflow using AWS Lambda, Amazon S3, Amazon DynamoDB and Amazon Rekognition

  •    Javascript

The Image Recognition and Processing Backend demonstrates how to use [AWS Step Functions] (https://aws.amazon.com/step-functions/) to orchestrate a serverless processing workflow using AWS Lambda, Amazon S3, Amazon DynamoDB and Amazon Rekognition. This workflow processes photos uploaded to Amazon S3 and extracts metadata from the image such as geolocation, size/format, time, etc. It then uses image recognition to tag objects in the photo. In parallel, it also produces a thumbnail of the photo.This repository contains sample code for all the Lambda functions depicted in the diagram below as well as an AWS CloudFormation template for creating the functions and related resources. There is also a test web app that you can run locally to interact with the backend.

lambda-refarch-imagerecognition - The Image Recognition and Processing Backend reference architecture demonstrates how to use AWS Step Functions to orchestrate a serverless processing workflow using AWS Lambda, Amazon S3, Amazon DynamoDB and Amazon Rekognition

  •    Javascript

The Image Recognition and Processing Backend demonstrates how to use AWS Step Functions to orchestrate a serverless processing workflow using AWS Lambda, Amazon S3, Amazon DynamoDB and Amazon Rekognition. This workflow processes photos uploaded to Amazon S3 and extracts metadata from the image such as geolocation, size/format, time, etc. It then uses image recognition to tag objects in the photo. In parallel, it also produces a thumbnail of the photo. This repository contains sample code for all the Lambda functions depicted in the diagram below as well as an AWS CloudFormation template for creating the functions and related resources. There is also a test web app that you can run locally to interact with the backend.

aws-cognito-angular-quickstart - An AngularV4-based QuickStart web app utilizing Amazon Cognito, S3, and DynamoDB (serverless architecture)

  •    TypeScript

This sample application can be deployed to either Elastic Beanstalk or S3. S3 will host this application as a static site while Elastic Beanstalk gives you the capability of adding backend operations to the application.createResources.sh requires your aws cli to be configured for JSON output.


s3-upload-stream - A Node.js module for streaming data to Amazon S3 via the multipart upload API

  •    Javascript

A pipeable write stream which uploads to Amazon S3 using the multipart file upload API. NOTE: This module is deprecated after the 2.1.0 release of the AWS SDK on Dec 9, 2014, which added S3.upload(). I highly recommend switching away from this module and using the official method supported by AWS.

s3-parallel-put - Parallel uploads to Amazon AWS S3

  •    Python

s3-parallel-put speeds the uploading of many small keys to Amazon AWS S3 by executing multiple PUTs in parallel. The program reads your credentials from the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

multer - Node.js middleware for handling `multipart/form-data`.

  •    Javascript

Multer is a node.js middleware for handling multipart/form-data, which is primarily used for uploading files. It is written on top of busboy for maximum efficiency. NOTE: Multer will not process any form which is not multipart (multipart/form-data).

S3 - A simple helper for easily uploading files to Amazon S3 from Meteor

  •    CoffeeScript

S3 provides a simple way for uploading files to the Amazon S3 service with a progress bar. This is useful for uploading images and files that you want accesible to the public. S3 is built on Knox and AWS-SDK. Both modules are made available on the server after installing this package. S3 now uploads directly from the client to Amazon. Client files will not touch your server.

node-s3-client - high level amazon s3 client for node.js

  •    Javascript

See also the companion CLI tool which is meant to be a drop-in replacement for s3cmd: s3-cli.This contains a reference to the aws-sdk module. It is a valid use case to use both this module and the lower level aws-sdk module in tandem.

aws-lambda-redshift-loader - Amazon Redshift Database Loader implemented in AWS Lambda

  •    Javascript

With this AWS Lambda function, it's never been easier to get file data into Amazon Redshift. You simply push files into a variety of locations on Amazon S3, and have them automatically loaded into your Amazon Redshift clusters.For automated delivery of streaming data to S3 and subsequently to Redshift, also consider using Amazon Kinesis Firehose.

aws-amplify-graphql - Sample using AWS Amplify and AWS AppSync together for user login and authorization when making GraphQL queries and mutations

  •    Javascript

Please submit issues to the appsync-sdk-js repository. This sample application shows how to use GraphQL to build an application that a user can login to the system, then upload and download photos which are private to them. The sample is written in React and uses AWS AppSync, Amazon Cognito, Amazon DynamoDB and Amazon S3 as well as the AWS Mobile CLI.

cloudserver - Zenko CloudServer, an open-source Node

  •    Javascript

CloudServer (formerly S3 Server) is an open-source Amazon S3-compatible object storage server that is part of Zenko, Scality’s Open Source Multi-Cloud Data Controller. CloudServer provides a single AWS S3 API interface to access multiple backend data storage both on-premise or public in the cloud.

Minio - Open source object storage server compatible with Amazon S3 APIs

  •    Go

Minio is an object storage server, It is compatible with Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups and container / VM images. Size of an object can range from a few KBs to a maximum of 5TB. Minio server is light enough to be bundled with the application stack, similar to NodeJS, Redis and MySQL.

goofys - a high-performance, POSIX-ish Amazon S3 file system written in Go

  •    Go

Goofys allows you to mount an S3 bucket as a filey system. It's a Filey System instead of a File System because goofys strives for performance first and POSIX second. Particularly things that are difficult to support on S3 or would translate into more than one round-trip would either fail (random writes) or faked (no per-file permission). Goofys does not have a on disk data cache (checkout catfs), and consistency model is close-to-open.

aws-serverless-workshops - Code and walkthrough labs to set up serverless applications for Wild Rydes workshops

  •    Javascript

This repository contains a collection of workshops and other hands on content that will guide you through building various serverless applications using AWS Lambda, Amazon API Gateway, Amazon DynamoDB, AWS Step Functions, Amazon Kinesis, and other services.Web Application - This workshop shows you how to build a dynamic, serverless web application. You'll learn how to host static web resources with Amazon S3, how to use Amazon Cognito to manage users and authentication, and how to build a RESTful API for backend processing using Amazon API Gateway, AWS Lambda and Amazon DynamoDB.

aws-serverless-workshops - Code and walkthrough labs to set up serverless applications for Wild Rydes workshops

  •    Javascript

This repository contains a collection of workshops and other hands on content that will guide you through building various serverless applications using AWS Lambda, Amazon API Gateway, Amazon DynamoDB, AWS Step Functions, Amazon Kinesis, and other services. Web Application - This workshop shows you how to build a dynamic, serverless web application. You'll learn how to host static web resources with Amazon S3, how to use Amazon Cognito to manage users and authentication, and how to build a RESTful API for backend processing using Amazon API Gateway, AWS Lambda and Amazon DynamoDB.

aws-sdk-android - AWS SDK for Android. For more information, see our web site:

  •    Java

The Amazon Web Services SDK for Android provides Android APIs for building software on AWS’ cost-effective, scalable, and reliable infrastructure products. The AWS SDK for Android allows developers to code against APIs for all of Amazon's infrastructure web services (Amazon S3, Amazon EC2, Amazon SQS, Auto Scaling, etc).

knox - S3 Lib

  •    Javascript

Node Amazon S3 Client.More options are documented below for features like other endpoints or regions.

mc - Minio Client is a replacement for ls, cp, mkdir, diff and rsync commands for filesystems and object storage

  •    Go

Minio Client (mc) provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff, find etc. It supports filesystems and Amazon S3 compatible cloud storage service (AWS Signature v2 and v4).then use the mc config command.





We have large collection of open source products. Follow the tags from Tag Cloud >>


Open source products are scattered around the web. Please provide information about the open source projects you own / you use. Add Projects.