mongo-backup-to-s3 - stream mongodb backups to S3, with ease

  •        385

Mongodb 3.2 now has similar utility for streaming backups.This module relies heavily on mongo-dump-stream so it can output the 'dump' to a stream, and so we don't have to use mongodump (which requires a child process).

https://github.com/bradoyler/mongo-backup-to-s3

Dependencies:

aws-sdk : ^2.2.26
memorystream : ^0.3.1
moment : ^2.10.6
mongo-dump-stream : ^0.3.3

Tags
Implementation
License
Platform

   




Related Projects

mgob - MongoDB dockerized backup agent. Runs schedule backups with retention, S3 & SFTP upload, notifications, instrumentation with Prometheus and more.

  •    Go

MGOB is a MongoDB backup automation tool built with golang. Its features include Schedule backups, Local backups retention, Upload to S3, Google Cloud and lot more. It is distributed as an Alpine Docker image.

backup - Easy full stack backup operations on UNIX-like systems.

  •    Ruby

This project is now in maintenance. No new features are planned. Backup is a system utility for Linux and Mac OS X, distributed as a RubyGem, that allows you to easily perform backup operations. It provides an elegant DSL in Ruby for modeling your backups. Backup has built-in support for various databases, storage protocols/services, syncers, compressors, encryptors and notifiers which you can mix and match. It was built with modularity, extensibility and simplicity in mind.

MySQL-backup-to-Amazon-S3 - Backup daily/weekly/monhtly all your MySQL databases to Amazon S3 via SH and s3cmd

  •    Shell

Amazon S3 can be an interestingly safe and cheap way to store your important data. Some of the most important data in the world is saved in... MySQL, and surely mine is quite important, so I needed such a script. If you have a 500mb database (that's 10 times larger than any small site), with the priciest plan, keeping 6 backups (two months, two weeks, two days) costs $0.42 a month ($0.14GB/month). With 99.999999999% durability and 99.99% availability. Uploads are free, downloads would happen only in case you actually need to retrieve the backup (which hopefully won't be needed, but first GB is free, and over that $0.12/GB).

backup-manager - Database backup manager for dumping to and restoring databases from S3, Dropbox, FTP, SFTP, and Rackspace Cloud

  •    PHP

This package provides a framework-agnostic database backup manager for dumping to and restoring databases from S3, Dropbox, FTP, SFTP, and Rackspace Cloud. Watch a video tour showing the Laravel driver in action to give you an idea what is possible.

laravel-backup - Backup and restore database support for Laravel 4 applications

  •    PHP

You can use the --keep-only-s3 option if you don't want to keep a local copy of the SQL dump. Uses the aws/aws-sdk-php-laravel package which needs to be configured.


SQL Server S3 Backup

  •    

SQL Server command line backup tool that will perform full, differential and transaction log backups, zip them up and then upload to an Amazon S3 bucket.

pgbackrest - Reliable PostgreSQL Backup & Restore

  •    C

pgBackRest aims to be a simple, reliable backup and restore system that can seamlessly scale up to the largest databases and workloads. Instead of relying on traditional backup tools like tar and rsync, pgBackRest implements all backup features internally and uses a custom protocol for communicating with remote systems. Removing reliance on tar and rsync allows for better solutions to database-specific backup challenges. The custom remote protocol allows for more flexibility and limits the types of connections that are required to perform a backup which increases security.

Priam - Co-Process for backup/recovery, Token Management, and Centralized Configuration management for Cassandra

  •    Java

Priam is a process/tool that runs alongside Apache Cassandra to automate Backup and recovery, Token management, Seed discovery, Configuration, Support AWS environment. It supports Support multi-region Cassandra deployment in AWS via public IP, Backup throttling, Uses Snappy compression to compress backup data on the fly, Backup SSTables from local ephemeral disks to S3 and lot more.

safe - simple backup for mysql, posgresql, svn and files to s3 or local filesystem

  •    Ruby

simple backup for mysql, posgresql, svn and files to s3 or local filesystem

safe - simple backup for mysql, posgresql, svn and files to s3 or local filesystem

  •    Ruby

simple backup for mysql, posgresql, svn and files to s3 or local filesystem

s3cmd - Official s3cmd repo -- Command line tool for managing Amazon S3 and CloudFront services

  •    Python

S3cmd requires Python 2.6 or newer. Python 3+ is also supported starting with S3cmd version 2. S3cmd (s3cmd) is a free command line tool and client for uploading, retrieving and managing data in Amazon S3 and other cloud storage service providers that use the S3 protocol, such as Google Cloud Storage or DreamHost DreamObjects. It is best suited for power users who are familiar with command line programs. It is also ideal for batch scripts and automated backup to S3, triggered from cron, etc.

Barman - Backup and Recovery manager for PostgreSQL

  •    Python

Barman (Backup and Recovery Manager) is an open source administration tool for disaster recovery of PostgreSQL servers . It allows your organisation to perform remote backups of multiple servers in business critical environments and to help DBAs during the recovery phase. Its features include backup catalogues, incremental backup, retention policies, remote backup and recovery, archiving and compression of WAL files and backups.

smart_open - Utils for streaming large files (S3, HDFS, gzip, bz2...)

  •    Python

There are a few optional keyword arguments that are useful only for S3 access. These are both passed to boto.s3_connect() as keyword arguments. The S3 reader supports gzipped content, as long as the key is obviously a gzipped file (e.g. ends with ".gz").

dt-s3-backup - Bash script for automated backups with duplicity and Amazon's S3 online storage

  •    Shell

Bash script for automated backups with duplicity and Amazon's S3 online storage

s3sync - Tool belt for managing your S3 buckets

  •    Ruby

I needed to backup some stuff once when I was in the woods. Unfortunately, I didn't find anything easy but elegant enough to sync my stuff with Amazon S3. This project started as a fork of the original s3sync command that had its last release in 2008. After a while it became a complete rewrite which might be considered good in a lot of cases, however, it also entails losing the maturity that the old code used to have.

laravel-backup - A package to backup your Laravel app

  •    PHP

This Laravel package creates a backup of your application. The backup is a zipfile that contains all files in the directories you specify along with a dump of your database. The backup can be stored on any of the filesystems you have configured in Laravel 5. Feeling paranoid about backups? No problem! You can backup your application to multiple filesystems at once.

Amanda - Network Backup Solution

  •    C

AMANDA, the Advanced Maryland Automatic Network Disk Archiver, is a backup system that allows the administrator of a LAN to set up a single master backup server to back up multiple hosts to a single large capacity tape or disk drive. Amanda uses native tools (such as GNUtar, dump) for backup and can back up a large number of workstations running multiple versions of Unix/Mac OS X/Linux/Windows.

C# Library and Code for Amazon S3

  •    

An advanced C# library for interfacing with the Amazon S3 system. Among its powerful features are: - Full support for data streaming. No need to load data into memory before sending to S3. - Data encryption. - Thread safety and live statistics. Perform multiple simultaneous up...

s3-upload-stream - A Node.js module for streaming data to Amazon S3 via the multipart upload API

  •    Javascript

A pipeable write stream which uploads to Amazon S3 using the multipart file upload API. NOTE: This module is deprecated after the 2.1.0 release of the AWS SDK on Dec 9, 2014, which added S3.upload(). I highly recommend switching away from this module and using the official method supported by AWS.

flexbackup

  •    Perl

Flexible backup script. Easy to configure, uses dump/afio/tar/cpio/star/etc. backend. Backup levels as in dump. Configurable backup sets. Writes to tapes, on-disk archive files, or on-disk directory trees. rpm-mode for only changed/unowned files.






We have large collection of open source products. Follow the tags from Tag Cloud >>


Open source products are scattered around the web. Please provide information about the open source projects you own / you use. Add Projects.