Displaying 1 to 20 from 24 results

goofys - a high-performance, POSIX-ish Amazon S3 file system written in Go

  •    Go

Goofys allows you to mount an S3 bucket as a filey system. It's a Filey System instead of a File System because goofys strives for performance first and POSIX second. Particularly things that are difficult to support on S3 or would translate into more than one round-trip would either fail (random writes) or faked (no per-file permission). Goofys does not have a on disk data cache (checkout catfs), and consistency model is close-to-open.

gcs-resumable-upload - Upload a file to Google Cloud Storage with built-in resumable behavior

  •    Javascript

If somewhere during the operation, you lose your connection to the internet or your tough-guy brother slammed your laptop shut when he saw what you were uploading, the next time you try to upload to that file, it will resume automatically from where you left off.If your upload was interrupted, next time you run the code, we ask the API how much data it has already, then simply dump all of the data coming through the pipe that it already has.




gcs-tools - GCS support for avro-tools and parquet-tools

  •    Java

Light weight wrapper that adds Google Cloud Storage (GCS) support to common Hadoop tools, including avro-tools, parquet-tools and proto-tools for Scio's Protobuf in Avro file, so that they can be used from regular workstations or laptops, outside of a Google Compute Engine (GCE) instance.It uses your existing OAuth2 credentials and allows authentication via a browser.

cloud-storage - Simple node js module for uploading and removing files from Google Cloud Storage

  •    Javascript

Simple wrapper for uploading and deleting files from Google Cloud Storage.


gcs-resumable-upload - Upload a file to Google Cloud Storage with built-in resumable behavior

  •    TypeScript

If somewhere during the operation, you lose your connection to the internet or your tough-guy brother slammed your laptop shut when he saw what you were uploading, the next time you try to upload to that file, it will resume automatically from where you left off. If your upload was interrupted, next time you run the code, we ask the API how much data it has already, then simply dump all of the data coming through the pipe that it already has.

jgscm - Jupyter support for Google Cloud Storage

  •    Python

Jupyter Google Storage Contents Manager allows working with Jupyter notebooks directly in Google Cloud Storage. It aims to be a complete drop-in replacement for the stock filesystem ContentsManager. Thus JGSCM is only compatible with a relatively modern IPython/Jupyter stack (version 4 and above). The root level of the virtual file system is the list of buckets, which are presented as directories. In turn, each bucket is presented as an ordinary folder where users can create files, subdirectories and notebooks. Besides, snapshots are completely supported too.

parse-server-gcs-adapter - parse-server adapter for Google Cloud Storage

  •    Javascript

GCLOUD_PROJECT and GOOGLE_APPLICATION_CREDENTIALS environment variables.

streamx - kafka-connect-s3 : Ingest data from Kafka to Object Stores(s3)

  •    Java

StreamX is a kafka-connect based connector to copy data from Kafka to Object Stores like Amazon s3, Google Cloud Storage and Azure Blob Store. It focusses on reliable and scalable data copying. It can write the data out in different formats (like parquet, so that it can readily be used by analytical tools) and also in different partitioning requirements. StreamX inherits rich set of features from kafka-connect-hdfs.

gcs-resumable-upload - Upload a file to Google Cloud Storage with built-in resumable behavior

  •    TypeScript

If somewhere during the operation, you lose your connection to the internet or your tough-guy brother slammed your laptop shut when he saw what you were uploading, the next time you try to upload to that file, it will resume automatically from where you left off. If your upload was interrupted, next time you run the code, we ask the API how much data it has already, then simply dump all of the data coming through the pipe that it already has.

storage - Go library providing common interface for working across multiple cloud storage backends

  •    Go

Go library providing a common interface for working across multiple storage backends. This code was originally part of the Helm project, ChartMuseum, but has since been released as a standalone package for others to use in their own projects.

gcs - Gluster Container Storage

  •    Ruby

As of now, one can already use Gluster as storage for containers by making use of different projects available in github repositories associated with Gluster (especially GlusterD1) & Heketi through gluster-kubernetes . The goal of the GCS initiative is to provide a new stack ( especially with GD2, gluster-csi-driver ) focused on easier integration, much more opinionated bases install, a better upgrade experience to deploy Gluster for container storage. We are primarily focused on integration with Kubernetes (k8s) through this initiative. The deploy/ directory contains instructions for installing GCS, either in a Vagrant-based test environment, or on your own cluster.

kubernetes-cloud-backup - Kubernetes Cloud Backup - backup your state to S3, Azure Storage, or Google Storage

  •    Shell

Kubernetes state backup solution, designed to be run as Kubernetes Job. Use the Helm chart and deploy it to Kubernetes as CronJob which will ensure cluster backups of Kubernetes resource definitions to your S3/GCS/Azure cloud storage.

kubernetes-nexus - Run Sonatype Nexus Repository Manager OSS on top of Kubernetes (GKE)

  •    Shell

Nexus Repository Manager OSS (3.13.0) on top of Kubernetes. The very first thing one must do after deploying Nexus is to log-in into the Nexus UI with the default credentials (admin:admin123 ) and proceed to change the password to something more secure. But one must not do it right away! One will do it after securing Nexus with HTTPS.

spark-on-k8s-gcp-examples - Example Spark applications that run on Kubernetes and access GCP products, e

  •    Java

This is not an official Google product. This repository is for examples showing how to access Google Cloud Platform products such as Google Cloud Storage (GCS), BigQuery, and Cloud PubSub as data sources or sinks in Spark on Kubernetes applications. Please refer to the README in each example directory for details on how to build and run the example.

fluent-plugin-gcs - Google Cloud Storage output plugin for Fluentd.

  •    Ruby

Google Cloud Storage output plugin for Fluentd. NOTE: fluent-plugin-gcs v0.4.0 is now beta.

fog-google - Fog for Google Cloud Platform

  •    Ruby

The main maintainers for the Google sections are @icco, @Temikus and @plribeiro3000. Please send pull requests to them. As of v1.0.0, fog-google includes google-api-client as a dependency, there is no need to include it separately anymore.






We have large collection of open source products. Follow the tags from Tag Cloud >>


Open source products are scattered around the web. Please provide information about the open source projects you own / you use. Add Projects.