Afero - A FileSystem Abstraction System for Go

  •        98

Afero is an filesystem framework providing a simple, uniform and universal API interacting with any filesystem, as an abstraction layer providing interfaces, types and methods. Afero has an exceptionally clean interface and simple design without needless constructors or initialization methods.Afero is also a library providing a base set of interoperable backend filesystems that make it easy to work with afero while retaining all the power and benefit of the os and ioutil packages.

Afero provides significant improvements over using the os package alone, most notably the ability to create mock and testing filesystems without relying on the disk.



Related Projects

goofys - a high-performance, POSIX-ish Amazon S3 file system written in Go

  •    Go

Goofys allows you to mount an S3 bucket as a filey system. It's a Filey System instead of a File System because goofys strives for performance first and POSIX second. Particularly things that are difficult to support on S3 or would translate into more than one round-trip would either fail (random writes) or faked (no per-file permission). Goofys does not have a on disk data cache (checkout catfs), and consistency model is close-to-open.

libfuse - The reference implementation of the Linux FUSE (Filesystem in Userspace) interface

  •    C

FUSE (Filesystem in Userspace) is an interface for userspace programs to export a filesystem to the Linux kernel. The FUSE project consists of two components: the fuse kernel module (maintained in the regular kernel repositories) and the libfuse userspace library (maintained in this repository). libfuse provides the reference implementation for communicating with the FUSE kernel module. A FUSE file system is typically implemented as a standalone application that links with libfuse. libfuse provides functions to mount the file system, unmount it, read requests from the kernel, and send responses back. libfuse offers two APIs: a "high-level", synchronous API, and a "low-level" asynchronous API. In both cases, incoming requests from the kernel are passed to the main program using callbacks. When using the high-level API, the callbacks may work with file names and paths instead of inodes, and processing of a request finishes when the callback function returns. When using the low-level API, the callbacks must work with inodes and responses must be sent explicitly using a separate set of API functions.

SeaweedFS - Simple and highly scalable distributed file system

  •    Go

SeaweedFS is a simple and highly scalable distributed file system. There are two objectives: to store billions of files! to serve the files fast! Instead of supporting full POSIX file system semantics, SeaweedFS choose to implement only a key~file mapping. Similar to the word "NoSQL", you can call it as "NoFS".

diskover - File system crawler, disk space usage, file search engine and file system analytics powered by Elasticsearch

  •    Python

diskover is an open source file system crawler and disk space usage software that uses Elasticsearch to index and manage data across heterogeneous storage systems. Using diskover, you are able to more effectively search and organize files and system administrators are able to manage storage infrastructure, efficiently provision storage, monitor and report on storage use, and effectively make decisions about new infrastructure purchases. As the amount of file data generated by business' continues to expand, the stress on expensive storage infrastructure, users and system administrators, and IT budgets continues to grow.

FSArchiver - Filesystem Archiver for Linux

  •    C

FSArchiver is a system tool that allows you to save the contents of a file-system to a compressed archive file. The file-system can be restored on a partition which has a different size and it can be restored on a different file-system. Unlike tar/dar, FSArchiver also creates the file-system when it extracts the data to partitions. Everything is checksummed in the archive in order to protect the data. If the archive is corrupt, you just loose the current file, not the whole archive.

lcfs - LCFS Graph driver for Docker

  •    C

tl;dr: Every time you build, pull or destroy a Docker container, you are using a storage driver. Current storage drivers like Device Mapper, AUFS, and Overlay2 implement container behavior using file systems designed to run a full OS. We are open-sourcing a file system that is purpose-built for the container lifecycle. We call this new file system Layer Cloning File System (LCFS). Because it is designed only for containers, it is up to 2.5x faster to build an image and up to almost 2x faster to pull an image. We're looking forward to working with the container community to improve and expand this new tool. Layer Cloning FileSystem (LCFS) is a new filesystem purpose-built to be a Docker storage driver. All Docker images are constructed of layers using storage drivers (graph drivers) like AUFS, OverlayFS, and Device Mapper. As a design principle, LCFS focuses on layers as the first-class citizen. The LCFS filesystem operates directly on top of block devices, as opposed to merging separate filesystems. Thereby, LCFS aims to directly manage at the container image’s layer level, eliminate the overhead of having a second filesystem that then is merged, and to optimize for density.

svfs - The Swift Virtual File System

  •    Go

SVFS is a Virtual File System over Openstack Swift built upon fuse. It is compatible with hubiC, OVH Public Cloud Storage and basically every endpoint using a standard Openstack Swift setup. It brings a layer of abstraction over object storage, making it as accessible and convenient as a filesystem, without being intrusive on the way your data is stored. This is not an official project of the Openstack community.

AlphaFS - Bringing Advanced Windows Filesystem Support to .NET


AlphaFS is a .NET library providing more complete Win32 file system functionality to the .NET platform than the standard System.IO classes. Features highlights: creating hardlinks, accessing hidden volumes, enumeration of volumes, transactional file operations and much more.

Cobra - A Commander for modern Go CLI interactions

  •    Go

Cobra is both a library for creating powerful modern CLI applications as well as a program to generate applications and command files. Cobra is a library providing a simple interface to create powerful modern CLI interfaces similar to git & go tools.

BrowserFS - BrowserFS is an in-browser filesystem that emulates the Node JS filesystem API and supports storing and retrieving files from various backends

  •    Javascript

BrowserFS is an in-browser file system that emulates the Node JS file system API and supports storing and retrieving files from various backends. BrowserFS also integrates nicely into the Emscripten file system. More backends can be defined by separate libraries, so long as they extend the BaseFileSystem class. Multiple backends can be active at once at different locations in the directory hierarchy.

BeeGFS - Parallel Cluster File System

  •    C

BeeGFS (formerly FhGFS) is the leading parallel cluster file system, developed with a strong focus on performance and designed for very easy installation and management. It transparently spreads user data across multiple servers. By increasing the number of servers and disks in the system, you can simply scale performance and capacity of the file system to the level that you need, seamlessly from small clusters up to enterprise-class systems with thousands of nodes.

Hugo - A Fast and Flexible Static Site Generator built with love in GoLang

  •    Go

A Fast and Flexible Static Site Generator built with love by spf13 and friends in Go.Hugo is a static HTML and CSS website generator written in Go. It is optimized for speed, ease of use, and configurability. Hugo takes a directory with content and templates and renders them into a full HTML website. Hugo is designed to work well for any kind of website including blogs, tumbles, and docs.

pyfilesystem2 - Python's Filesystem abstraction layer

  •    Python

Python's Filesystem abstraction layer. Think of PyFilesystem's FS objects as the next logical step to Python's file objects. In the same way that file objects abstract a single file, FS objects abstract an entire filesystem.

OrangeFS - Scale-out Network File System

  •    C

OrangeFS is a scale-out network file system designed for use on high-end computing (HEC) systems that provides very high-performance access to multi-server-based disk storage, in parallel. The OrangeFS server and client are user-level code, making them very easy to install and manage. OrangeFS has optimized MPI-IO support for parallel and distributed applications, and it is leveraged in production installations and used as a research platform for distributed and parallel storage.

dbxfs - User-space file system for Dropbox

  •    Python

dbxfs has been tested on OpenBSD, Linux, and macOS but it should run on any POSIX system that provides a FUSE-compatible library or has the ability to mount SMB shares. Windows support is coming very soon. It runs on non-x86 architectures like ARM. It doesn't require a specific file system. It is written for Python 3.5+ and is licensed under the GPLv3.

realize - Go build system with file watcher, live reload and output streams

  •    Go

Automate the most recurring operations needed for development, define what you need only one time, integrate additional tools of third party, define custom cli commands and reload projects at each file change without stop to write code.Various operations can be programmed for each project, which can be executed at startup, at stop, and at each file change.

gocryptfs - Encrypted overlay filesystem written in Go.

  •    Go

An encrypted overlay filesystem written in Go. Official website: (markdown source). gocryptfs is built on top the excellent go-fuse FUSE library. This project was inspired by EncFS and strives to fix its security issues while providing good performance (benchmarks). For details on the security of gocryptfs see the Security design document.