kvdo - A pair of kernel modules which provide pools of deduplicated and/or compressed block storage.

  •        55

A pair of kernel modules which provide pools of deduplicated and/or compressed block storage. VDO (which includes kvdo and vdo) is software that provides inline block-level deduplication, compression, and thin provisioning capabilities for primary storage. VDO installs within the Linux device mapper framework, where it takes ownership of existing physical block devices and remaps these to new, higher-level block devices with data-efficiency capabilities.

https://github.com/dm-vdo/kvdo

Tags
Implementation
License
Platform

   




Related Projects

SCEC VDO

  •    Java

3D visualization software that allows the user to display earthquakes, faults and other related features under the surface of the earth. SCEC VDO is also capable of exporting quality movies for educational and media purposes.

VDO

  •    Delphi

VDO

Sailfish OS - Mobile Operating System

  •    C++

Sailfish OS is a mobile operating system built on top of Mer project and Qt5. The UI is built with QML and QtQuick2 and Wayland. It is compatible with Android.

kernel-module-fun - Some fun things you can do with kernel modules (all "bad ideas")

  •    C

I didn't know at all how kernel modules worked. This is me learning how. This is all tested using the 3.5.0-18 kernel. hello-packet.c: logs every time your computer receives a packet. This one could easily be modified to drop packets 50% of the time.

fibratus - Tool for exploration and tracing of the Windows kernel

  •    Python

Fibratus is a tool which is able to capture the most of the Windows kernel activity - process/thread creation and termination, context switches, file system I/O, registry, network activity, DLL loading/unloading and much more. The kernel events can be easily streamed to a number of output sinks like AMQP message brokers, Elasticsearch clusters or standard output stream. You can use filaments (lightweight Python modules) to extend Fibratus with your own arsenal of tools and so leverage the power of the Python's ecosystem. Download the latest release (Windows installer). The changelog and older releases can be found here.


Borg - Deduplicating archiver with compression and authenticated encryption

  •    C

BorgBackup (short: Borg) is a deduplicating backup program. Optionally, it supports compression and authenticated encryption. The main goal of Borg is to provide an efficient and secure way to backup data. The data deduplication technique used makes Borg suitable for daily backups since only changes are stored. The authenticated encryption technique makes it suitable for backups to not fully trusted targets.

linux - Linux Kernel development - MMC, SD, UFS and other embedded storage related development

  •    C

Linux Kernel development - MMC, SD, UFS and other embedded storage related development

BadMEM patches/utils for Linux Kernel

  •    C

Production of a patch to make (partly) buggy memory modules work in a (vanilla)-Linux kernel; kernel patch and additional (user-space) utilities

Linux Kernel Module Programming Guide

  •    

The Linux Kernel Module Programming Guide (LKMPG or just mpg) is a guide that teaches the basics of writing modules and device drivers for the Linux Kernel.

rdedup - Data deduplication engine, supporting optional compression and public key encryption.

  •    Rust

See wiki for current project status. rdedup is a data deduplication engine and a backup software.

f-stack - F-Stack is an user space network development kit with high performance based on DPDK, FreeBSD TCP/IP stack and coroutine API

  •    C

With the rapid development of Network Interface Cards the poor performance of data packet processing with the Linux kernel has become the bottleneck in modern network systems. Yet, the increasing demands of the Internet's growth demand a higher performant network processing solution. Kernel bypass has emerged to catch more and more attention. There are various similar technologies such as: DPDK, NETMAP and PF_RING. The main idea of kernel bypass is that Linux is only used to deal with control flow; all data streams are processed in user space. Therefore, kernel bypass can avoid performance bottlenecks caused by kernel packet copying, thread scheduling, system calls, and interrupts. Furthermore, kernel bypass can achieve higher performance with multi-optimizing methods. Within various techniques, DPDK has been widely used because of it's more thorough isolation from kernel scheduling and active community support. To deal with the increasingly severe DDoS attacks the authorized DNS server of Tencent Cloud DNSPod switched from Gigabit Ethernet to 10-Gigabit at the end of 2012. We faced several options: one is to continue to use the original network stack in the Linux kernel, another is to use kernel bypass techniques. After several rounds of investigation; we finally chose to develop our next generation of DNS server based on DPDK. The reason is DPDK provides ultra-high performance and can be seamlessly extended to 40G, or even 100G NIC, in the future.

raspberry-pi-os - Learning operating system development using Linux kernel and Raspberry Pi

  •    C

This repository contains a step-by-step guide that teaches how to create a simple operating system (OS) kernel from scratch. I call this OS Raspberry Pi OS or just RPi OS. The RPi OS source code is largely based on Linux kernel, but the OS has very limited functionality and supports only Raspberry PI 3. Each lesson is designed in such a way that it first explains how some kernel feature is implemented in the RPi OS, and then it tries to demonstrate how the same functionality works in the Linux kernel. Each lesson has a corresponding folder in the src directory, which contains a snapshot of the OS source code at the time when the lesson had just been completed. This allows the introduction of new concepts gracefully and helps readers to follow the evolution of the RPi OS. Understanding this guide doesn't require any specific OS development skills.

Two Kernel Monte

  •    C

Two Kernel Monte is a Linux kernel module which allows Linux to load another kernel image into RAM and restart the machine from that kernel. The loader supports initial RAM disks and passing arbitrary kernel command line parameters to the new kernel.

apollo-kernel - Collections of Apollo Kernels

  •    Shell

The Apollo Kernel provides the necessary kernel level support to run Apollo software stack. In the first release, we add the most popular solution, Linux Kernel, under the linux directory. Apollo Linux Kernel is based on official Linux Kernel 4.4.32 with some modifications.

matchbox - Network boot and provision Container Linux clusters (e.g. etcd3, Kubernetes, more)

  •    Go

matchbox is a service that matches bare-metal machines (based on labels like MAC, UUID, etc.) to profiles that PXE boot and provision Container Linux clusters. Profiles specify the kernel/initrd, kernel arguments, iPXE config, GRUB config, Container Linux Config, or other configs a machine should use. Matchbox can be installed as a binary, RPM, container image, or deployed on a Kubernetes cluster and it provides an authenticated gRPC API for clients like Terraform.

libfuse - The reference implementation of the Linux FUSE (Filesystem in Userspace) interface

  •    C

FUSE (Filesystem in Userspace) is an interface for userspace programs to export a filesystem to the Linux kernel. The FUSE project consists of two components: the fuse kernel module (maintained in the regular kernel repositories) and the libfuse userspace library (maintained in this repository). libfuse provides the reference implementation for communicating with the FUSE kernel module. A FUSE file system is typically implemented as a standalone application that links with libfuse. libfuse provides functions to mount the file system, unmount it, read requests from the kernel, and send responses back. libfuse offers two APIs: a "high-level", synchronous API, and a "low-level" asynchronous API. In both cases, incoming requests from the kernel are passed to the main program using callbacks. When using the high-level API, the callbacks may work with file names and paths instead of inodes, and processing of a request finishes when the callback function returns. When using the low-level API, the callbacks must work with inodes and responses must be sent explicitly using a separate set of API functions.

firmware - This repository contains pre-compiled binaries of the current Raspberry Pi kernel and modules, userspace libraries, and bootloader/GPU firmware

  •    C

This repository contains pre-compiled binaries of the current Raspberry Pi kernel and modules, userspace libraries, and bootloader/GPU firmware.

Linux Kernel Architecture Book

  •    

Functional and design specifications documents for Linux 2.4 kernel subsystems, services and the modules. This is a collaborative work by engineers in industry and in the open source community, with open peer review.

Xtables-addons

  •    C

Xtables-addons is a package that obsoletes the old patch-o-matic repository for the Linux kernel and iptables. Instead of patching the kernel source, extensions are built as modules and thus allow extending kernels without recompilation.

linux-kernel-module-cheat - Run one command, get a QEMU or gem5 Buildroot BusyBox virtual machine built from source with several minimal Linux kernel 4

  •    C

Run one command, get a QEMU or gem5 Buildroot BusyBox virtual machine built from source with several minimal Linux kernel 4.16 module development example tutorials with GDB and KGDB step debugging and minimal educational hardware models. "Tested" in x86, ARM and MIPS guests, Ubuntu 18.04 host. This is the most native setup, and therefore the best one if you are on one of the supported Ubuntu: 16.04 or 18.04.