cstore_fdw - Columnar store for analytics with Postgres, developed by Citus Data

  •        62

Cstore_fdw is an open source columnar store extension for PostgreSQL. Columnar stores provide notable benefits for analytics use cases where data is loaded in batches. Cstore_fdw’s columnar nature delivers performance by only reading relevant data from disk, and it may compress data 6x-10x to reduce space requirements for data archival. Cstore_fdw is developed by Citus Data and can be used in combination with Citus, a postgres extension that intelligently distributes your data and queries across many nodes so your database can scale and your queries are fast. If you have any questions about how Citus can help you scale or how to use Citus in combination with cstore_fdw, please let us know.

https://github.com/citusdata/cstore_fdw

Tags
Implementation
License
Platform

   




Related Projects

bcolz - A columnar data container that can be compressed.

  •    C

bcolz provides columnar, chunked data containers that can be compressed either in-memory and on-disk. Column storage allows for efficiently querying tables, as well as for cheap column addition and removal. It is based on NumPy, and uses it as the standard data container to communicate with bcolz objects, but it also comes with support for import/export facilities to/from HDF5/PyTables tables and pandas dataframes. bcolz objects are compressed by default not only for reducing memory/disk storage, but also to improve I/O speed. The compression process is carried out internally by Blosc, a high-performance, multithreaded meta-compressor that is optimized for binary data (although it works with text data just fine too).

ClickHouse - Columnar DBMS and Real Time Analytics

  •    C++

ClickHouse is an open source column-oriented database management system capable of real time generation of analytical data reports using SQL queries. It is Linearly Scalable, Blazing Fast, Highly Reliable, Fault Tolerant, Data compression, Real time query processing, Web analytics, Vectorized query execution, Local and distributed joins. It can process hundreds of millions to more than a billion rows and tens of gigabytes of data per single server per second.

zson - ZSON is a PostgreSQL extension for transparent JSONB compression

  •    C

ZSON is a PostgreSQL extension for transparent JSONB compression. Compression is based on a shared dictionary of strings most frequently used in specific JSONB documents (not only keys, but also values, array elements, etc). In some cases ZSON can save half of your disk space and give you about 10% more TPS. Memory is saved as well. See docs/benchmark.md. Everything depends on your data and workload, though. Don't believe any benchmarks, re-check everything on your data, configuration, hardware, workload and PostgreSQL version.

InfiniDB - Scale-up analytics database engine for data warehousing and business intelligence

  •    C++

InfiniDB Community Edition is a scale-up, column-oriented database for data warehousing, analytics, business intelligence and read-intensive applications. InfiniDB's data warehouse columnar engine is multi-terabyte capable and accessed via MySQL.

Pinot - A realtime distributed OLAP datastore

  •    Java

Pinot is a realtime distributed OLAP datastore, which is used at LinkedIn to deliver scalable real time analytics with low latency. It can ingest data from offline data sources (such as Hadoop and flat files) as well as online sources (such as Kafka). Pinot is designed to scale horizontally, so that it can scale to larger data sets and higher query rates as needed.


FiloDB - Distributed. Columnar. Versioned. Streaming. SQL.

  •    Scala

High-performance distributed analytical database + Spark SQL queries + built for streaming. Columnar, versioned layers of data wrapped in a yummy high-performance analytical database engine.

EventQL - The database for large-scale event analytics

  •    C++

EventQL is a distributed, column-oriented database built for large-scale event collection and analytics. It runs super-fast SQL and MapReduce queries. Its features include Automatic partitioning, Columnar storage, Standard SQL support, Scales to petabytes, Timeseries and relational data, Fast range scans and lot more.

mysql_fdw - PostgreSQL foreign data wrapper for MySQL

  •    C

This PostgreSQL extension implements a Foreign Data Wrapper (FDW) for MySQL. Please note that this version of mysql_fdw works with PostgreSQL and EDB Postgres Advanced Server 9.3, 9.4, 9.5, 9.6, 10 and 11.

pg_pathman - Partitioning tool for PostgreSQL

  •    C

The pg_pathman module provides optimized partitioning mechanism and functions to manage partitions. Take a look at our Wiki out there.

eventstore - CQRS event store using PostgreSQL for persistence

  •    Elixir

CQRS event store implemented in Elixir. Uses PostgreSQL as the underlying storage engine. Requires Elixir v1.5 and PostgreSQL v9.5 or newer.

Flyimg - Microservice to resize and crop images on the fly

  •    PHP

Image resizing, cropping and compression on the fly with the impressive MozJPEG compression algorithm. One Docker container to build your own Cloudinary-like service.You pass the image URL and a set of keys with options, like size or compression. Flyimg will fetch the image, convert it, store it, cache it and serve it. The next time the request comes, it will serve the cached version.

Infobright - The Database for Analytics

  •    C++

Infobright combines a columnar database with our Knowledge Grid architecture to deliver a self-managing, self-tuning database optimized for analytics. Infobright eliminates the need to create indexes, partition data, or do any manual tuning to achieve fast response for queries and reports.

amazon-redshift-utils - Amazon Redshift Utils contains utilities, scripts and view which are useful in a Redshift environment

  •    Python

Copyright 2014 Amazon.com, Inc. or its affiliates. All Rights Reserved.Amazon Redshift is a fast, fully managed, petabyte-scale data warehouse solution that uses columnar storage to minimise IO, provide high data compression rates, and offer fast performance. This GitHub provides a collection of scripts and utilities that will assist you in getting the best performance possible from Amazon Redshift.

MapD - The MapD Core database

  •    C++

MapD Core is an in-memory, column store, SQL relational database that was designed from the ground up to run on GPUs. MapD Core is the foundational element of a larger data exploration platform that emphasizes speed at scale. By taking advantage of the parallel processing power of the hardware, MapD Core can query billions of rows in milliseconds. Furthermore, by using the graphics pipelines of GPUs, MapD Core can render graphics directly from the server.

Barman - Backup and Recovery manager for PostgreSQL

  •    Python

Barman (Backup and Recovery Manager) is an open source administration tool for disaster recovery of PostgreSQL servers . It allows your organisation to perform remote backups of multiple servers in business critical environments and to help DBAs during the recovery phase. Its features include backup catalogues, incremental backup, retention policies, remote backup and recovery, archiving and compression of WAL files and backups.

Marten - Postgresql as a Document Database and Event Store for .Net Applications

  •    CSharp

The Marten library provides .NET developers with the ability to use the proven PostgreSQL database engine and its fantastic JSON support as a fully fledged document database. The Marten team believes that a document database has far reaching benefits for developer productivity over relational databases with or without an ORM tool.Marten also provides .NET developers with an ACID-compliant event store with user-defined projections against event streams.

pgbackrest - Reliable PostgreSQL Backup & Restore

  •    C

pgBackRest aims to be a simple, reliable backup and restore system that can seamlessly scale up to the largest databases and workloads. Instead of relying on traditional backup tools like tar and rsync, pgBackRest implements all backup features internally and uses a custom protocol for communicating with remote systems. Removing reliance on tar and rsync allows for better solutions to database-specific backup challenges. The custom remote protocol allows for more flexibility and limits the types of connections that are required to perform a backup which increases security.

Bebop - Simple discussion board / forum web application.

  •    Go

Bebop is a simple discussion board / forum web application.Create a new empty database (MySQL оr PostgreSQL) that will be used as a data store and a database user with all privileges granted on this database.

mobility - Pluggable Ruby translation framework

  •    Ruby

Mobility is a gem for storing and retrieving translations as attributes on a class. These translations could be the content of blog posts, captions on images, tags on bookmarks, or anything else you might want to store in different languages. For examples of what Mobility can do, see the Companies using Mobility section below. Storage of translations is handled by customizable "backends" which encapsulate different storage strategies. The default way to store translations is to put them all in a set of two shared tables, but many alternatives are also supported, including translatable columns and model translation tables, as well as database-specific storage solutions such as json/jsonb and Hstore (for PostgreSQL).

bedquilt-core - A JSON document store on PostgreSQL

  •    PLpgSQL

BedquiltDB is a A JSON document-store built on PostgreSQL. Release packages for BedquiltDB can be found on pgxn.