clickhouse_sinker - Easily load data from kafka to ClickHouse with high performance

  •        316

clickhouse_sinker is a sinker program that consumes kafka message and import them to ClickHouse.

https://github.com/housepower/clickhouse_sinker

Tags
Implementation
License
Platform

   




Related Projects

cds - Data syncing in golang for ClickHouse.

  •    Go

Data syncing in golang for ClickHouse. Automatically synchronizing data from MySQL/MongoDB data source to ClickHouse cluster in real time.

clickhouse-driver - ClickHouse Python Driver with native interface support

  •    Python

ClickHouse Python Driver with native (TCP) interface support. Documentation is available at https://clickhouse-driver.readthedocs.io.

synch - Sync data from the other DB to ClickHouse(cluster)

  •    Python

Sync data from other DB to ClickHouse, current support postgres and mysql, and support full and increment ETL. synch will read default config from ./synch.yaml, or you can use synch -c specify config file.

ClickHouse - Column-oriented database management system that allows generating analytical data reports in real time

  •    C++

ClickHouse is a fast open-source OLAP database management system. It is column-oriented and allows to generate analytical reports using SQL queries in real-time. Large queries are parallelized using multiple cores, taking all the necessary resources available on the current server. It supports distributed processing, data can reside on different shards. Each shard can be a group of replicas used for fault tolerance. All shards are used to run a query in parallel, transparently for the user.


Tabix - SQL Editor & Open source simple business intelligence for Clickhouse.

  •    Javascript

Tabix is a SQL Editor & Open source simple business intelligence for Clickhouse. No need to install, it works from the browser. It provides support to Draw charts, Maps of the world, Metrics RealTime charts from system.metrics, Displays database and tables as tree and lot more.

ClickHouse - Columnar DBMS and Real Time Analytics

  •    C++

ClickHouse is an open source column-oriented database management system capable of real time generation of analytical data reports using SQL queries. It is Linearly Scalable, Blazing Fast, Highly Reliable, Fault Tolerant, Data compression, Real time query processing, Web analytics, Vectorized query execution, Local and distributed joins. It can process hundreds of millions to more than a billion rows and tens of gigabytes of data per single server per second.

generate-schema - Convert JSON Objects to MySQL Schema, JSON Schema, Mongoose Schema, Google BigQuery, or a Generic template for Swagger files, documentation, code generation, and more

  •    Javascript

Convert JSON Objects to MySQL Table Schema, JSON Schema, Mongoose Schema, ClickHouse Schema, Google BigQuery, or a Generic template for documentation, code generation, and more. Optionally, add -g to the above if you want the generate-schema command line executable.

MLCraft - Low-code business intelligence tool and a data science workflow

  •    Javascript

MLCraft is an open-source low-code business intelligence tool and a data science workflow. MLCraft was designed to query the data from several data warehouses and run machine learning experiments. Cube.js is used as a primary query layer and makes it suitable for handling trillions of data points. It is a full-stack data science platform that provides everything you need to build, manage and automate machine learning

Plausible Analytics - Simple, open-source, lightweight (< 1 KB) and privacy-friendly web analytics alternative to Google Analytics

  •    Elixir

Plausible Analytics is a simple, lightweight (< 1 KB), open-source and privacy-friendly alternative to Google Analytics. It doesn’t use cookies and is fully compliant with GDPR, CCPA and PECR. You can self-host Plausible or have us run it for you in the Cloud. It is easy to understand and it cuts through the noise. Check your site traffic and get all the essential insights on one page in one minute. There are no layers of menus, there is no need for you to build custom reports, custom dashboards or PowerPoint documents.

GrowthBook - The Open Source A/B Testing Platform

  •    Typescript

Growth Book is an open source experimentation platform designed for companies that want to bring A/B testing in-house, but don't want to build it themselves. It supports experimentation with deep code integration or using a visual front-end editor.

Kafka-Message-Server - Example application based on Apache Kafka framework to show it usage as distributed message server

  •    Java

Apache kafka is yet another precious gem from Apache Software Foundation. Kafka was originally developed at Linkedin and later on became a member of Apache project. Apache Kafka is a distributed publish-subscribe messaging system. Kafka differs from traditional messaging system as it is designed as distributed system, persists messages on disk and supports multiple subscribers. Kafka-Message-Server is an sample application for demonstrating kafka usage as message-server. Please follow the below instructions for productive use of the sample application.

kafka-websocket - Websocket server interface for Kafka distributed message broker

  •    Java

kafka-websocket is a simple websocket server interface to the kafka distributed message broker. It supports clients subscribing to topics, including multiple topics at once, and sending messages to topics. Messages may be either text or binary, the format for each is described below. A client may produce and consume messages on the same connection.

Project Server 2007 Bulk Edit

  •    

The purpose of this project is to allow the bulk edit of resource data. With large numbers of resources Resource Center can take a long time to load, primarily due to the security checks required. This tool goes directly against the Project Server Reporting store to pull back t

EFCore

  •    CSharp

EntityFrameworkCore extensions: Bulk operations (Insert, Update, Delete, Read, Upsert, Sync) and Batch (Delete, Update). Library is Lightweight and very Efficient, having all mostly used CRUD operation. Was selected in top 20 EF Core Extensions recommended by Microsoft. It is targeting NetStandard 2.0 so it can be used on project targeting NetCore(2.0+) or NetFramework(4.6.1+). Current version is using EF Core 2.2 and at the moment supports Microsoft SQL Server(2008+) and SQLite. EFCore/v.Nuget: EFCore2.1/v2.4.1 EFCore2.0/v2.0.8, and for EF Core 1.x use 1.1.0 (targeting NetStandard 1.4) Under the hood uses SqlBulkCopy for Insert, for Update/Delete combines BulkInsert with raw Sql MERGE. For SQLite there is no BulkCopy, instead library uses plain SQL combined with UPSERT. Bulk Tests can not have UseInMemoryDb because InMemoryProvider does not support Relational-specific methods. If Windows Authentication is used then in ConnectionString there should be Trusted_Connection=True; because Sql credentials are required to stay in connection.

SQL Compact Bulk Insert Library

  •    

.NET Library for loading data fast (doing bulk inserts) into a SQL Server Compact database file. Attempts to mimic the SQLClient SqlBulkCopy API.

BCP2SQLAzure: Free tool to import/export data between SQL Azure and SQL Server

  •    

SQL Azure BCP makes it easier for database admins and developers, to do bulk copy loads into SQL Azure. This bcp replacement allows you to define each load using an easily XML file. It´s developed using C#.

SQL sliced bulk inserter

  •    

Alternative approach to bulk insert delimited files when extreme performance is required. Input: some large delimited file (GBs) Operation: BULK INSERT Destination: SQL Server table(s) Method: Barbaric approach Technology: SQLNCLI, Mem mapped files, Windows 2008, C++2008

Headwind MDM – Open Source Mobile Device Management System for Android

  •    Java

Headwind MDM is the software for managing corporate mobile (Android-based) devices. System administrators can track phones and tablets, perform bulk installation and update of mobile applications, prevent user from entering the device settings and running unwanted apps.

Bulk JSON HttpHandler

  •    ASPNET

The Bulk Json Handler allows in ASP.NET to accomplish in one call, execute multiple json methods on the server and push return values from one method to another based on JSON information sent to server without client interaction.