nifi-minio - A custom ContentRepository implementation for NiFi to persist data to MinIO Object Storage

  •        250

Now visit http://localhost:8080/nifi to start configuring data flows.

https://github.com/minio/nifi-minio


Tags
Implementation
License
Platform

   




Related Projects

awesome-nifi - A list of useful Apache NiFi resources, processor bundles and tools

  •    

Copyright (c) 2017 Joey Frazee. awesome-nifi is licensed under the Apache License 2.0.

Apache NiFi - An easy to use, powerful, and reliable system to process and distribute data

  •    Java

Apache NiFi supports powerful and scalable directed graphs of data routing, transformation, and system mediation logic. Data flow can be tracked and modified at run time. It automates the movement of data between disparate data sources and systems, making data ingestion fast, easy and secure. The project was created by the United States National Security Agency (NSA).

fractalide - Reusable Reproducible Composable Software

  •    Racket

Fractalide is a free and open source service programming platform using dataflow graphs. Graph nodes represent computations, while graph edges represent typed data (may also describe tensors) communicated between them. This flexible architecture can be applied to many different computation problems, initially the focus will be Microservices to be expanded out into the Internet of Things. Fractalide is in the same vein as the NSA’s Niagrafiles (now known as Apache-NiFi) or Google’s TensorFlow but stripped of all Java, Python and GUI bloat. Fractalide faces big corporate players like Ab Initio, a company that charges a lot of money for dataflow solutions.


console - Management UI for MinIO and MinIO operator

  •    TypeScript

All console needs is a MinIO user with admin privileges and URL pointing to your MinIO deployment.

operator - MinIO Operator creates/configures/manages MinIO clusters on Kubernetes

  •    Go

MinIO is a Kubernetes-native high performance object store with an S3-compatible API. The MinIO Kubernetes Operator supports deploying MinIO Tenants onto private and public cloud infrastructures ("Hybrid" Cloud). The MinIO Console provides a graphical user interface (GUI) for interacting with MinIO Tenants.

minio-go - Minio Client SDK for Go

  •    Go

The Minio Go Client SDK provides simple APIs to access any Amazon S3 compatible object storage.This quickstart guide will show you how to install the Minio client SDK, connect to Minio, and provide a walkthrough for a simple file uploader. For a complete list of APIs and examples, please take a look at the Go Client API Reference.

Minio - Open source object storage server compatible with Amazon S3 APIs

  •    Go

Minio is an object storage server, It is compatible with Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups and container / VM images. Size of an object can range from a few KBs to a maximum of 5TB. Minio server is light enough to be bundled with the application stack, similar to NodeJS, Redis and MySQL.

stock-analysis-engine - Backtest 1000s of minute-by-minute trading algorithms for training AI with automated pricing data from: IEX, Tradier and FinViz

  •    Jupyter

Build and tune investment algorithms for use with artificial intelligence (deep neural networks) with a distributed stack for running backtests using live pricing data on publicly traded companies with automated datafeeds from: IEX Cloud, Tradier and FinViz (includes: pricing, options, news, dividends, daily, intraday, screeners, statistics, financials, earnings, and more). This will pull Redis and Minio docker images.

Coolify - An open-source, hassle-free, self-hostable Heroku & Netlify alternative

  •    Typescript

Coolify is an open-source, hassle-free, self-hostable Heroku & Netlify alternative. You can deploy Static Sites, any Node.js based applications, PHP, Rust applications hassle-free with automatic reverse proxy and SSL certs. You can create your own instance of MongoDB, MySQL, PostgreSQL, CouchDB, RedisDB, WordPress, Plausible Analytics, NocoDB, MinIO or VSCode Server.

mc - Minio Client is a replacement for ls, cp, mkdir, diff and rsync commands for filesystems and object storage

  •    Go

Minio Client (mc) provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff, find etc. It supports filesystems and Amazon S3 compatible cloud storage service (AWS Signature v2 and v4).then use the mc config command.

docs - MinIO Object Storage Documentation

  •    Python

Artifacts output to the build/ directory as HTML. Just open index.html to get started poking around. Still need to work out deployment steps, but this should work for testing locally.

Sidekick - High Performance HTTP Sidecar Load Balancer

  •    Go

sidekick is a high-performance sidecar load-balancer. By attaching a tiny load balancer as a sidecar to each of the client application processes, you can eliminate the centralized loadbalancer bottleneck and DNS failover management. sidekick automatically avoids sending traffic to the failed servers by checking their health via the readiness API and HTTP error returns.

swarrot - A lib to consume message from any Broker

  •    PHP

Swarrot is a PHP library to consume messages from any broker. Heavily inspired by stackphp/builder you can use Swarrot\Processor\Stack\Builder to stack your processors. Using the built in processors or by creating your own, you can extend the behavior of your base processor. In this example, your processor is decorated by 2 other processors. The ExceptionCatcherProcessor which decorates your own with a try/catch block and the MaxMessagesProcessor which stops your worker when some messages have been consumed.

Intel-Linux-Processor-Microcode-Data-Files

  •    

The Intel Processor Microcode Update (MCU) Package provides a mechanism to release updates for security advisories and functional issues, including errata. In addition, MCUs are responsible for starting the SGX enclave (on processors that support the SGX feature), implementing complex behaviors (such as assists), and more. The preferred method to apply MCUs is using the system BIOS. For a subset of Intel's processors, the MCU can also be updated at runtime using the operating system. The Intel Microcode Package shared here contains updates for those processors that support OS loading of MCUs. Updating your microcode can help to mitigate certain potential security vulnerabilities in CPUs as well as address certain functional issues that could, for example, result in unpredictable system behavior such as hangs, crashes, unexpected reboots, data errors, etc. To learn more about applying MCUs to an Intel processor, see Microcode Update Guidance.

ratchet - A library for performing data pipeline / ETL tasks in Go.

  •    Go

The Go programming language's simplicity, execution speed, and concurrency support make it a great choice for building data pipeline systems that can perform custom ETL (Extract, Transform, Load) tasks. Ratchet is a library that is written 100% in Go, and let's you easily build custom data pipelines by writing your own Go code. Each data processor is receiving, processing, and then sending data to the next stage in the pipeline. All data processors are running in their own goroutine, so all processing is happening concurrently. Go channels are connecting each stage of processing, so the syntax for sending data will be intuitive for anyone familiar with Go. All data being sent and received is JSON, which provides for a nice balance of flexibility and consistency.

Rome - A cache tool for Carthage

  •    Haskell

as a shared cache for frameworks built with Carthage. This will download Rome to the Pods/ folder during your next pod install execution and will allow you to invoke it via ${PODS_ROOT}/Rome/rome in your Script Build Phases.

Nuage - A Dropbox-like interface for your existing data (with support for many backends)

  •    Javascript

A Dropbox-like file manager that let you manage your data anywhere it is located. It could be FTP, SFTP, WebDAV, Git, S3, Minio, Dropbox, Google Drive. It helps to manage your files from a browser, Upload files and folders, Audio player, Video player, Image viewer and lot more.






We have large collection of open source products. Follow the tags from Tag Cloud >>


Open source products are scattered around the web. Please provide information about the open source projects you own / you use. Add Projects.