Displaying 1 to 13 from 13 results

awesome-apache-airflow - Curated list of resources about Apache Airflow

  •    

This is a curated list of resources about Apache Airflow (incubating). Please feel free to contribute any items that should be included. Items are generally added at the top of each section so that more fresh items are featured more prominently.

airflow-on-kubernetes - A guide to running Airflow on Kubernetes

  •    

Recently I spend quite some time diving into Airflow and Kubernetes. While there are reports of people using them together, I could not find any comprehensive guide or tutorial. Also, there are many forks and abandoned scripts and repositories. So you need to do some research. Here I write down what I've found, in the hope that it is helpful to others. Because things move quickly, I've decided to put this on Github rather than in a blog post, so it can be easily updated. Please do not hesitate to provide updates, suggestions, fixes etc.




airflow-operator - Kubernetes custom controller and CRDs to managing Airflow

  •    Go

This is not an officially supported Google product. The Airflow Operator is still under active development and has not been extensively tested in production environment. Backward compatibility of the APIs is not guaranteed for alpha releases.

astronomer - Apache Airflow as a Service on Kubernetes

  •    Dockerfile

Astronomer is a commercial "Airflow as a Service" platform that runs on Kubernetes. Source code is made avaialable for the benefit of our customers, if you'd like to use the platform reach out for a license or try out Astronomer Cloud. Yes.

dag-factory - Dynamically generate Apache Airflow DAGs from YAML configuration files

  •    Python

dag-factory is a library for dynamically generating Apache Airflow DAGs from YAML configuration files. To install dag-factory run pip install dag-factory. It requires Python 3.6.0+ and Apache Airflow 1.9+.


pipeline-editor - Common pipeline-editor components used in different clients (e

  •    TypeScript

A react component for editing pipeline files. Used across all Elyra applications and browser extensions.

aws-airflow-demo - Project files for the post: Running PySpark Applications on Amazon EMR using Apache Airflow: Using the new Amazon Managed Workflows for Apache Airflow (MWAA) on AWS

  •    Python

Project files for the post, Running PySpark Applications on Amazon EMR using Apache Airflow: Using the new Amazon Managed Workflows for Apache Airflow (MWAA) on AWS. Please see post for complete instructions on using the project's files. Below is the final high-level architecture for the post’s demonstration. The diagram shows the approximate route of a DAG Run request, in red. The diagram includes an optional S3 Gateway VPC endpoint, not detailed in the post, but recommended for additional security.

airflow-client-go - Apache Airflow - OpenApi Client for Go

  •    Go

Go Airflow OpenAPI client generated from openapi spec. See README for full client API documentation.

airflow-client-python - Apache Airflow - OpenApi Client for Python

  •    Python

NOTE: The Apache Airflow Client is still under active development and some methods or APIs might be broken. Please raise an issue in github if you encounter any such issues. Or install via Setuptools.

paperboy - A web frontend for scheduling Jupyter notebook reports

  •    Python

Paperboy is a production-grade application for scheduling reports. It has a flexible architecture and extensible APIs, and can integrate into a wide variety of deployments. It is composed of various industrial-strength technologies from the open source world. Paperboy requires Python and Node.js, which can be installed from conda-forge if conda is available.






We have large collection of open source products. Follow the tags from Tag Cloud >>


Open source products are scattered around the web. Please provide information about the open source projects you own / you use. Add Projects.