Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.
Construct Apache Airflow DAGs Declaratively via YAML configuration files
A CLI tool to streamline getting started with Apache Airflow™ and managing multiple Airflow projects
A Python 3 package for learning Bayesian Networks (DAGs) from data. Official implementation of the paper "DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterization"
This provider contains operators, decorators and triggers to send a ray job from an airflow task
Orchestrate your Databricks notebooks in Airflow and execute them as Databricks Workflows
Stop creating bad DAGs! Use this tool to measure and compare the parse time of your DAGs, identify bottlenecks, and optimize your Airflow environment for better performance.
This project demonstrates how to build and automate an ETL pipeline written in Python and schedule it using open source Apache Airflow orchestration tool on AWS EC2 instance.
Opinionated framework based on Airflow 2.0 for building pipelines to ingest data into a BigQuery data warehouse
dagrad is a Python package that provides an extensible, modular platform for developing and experimenting with differentiable (gradient-based) structure learning methods.
A production-grade MLOps pipeline for real-time USD volatility forecasting using EUR/USD forex data. This project demonstrates end-to-end ML lifecycle management with automated data ingestion, model training, deployment, and monitoring.
This a repo that was created to learn more about Airflow and develop awesome data engineering projects. 🚀🚀
The dag-checks consist of checks that can help you in maintaining your Apache Airflow instance.
This repo consist of codes to fetch covid data and store them into bigquery, scheduled by apache-airflow.
This is a repo to keep track of all my projects related to data engineering
Add a description, image, and links to the dags topic page so that developers can more easily learn about it.
To associate your repository with the dags topic, visit your repo's landing page and select "manage topics."