airflow

#计算机科学#Apache Airflow 是一个workflow工作流调度、编排、监控平台

#计算机科学#Kubernetes 工作流引擎

一个分布式易扩展的可视化DAG工作流任务调度系统。致力于解决数据处理流程中错综复杂的依赖关系，使调度系统在数据处理流程中开箱即用

#计算机科学#PipelineAI

#编辑器#Build data pipelines, the easy way 🛠️

#Awesome#Curated list of resources about Apache Airflow

Docker Apache Airflow

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualizati...

#计算机科学#Elyra extends JupyterLab with an AI centric approach.

A series of DAGs/Workflows to help maintain the operation of Airflow

Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

#面试#More than 2000+ Data engineer interview questions.

Construct Apache Airflow DAGs Declaratively via YAML configuration files

#网络爬虫#Example end to end data engineering project.

#计算机科学#A Data Engineering & Machine Learning Knowledge Hub

Run your dbt Core or dbt Fusion projects as Apache Airflow DAGs and Task Groups with a few lines of code

Personal Data Engineering Projects

🌀 𝗧𝗵𝗲 𝗙𝘂𝗹𝗹 𝗦𝘁𝗮𝗰𝗸 𝟳-𝗦𝘁𝗲𝗽𝘀 𝗠𝗟𝗢𝗽𝘀 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 | 𝗟𝗲𝗮𝗿𝗻 𝗠𝗟𝗘 & 𝗠𝗟𝗢𝗽𝘀 for free by designing, building and deploying an end-to-end ML batch system ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤...

Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.