etl-pipelines

#自然语言处理#Fast and efficient unstructured data extraction. Written in Rust with bindings for many languages.

The simplest way to run Python on lot's of computers.

Data pipelines from re-usable components

The open-source Useful SDK. One python decorator in the Useful library allows for full observability of Python functions within an ETL.

#网络爬虫#A project structure for doing and sharing data engineer work.

Clean API primitives for data cleaning in Pyspark. Inspired by PyJanitor, Dataprep.AI and Shadcn.

Lien de l'application

DataSift auto applies a data pre-processing pipeline to Data Science Projects.

e-Portfolio showcasing my personal projects.

Build ETL piplines on AirFlow to load data from BigQuery and store it in MySQL

Sparkify project

Big Data ETL pipeline for Brazilian e-commerce data. Implements data ingestion, transformation, and storage using Apache Spark, Hadoop, and SQL. Designed for scalable data processing and analytics.

End To End MLOPS Project With ETL Pipelines- Building Network Security System

This project demonstrates a complete ETL pipeline for Formula 1 racing data using Azure Databricks, Delta Lake, and Azure Data Factory. It covers data ingestion, transformation with PySpark and Spark ...

JSON-driven ETL pipeline framework prototype

project in process

#博客#For scribes of Thoth in the shell — your codebrain’s sacred scroll.

This repo contains the DAGs that run on my local Airflow environment. I use the local environment to test my DAGs before deploying them to virtual machines via Kubernetes

#计算机科学#A deployed machine learning model that has the capability to automatically classify the incoming disaster messages into related 36 categories. Project developed as a part of Udacity's Data Science Nan...

An extension that registers all pharmacies in Argentina.