change-data-capture · GitHub Topics

Airbyte 开源 EL(T) 平台，帮助用户将数据从应用程序，API 和数据库中同步到数据仓库

data pipeline 数据分析 data-engineering Java Python etl change-data-capture data-collection data-integration elt BigQuery redshift snowflake data-pipeline sql-server MySQL PostgreSQL s3 自托管

Python 18.43 k

17 小时前

debezium / debezium

Debezium为捕获数据更改(change data capture,CDC)提供了一个低延迟的流式处理平台

change-data-capture kafka-connect apache-kafka debezium cdc 数据库 kafka kafka-producer event-streaming data-pipeline

Java 11.49 k

3 天前

apache / seatunnel

SeaTunnel （原名为 waterdrop）是一个易用的支持海量数据实时同步的高性能分布式数据集成平台，每天可以稳定同步数百亿数据

data-integration high-performance offline real-time apache batch cdc change-data-capture data-ingestion elt streaming

Java 8.57 k

3 天前

supabase / realtime

Broadcast, Presence, and Postgres Changes via WebSockets

Elixir PostgreSQL realtime phoenix phoenix-framework cdc change-data-capture crdt distributed-systems Publish-subscribe pattern

Elixir 7.14 k

4 天前

apache / flink-cdc

Flink CDC Connector 是ApacheFlink的一组数据源连接器

change-data-capture cdc batch data-integration data-pipeline distributed elt etl flink kafka MySQL paimon PostgreSQL real-time schema-evolution

Java 6.1 k

2 天前

cocoindex-io / cocoindex

#大语言模型#Real-time data transformation framework for AI. Ultra performant, with incremental processing.

Rust 1.88 k

2 天前

debezium / debezium-examples

Examples for running Debezium (Configuration, Docker Compose files etc.)

debezium change-data-capture docker-compose-files Example

JavaScript 1.58 k

18 天前

sequinstream / sequin

Postgres change data capture to streams and queues like Kafka, SQS, HTTP endpoints, and more

cdc change-data-capture Elixir Erlang kafka PostgreSQL queue streaming

Elixir 1.36 k

2 天前

toluaina / pgsync

Postgres to Elasticsearch/OpenSearch sync

elasticsearch PostgreSQL change-data-capture SQL opensearch Python etl kibana

Python 1.29 k

4 小时前

drasi-project / drasi-platform

The Data Change Processing platform

cdc change-data-capture change-detection

Rust 1.14 k

6 天前

linkedin / brooklin

An extensible distributed system for reliable nearline data streaming at scale

distributed-systems data-streaming scalability kafka-mirror-maker kafka change-data-capture Java linkedin

Java 940

1 年前

moiot / gravity

A Data Replication Center

MySQL storage replication change-data-capture gravity

Go 920

2 年前

datazip-inc / olake

Fastest open-source tool for replicating Databases to Data Lake in Open Table Formats like Apache Iceberg. ⚡ Efficient, quick and scalable data ingestion for real-time analytics. Supporting Postgres,...

cdc change-data-capture data-pipeline 数据库 elt lakehouse replication apache-iceberg parquet s3

Go 889

3 天前

artie-labs / transfer

Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift, Databricks) in real-time.

snowflake cdc change-data-capture Go BigQuery kafka apache-kafka data-integration data-pipelines 数据库 debezium elt redshift

Go 655

9 天前