avro · GitHub Topics

apache / avro

Apache Avro 是通常用于RPC远程调用的序列化框架，类似于Thrift、Protocol Buffers，特点是Avro并不需要生成代码，模式和数据存放在一起

C#Ruby C++Python PHP Java C avro bigdata .NET Perl

Java 3.09 k

1 天前

confluentinc / schema-registry

Confluent Schema Registry for Kafka

schema-registry kafka schema schemas avro avro-schema REST API confluent protobuf protobuf-schema JSON JSON Schema

Java 2.3 k

3 天前

dflemstr / rq

Record Query - A tool for doing record analysis and transformation

Rust protobuf command-line-tool JSON avro messagepack YAML toml JavaScript lodash

Rust 2.29 k

1 年前

confluentinc / examples

Apache Kafka, Apache Flink and Confluent Platform examples and demos

kafka Demo Example confluent cloud connect connector jdbc debezium quickstart schema-registry 监控 avro Kubernetes cdc SQL Docker 微服务

Java 1.98 k

2 天前

capitalone / DataProfiler

#自然语言处理#What's in your data? Extract schema, statistics and entities from datasets

Python 隐私 pii 自然语言处理数据科学 gdpr 数据分析 avro sensitive-data 安全 pandas CSV tabular-data dataset graph-data 机器学习

Python 1.49 k

3 个月前

OBenner / data-engineering-interview-questions

#面试#More than 2000+ Data engineer interview questions.

data-engineering 面试 hadoop hadoop-hdfs Apache Spark flink SQL kafka hive impala airflow Amazon Web Services Azure Apache Cassandra flume hbase avro nifi 数据结构

1.35 k

5 个月前

mtth / avsc

Avro for JavaScript ⚡

avro serialization JavaScript TypeScript schema-evolution big-data encoding

JavaScript 1.34 k

2 个月前

pmacct / pmacct

pmacct is a small set of multi-purpose passive network monitoring tools [NetFlow IPFIX sFlow libpcap BGP BMP RPKI IGP Streaming Telemetry].

netflow ipfix sflow bgp bmp kafka rabbitmq libpcap geoip2 ndpi MySQL PostgreSQL sqlite3 SQL JSON avro

C 1.12 k

6 天前

linkedin / goavro

Goavro is a library that encodes and decodes Avro data.

avro Go

Go 1.03 k

7 天前

bigdatagenomics / adam

ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.

Apache Spark big-data Bioinformatics genomics parquet avro Scala Java Python R

Scala 1.03 k

1 个月前

deviceinsight / kafkactl

Command Line Tool for managing Apache Kafka

apache-kafka avro 命令行界面 kafka Go Zsh friendly interactive shell

Go 913

4 天前

Cinchoo / ChoETL

ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

CSV Parser writer reader flat XML JSON keyvalue etl etl-framework C#.NET parquet YAML avro

C# 825

9 天前

HariSekhon / DevOps-Python-tools

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Functions, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML...

cloudformation Python hbase JSON avro parquet Apache Spark pyspark travis-ci elasticsearch solr hadoop hdfs dockerhub Docker Linux Amazon Web Services DevOps Google 云 gcf

Python 798

2 个月前

thekvs / cpp-serializers

Benchmark comparing various data serialization libraries (thrift, protobuf etc.) for C++

C++serialization protobuf thrift flatbuffers cereal performance-testing boost msgpack avro

C++ 737

5 年前

miguno / kafka-storm-starter

[PROJECT IS NO LONGER MAINTAINED] Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization forma...

apache-kafka kafka storm Apache Spark Scala integration avro

Scala 724

3 年前

sksamuel / avro4s

Avro schema generation and serialization / deserialization for Scala

Scala avro avro-schema schema-generation serialization

Scala 723

15 天前

RandomFractals / vscode-data-preview

Data Preview 🈸 extension for importing 📤 viewing 🔎 slicing 🔪 dicing 🎲 charting 📊 & exporting 📥 large JSON array/config, YAML, Apache Arrow, Avro, Parquet & Excel data files

Visual Studio Code 插件 CSV arrow data viewer JSON array configuration YAML excel avro parquet

TypeScript 573

1 年前