spark-streaming · GitHub Topics

Angel-ML / angel

#计算机科学#A Flexible and Powerful Parameter Server for large-scale machine learning

机器学习 parameter-server Apache Spark Scala model high-dimensional online-learning spark-streaming

Java 6.76 k

4 天前

lw-lin / CoolplaySpark

酷玩 Spark: Spark 源代码解析、Spark 类库等

Apache Spark spark-streaming

Scala 3.49 k

3 年前

LuckyZXL2016 / Movie_Recommend

基于Spark的电影推荐系统，包含爬虫项目、web网站、后台管理系统以及spark推荐系统

spark-mllib spark-streaming ssm-maven scrapy Scala hadoop nginx hive MySQL

Java 2.95 k

6 年前

dotnet / spark

#计算机科学#.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

Apache Spark C#.NET analytics bigdata spark-streaming spark-sql 机器学习 F#dotnet-standard streaming Azure hdinsight databricks emr Microsoft

C# 2.07 k

1 个月前

jacksu / utils4s

scala、spark使用过程中，各种测试用例以及相关资料整理

Apache Spark breeze Scala akka spark-streaming

Scala 1.09 k

6 年前

edp963 / wormhole

Wormhole is a SPaaS (Stream Processing as a Service) Platform

stream-processing spark-streaming

JavaScript 977

3 年前

microsoft / Mobius

C# and F# language binding and extensions to Apache Spark

Apache Spark dataframe dataset streaming C#spark-streaming F#bigdata mapreduce

C# 940

1 年前

cdapio / cdap

An open source framework for building data analytic applications.

unified integration platform dataset mapreduce Apache Spark spark-streaming Java cdap Python middleware

Java 778

3 天前

lw-lin / streaming-readings

Streaming System 相关的论文读物

stream-processing streaming flink spark-streaming storm heron dataflow drizzle Apache Spark streaming-engine stream-processing-engine

733

3 年前

Stratio / sparta

Real Time Analytics and Data Pipelines based on Spark Streaming

streaming-data Scala Apache Spark streaming spark-streaming olap kafka hdfs workflow analytics real-time sparksql lambda triggers

Scala 526

6 年前

spirom / LearningSpark

Scala examples for learning to use Spark

Scala Apache Spark spark-streaming sparksql

Scala 444

5 年前

databrickslabs / dbldatagen

Generate relevant synthetic data quickly for your projects. The Databricks Labs synthetic data generator (aka `dbldatagen`) may be used to generate large simulated / synthetic data sets for test, POC...

pyspark Python data-generation faker Apache Spark spark-streaming deltalake databricks synthetic-data

Python 407

1 个月前

harbby / sylph

Stream computing platform for bigdata

Java flink spark-streaming big-data SQL

Java 403

1 年前

microsoft / data-accelerator

Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsigh...

Apache Spark spark-streaming spark-sql sparksql streaming-data streaming servicefabric Node.js Docker hdinsight cosmosdb React Azure iothub big-data Internet of things kafka kafka-streams

C# 302

3 个月前

databrickslabs / dqx

Databricks framework to validate Data Quality of pySpark DataFrames

data-profiling data-quality data-quality-checks data-quality-monitoring databricks Apache Spark spark-streaming dlt

Python 275

8 天前

paypal / gimel

Big Data Processing Framework - Unified Data API or SQL on Any Storage

Apache Spark spark-streaming big-data paypal kafka Apache Cassandra hbase elasticsearch jdbc teradata restapi Scala pyspark Python

Scala 245

6 个月前

Azure / azure-event-hubs-spark

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

Apache Spark spark-streaming Azure Scala real-time streaming apache Microsoft event-hubs connector databricks stream bigdata ingestion kafka

Scala 236

4 个月前

mkuthan / example-spark

Spark, Spark Streaming and Spark SQL unit testing strategies

Apache Spark spark-streaming Testing

Scala 217

9 年前

Chabane / bigdata-playground

#计算机科学#A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apach...

Docker spark-sql Scala kafka hbase parquet avro Node.js Angular GraphQL MongoDB 机器学习 big-data hadoop Apache Spark apache-flink spark-streaming twitter-api Python kops

TypeScript 209

6 年前

spirom / spark-streaming-with-kafka

Self-contained examples of Apache Spark streaming integrated with Apache Kafka.

Apache Spark kafka Scala spark-streaming

Scala 199

7 年前