data-collection · GitHub Topics

#前端开发#A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。

code-free 爬虫 GUI layman spider parameters Web input-parameters 前端 HTML batch-processing batch-script visual 可视化 visualprogramming scraper data-collection rpa Robotics

JavaScript 39.06 k

21 天前

airbytehq / airbyte

Airbyte 开源 EL(T) 平台，帮助用户将数据从应用程序，API 和数据库中同步到数据仓库

data pipeline 数据分析 data-engineering Java Python etl change-data-capture data-collection data-integration elt BigQuery redshift snowflake data-pipeline sql-server MySQL PostgreSQL s3 自托管

Python 18.43 k

13 小时前

snowplow / snowplow

The leader in Customer Data Infrastructure

analytics data data-pipeline data-collection product-analytics

Scala 6.93 k

11 天前

cloudquery / cloudquery

一个高性能ELT 框架，powered by Apache Arrow

Amazon Web Services Google 云 Azure SQL data-integration elt etl etl-framework BigQuery data-collection data-engineering Kubernetes data airbyte GitHub API 数据分析 Google Go cspm attack-surface-management

Go 6.12 k

2 天前

jitsucom / jitsu

Jitsu is an open-source Segment alternative. Fully-scriptable data ingestion engine for modern data teams. Set-up a real-time data pipeline in minutes, not days

data-integration clickhouse Go BigQuery data-collection redshift snowflake PostgreSQL

TypeScript 4.31 k

6 天前

mendableai / firecrawl-mcp-server

Official Firecrawl MCP Server - Adds powerful web scraping to Cursor, Claude and any other LLM clients.

batch-processing claude content-extraction data-collection firecrawl firecrawl-ai llm-tools mcp-server model-context-protocol search-api web-crawler web-scraping javascript-rendering

JavaScript 3.42 k

11 天前

pyper-dev / pyper

Concurrent Python made simple

asyncio concurrency Python threading data-pipelines data-processing multiprocessing parallel-computing data data-collection data-engineering

Python 1.43 k

4 个月前

Decodo / Decodo

#网络爬虫#HTTP(S)/SOCKS5 rotating residential proxies - code examples & general information.

proxy proxies https-proxy proxy-server Python data-collection proxy-list python-scraper scraping web-scraping data-gathering http-proxy ip-rotation socks5-proxy

Java 1.13 k

12 天前

plan-player-analytics / Plan

Player Analytics plugin for Minecraft Server platforms - View player activity of your server with ease. 📆

analytics 统计可视化 MySQL SQLite webserver data-collection sponge-plugin spigot-plugin bungeecord-plugin bukkit-plugin Hacktoberfest FabricMC

Java 932

7 天前

getodk / collect

#安卓#ODK Collect is an Android app for filling out forms. It's been used to collect billions of data points in challenging environments around the world. Contribute and make the world a better place! ✨📋✨

odk global-development data-collection global-health mhealth xforms Android Java social-impact mobile-data-collection

Java 732

12 天前

chaoss / augur

Python library and web service for Open Source Software Health and Sustainability metrics & data collection. You can find our documentation and new contributor information easily here: https://oss-aug...

Linux Open Source GitHub 数据可视化 facade Git 监控 sustainability health python-library data-collection data-modeling Unix research Hacktoberfest hacktoberfest2020

Python 635

5 天前

brightdata / brightdata-mcp

#网络爬虫#A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public web access.

大语言模型 mcp modelcontextprotocol scraping ai-agents browser-automation data-collection data-extraction mcp-server structured-data web-crawling web-scraping

JavaScript 614

10 天前

pnoker / iot-dc3

IoT DC3 is a 100% open-source, distributed Internet of Things (IoT) platform built on Spring Cloud. It accelerates IoT project development and simplifies IoT device management, offering a comprehensiv...