GitHub 中文社区
回车: Github搜索    Shift+回车: Google搜索
论坛
排行榜
趋势
登录

©2025 GitHub中文社区论坛GitHub官网网站地图GitHub官方翻译

  • X iconGitHub on X
  • Facebook iconGitHub on Facebook
  • Linkedin iconGitHub on LinkedIn
  • YouTube iconGitHub on YouTube
  • Twitch iconGitHub on Twitch
  • TikTok iconGitHub on TikTok
  • GitHub markGitHub’s organization on GitHub
集合主题趋势排行榜
#

爬虫

css logo

A Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web and that is typically operated by search engines for the purpose of Web indexing (web spidering).

Website
Wikipedia
维基百科
scrapy/scrapy
https://static.github-zh.com/github_avatars/scrapy?size=40
scrapy / scrapy

#爬虫框架#一款流行,高效,生态丰富的Python爬虫框架

Pythonscrapingcrawling框架爬虫Hacktoberfestweb-scrapingweb-scraping-python
Python 57.07 k
1 天前
https://static.github-zh.com/github_avatars/mendableai?size=40
mendableai / firecrawl

#网络爬虫#Firecrawl 是一种 API 服务,它爬取URL并将其转换为清洗过的 markdown 或结构化数据

人工智能爬虫dataMarkdownscraperhtml-to-markdown大语言模型ragscrapingweb-crawlerai-scrapingwebscraping
TypeScript 39.99 k
2 天前
https://static.github-zh.com/github_avatars/NaiboWang?size=40
NaiboWang / EasySpider

#前端开发#A visual no-code/code-free web crawler/spider易采集:一个可视化浏览器自动化测试/数据采集/爬虫软件,可以无代码图形化的设计和执行爬虫任务。别名:ServiceWrapper面向Web应用的智能化服务封装系统。

code-free爬虫GUIlaymanspiderparametersWebinput-parameters前端HTMLbatch-processingbatch-scriptvisual可视化visualprogrammingscraperdata-collectionrpaRobotics
JavaScript 39.06 k
21 天前
https://static.github-zh.com/github_avatars/iawia002?size=40
iawia002 / lux

#网络爬虫#一个Go语言开发命令行视频下载工具

下载器Go爬虫scraperVideo哔哩哔哩YouTubeyoukuiqiyitumblrqqdownload
Go 29.73 k
1 个月前
https://static.github-zh.com/github_avatars/gocolly?size=40
gocolly / colly

#爬虫框架#一个快速优雅的Golang爬虫框架

Goscraper框架爬虫scrapingcrawlingspider
Go 24.32 k
5 天前
https://static.github-zh.com/github_avatars/jhao104?size=40
jhao104 / proxy_pool

#网络爬虫#Python ProxyPool for web spider

爬虫proxyspiderRedisHTTP
Python 22.45 k
4 个月前
https://static.github-zh.com/github_avatars/ScrapeGraphAI?size=40
ScrapeGraphAI / Scrapegraph-ai

#网络爬虫#Python scraper based on AI

scrapingscraping-pythonautomated-scraper大语言模型人工智能web-crawlerweb-scrapingai-scraping爬虫html-to-markdownMarkdownrag
Python 20 k
2 天前
apify/crawlee
https://static.github-zh.com/github_avatars/apify?size=40
apify / crawlee

#网络爬虫#Crawlee - 一个用于Node.js 开发的网页爬虫和浏览器自动化库

web-scrapingweb-crawlingnpmheadless-chromePuppeteer自动化apifyscrapingcrawling爬虫headlessscraperweb-crawlerJavaScriptNode.jsPlaywrightTypeScript
TypeScript 17.92 k
2 天前
https://static.github-zh.com/github_avatars/binux?size=40
binux / pyspider

#爬虫框架#python爬虫框架。简单易上手,自带在线编程和任务管理界面

Python爬虫
Python 16.67 k
1 年前
https://static.github-zh.com/github_avatars/codelucas?size=40
codelucas / newspaper

#网络爬虫#一个Python数据采集框架,能自动提取新闻、文章的标题、关键词、作者、摘要、正文等元数据

Pythonnews爬虫crawlingscrapernews-aggregator
HTML 14.61 k
3 个月前
https://static.github-zh.com/github_avatars/shengqiangzhang?size=40
shengqiangzhang / examples-of-web-crawlers

#网络爬虫#一些非常有趣的python爬虫例子,对新手比较友好,主要爬取淘宝、天猫、微信、微信读书、豆瓣、QQ等网站。(Some interesting examples of python crawlers that are friendly to beginners. )

爬虫spidertaobaotmallExamplePythonSeleniumpyquerystockfundmultithreadingWeChat
Python 14.24 k
1 年前
https://static.github-zh.com/github_avatars/projectdiscovery?size=40
projectdiscovery / katana

#网络爬虫#下一代爬虫框架

爬虫web-spidergocrawlerspider-framework命令行界面headless
Go 13.81 k
6 天前
Evil0ctal/Douyin_TikTok_Download_API
https://static.github-zh.com/github_avatars/Evil0ctal?size=40
Evil0ctal / Douyin_TikTok_Download_API

#网络爬虫#🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音、快手、TikTok、Bilibili数据爬取工具,支持API调用,在线批量解析及下载。

PythonpywebioTikTokdouyinAPIscraperFastAPIno-watermarkonline-parsingasyncdouyin-tiktok-apidouyin-tiktok-download爬虫spiderweb-scrapingtiktok-scraperdouyin-scraperdouyin-apitiktok-apitiktok-signature
Python 13.03 k
3 个月前
crawlab-team/crawlab
https://static.github-zh.com/github_avatars/crawlab-team?size=40
crawlab-team / crawlab

#网络爬虫#Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台,支持任何语言和框架

webcrawlerscrapycrawlabspiders-managementGoscrapyd-uispider爬虫webspiderweb-crawlerDockerplatformcrawling-tasks
Go 11.77 k
2 天前
https://static.github-zh.com/github_avatars/s0md3v?size=40
s0md3v / Photon

#夺旗赛 (CTF) 和网络安全资源#Incredibly fast crawler designed for OSINT.

爬虫spiderPythonOSINTinformation-gathering
Python 11.65 k
3 个月前
https://static.github-zh.com/github_avatars/code4craft?size=40
code4craft / webmagic

#网络爬虫#webmagic是一个开源的Java垂直爬虫框架,目标是简化爬虫的开发流程,让开发者专注于逻辑功能的开发。webmagic的核心非常简单,但是覆盖爬虫的整个流程,也是很好的学习爬虫开发的材料。

爬虫Javascraping框架
Java 11.57 k
1 个月前
https://static.github-zh.com/github_avatars/injetlee?size=40
injetlee / Python

#网络爬虫#Python脚本。模拟登录知乎, 爬虫,操作excel,微信公众号,远程开机

Python爬虫WeChatexcel
Python 10.11 k
2 年前
https://static.github-zh.com/github_avatars/ssssssss-team?size=40
ssssssss-team / spider-flow

#网络爬虫#新一代爬虫平台,以图形化方式定义爬虫流程,不写代码即可完成爬虫。

spider爬虫jsoupxpathweb-spiderwebspiderwebcrawlerweb-crawlerspider-flow
Java 10.01 k
2 年前
https://static.github-zh.com/github_avatars/guyueyingmu?size=40
guyueyingmu / avbook

#网络爬虫#AV 电影管理系统, avmoo , javbus , javlibrary 爬虫,线上 AV 影片图书馆,AV 磁力链接数据库,Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database

javbusavmoojavlibraryspider爬虫Laravelscraperadultmagnet-linkmagnet数据库adult-videoguzzlehttp
PHP 9.71 k
1 年前
https://static.github-zh.com/github_avatars/TeamWiseFlow?size=40
TeamWiseFlow / wiseflow

#网络爬虫#Use LLMs to dig out what you care about from massive amounts of information and a variety of sources daily.

爬虫information-gathering大语言模型scraper
Python 7.56 k
8 小时前
loading...