1209 Open source projects that are alternatives of or similar to OpenScraper

A python web scraper for public email lists.

A Facebook crawler

Creating Scrapy scrapers via the Django admin interface

豆瓣电影top250、斗鱼爬取json数据以及爬取美女图片、淘宝、有缘、CrawlSpider爬取红娘网相亲人的部分基本信息以及红娘网分布式爬取和存储redis、爬虫小demo、Selenium、爬取多点、django开发接口、爬取有缘网信息、模拟知乎登录、模拟github登录、模拟图虫网登录、爬取多点商城整站数据、爬取微信公众号历史文章、爬取微信群或者微信好友分享的文章、itchat监听指定微信公众号分享的文章

Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.

[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。

Free proxy server, continuously crawling and providing proxies, based on Tornado and Scrapy. 免费代理服务器，基于Tornado和Scrapy，在本地搭建属于自己的代理池

High performance crawler for Nodejs

快速、简洁且强大的PHP爬虫框架

Docker microservice & Crawler by scrapy

HTTP API for Scrapy spiders

Get Aliexpress product details in JSON

A web spider for shodan.io without using the Developer API.

Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.

一个基于webmagic框架二次开发的java爬虫框架实战，已实现能爬取腾讯，搜狐，今日头条（单独集成功能）等资讯内容，配合elasticsearch框架用法，实现了自动爬虫，已投入线上生产使用。

📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.

163music spider by scrapy.

The Zipru scraper developed in the Advanced Web Scraping Tutorial.

Scraping scripts for various websites.

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

A scrapy spider that scraps times and prices from Voyages Sncf. It uses scrapyrt to provide an API interface.

AV 电影管理系统， avmoo , javbus , javlibrary 爬虫，线上 AV 影片图书馆，AV 磁力链接数据库，Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database

An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。

Iota is a web scraper which can find all of the images and links/suburls on a webpage

招聘网数据爬虫

admin ui for scrapy/open source scrapinghub

Elegant Scraper and Crawler Framework for Golang

爬虫工程师常用的 Chrome 插件 | Chrome extensions used by crawler developer

10 photo website spiders, 10 个国外图库的 scrapy 爬虫代码

Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js

Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.

Linkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy

Fresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion

基于python语言开发的千万级别搜索引擎

A collection of awesome web crawler,spider in different languages

Crawly, a high-level web crawling & scraping framework for Elixir.

OSINT Swiss Army Knife

A high performance web crawler in Elixir.

Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. DEMO 👉

🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音|TikTok数据爬取工具，支持API调用，在线批量解析及下载。

A web crawler (for bug hunting) that gathers more than you can imagine.

Scrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!

robots.txt file parsing and checking for R

NScrapy is a .net core corss platform Distributed Spider Framework which provide an easy way to write your own Spider

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

Dynamic configurable crawl (动态可配置化爬虫)

crawler framework, distributed crawler extractor

Araignée LeBonCoin avec Scrapy et ElasticSearch

The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url

日常爬虫

图片爬取下载工具，极速爬取下载 站酷https://www.zcool.com.cn/, CNU 视觉 http://www.cnu.cc/ 设计师/用户 上传的 图片/照片/插画。

爬虫学习仓库，适合零基础的人学习，对新手比较友好

Grabs all of the audio files from all of the Blinkist books

免费 IP 代理池。Scrapy 爬虫框架插件

ScrapingOutsourcing专注分享爬虫代码 尽量每周更新一个

💐Marmot | Web Crawler/HTTP protocol Download Package 🐭

Indonesia Index News Crawler, including 10 online media

A web crawler for Go