Mixnode is a fast, flexible, massively scalable platform to extract and analyze data from the web. Mixnode allows you to think of all resources on the web as rows in a database table; a giant database table with billions of rows that you can query using the standard Structured Query Language (SQL). So, rather than running web crawlers/scrapers you can write simple queries in an ultra-flexible language to retrieve all sorts of interesting information from the web.
grab-site is a crawler for archiving websites to WARC files. It includes a dashboard for monitoring multiple crawls, and suppo ...
Apify is the easiest way to run headless Chrome jobs in the cloud. It comes with an advanced web crawler that enables the scra ...
StormCrawler is an open source SDK for building distributed web crawlers with Apache Storm. The project is under Apache licens ...
Scrapy is an open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet ex ...
Common Crawl builds and maintains an open repository of web crawl data that can be accessed and analyzed by anyone
Extracty can extract any web data and create an API to the webpage's information.
ScrapeStorm is a AI-Powered visual web scraping tool,which can be used to extract data from almost any websites without writin ...
ACHE is a web crawler for domain-specific search
Darcy Ripper is a powerful pure Java multi-platform web crawler with great work load and speed capabilities. Darcy is a standa ...
ProxyCrawl helps you stay anonymous while crawling the web, web crawling protection the way it should be. Get data for your SE ...
Apache Nutch is a highly extensible and scalable open source web crawler software project. Nutch is coded entirely in the Java ...
Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project. Heritrix (sometim ...
Turn web page content into structured data all without coding. Important - Mozenda uses a Windows application that must be in ...