Symfony DomCrawler Component
Simple, elegant, extensible PHP Web Scraper (crawler/spider),Use the css3 dom selector,Based on phpQuery! ??????????PHP????(??)???phpQuery?
A configurable and extensible PHP web spider
Crawl all internal links found on a website
CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent
Package to provide api-like access to foreign sites based on html parsing
jQuery-inspired DOM manipulation extension for Symfony's Crawler
Beanbun ?? PHP ???????????????????????????
A simple and lightweight cURL library with support for asynchronous requests.
A crawler to find all unique internal pages on a given website
CLI tool to crawl a website and check HTTP status code
A powerful web crawler and web scraper with Blackfire support
Given a robots.txt file, user agent and URL path will tell you whether you're allowed to access a page
A Laravel package to detect web crawlers via the user agent
The standalone PHP Browscap parser Crossjoin\Browscap detects browser properties as well as device information based on the user agent string of the requesting browsers and search engines, using the data from the Browser Capabilities Project. It's several hundred times faster than the build-in PHP function get_browser(), and faster than other Browscap PHP libraries, with much lower memory consumption. Optionally Crossjoin\Browscap automatically updates the Browscap data, so you're always up-to-date. The newest version is build for PHP 7.x, for PHP >= 5.6 use version 2.x, for PHP >= 5.3 use version 1.x.
Check all links in a Laravel app
PHP library for OpenSearchServer: professionnal search engine, crawlers (web, file, database), REST APIs, .... This library uses OpenSearchServer's V2 API.
PHP POO cURL wrapper (PSR friendly) : light and simple.
A web scraper for PHP to easily extract data from web pages
php crawler library
whm library for crwling websites
Detect robots and crawlers.
Fat-free, standalone and fast web crawler (bot) detector
An full search page crawler to enable complex and customized searching abilities.