Web browser automation tools

Open source Tools

Automa

Automa is a browser extension for browser automation. From auto-fill forms, doing a repetitive task, taking a screenshot, to scraping data of the website
Workflow is executed using the local browser

Scrapy

From Scrapy

Scrapy is an open source and collaborative Python framework for web scraping, extracting the data you need from websites.

It's the de-facto standard in the industry for web scraping in Python. Best choice for website that doesn't have any particular website anti-bot tool.

Playwright

From Playwright

Go to text →

Playwright is a testing tool for web application, useful also for web scraping, released on 2021.

Best choice when there's need of a fully rendered browser to scrape a website.

This configuration is more computing power intensive than a simply Scrapy installation so is used only when a fully rendered browser is needed

Puppeteer

From Puppeteer

Go to text →

Puppeteer is a Node.js library which provides a high-level API to control Chrome or Chromium over the DevTools Protocol.

Puppeteer is a browser automation tool useful for web scraping. Its features are very similar to Playwright.

Best choice when we need a full browser rendering the page and Playwright and Selenium can't help. Or we want to control via Javascript instead of Python.

Selenium

From Selenium

Go to text →

Selenium Webdriver is a web application testing suite used also for web scraping.

Playwright is preferred to Selenium Webdriver, as it has a more similar behaviour to a real user, easier to install and configure.

PageProbe

extension for Firefox only
Unlimited number of trackers. Fully local solution - does not require an account
Create automatic monitors for detecting and scanning changes on web pages

Headless Recorder

extension for Chromium only
open-source extension to record browser interaction and generating Puppeteer & Playwright scripts

AutoHotkey

AutoHotkey is a free, open-source scripting language for Windows that allows users to easily create small to complex scripts for all kinds of tasks such as: form fillers, auto-clicking, macros, etc.
AutoHotkey gives you the freedom to automate any desktop task.

Commercial tools

Microsoft Power Automate

Pricing starts at USD 15/month. If you are a Windows 10/11 user, you have access to personal desktop flows in Power Automate for desktop for free. 1
This tool does not only automate web browser but also other process automation.

Rocketride

Pricing starts at USD 9/month. Free for 500 local runs, 30 cloud runs
The task runner uses Playwright
Typical things to automate: Web scraping, Fill out forms, Download files, Periodically check for changes, Automate actions
Hacker News discussion

Browserflow

Pricing starts at USD 49/month. Free for desktop runs, but 1 minute per run
Automatically perform actions on websites as if you were doing them

visualping

Pricing starts at USD 14/month. Free for limited use
Website change detection and alerts

Distill.io

Pricing starts at USD 15/month. Free for limited use
automated tracking website updates

ScrapFly

Pricing starts at USD 15/month. Free for 1000 API calls / month

ScrapingBee

Pricing starts at USD 49/month. Free for 1000 API calls

ScraperAPI

Pricing starts at USD 29/month. Free for 1000 web pages per month.

Morph.io

Free to use. But can subscribe as a supporter, starting at USD 14/month.

Scraping Fish

Pricing starts at USD 0.002 per each API calls.

Crawly - Web Crawler by Diffbot

Turn websites into data. Crawly spiders and extracts complete structured data from an entire website.

Browse AI

Train a robot to scrape any website in 2 mins with no-code
Pricing starts at USD 49/month. Limited free plan available

Scrape Owl

Simple and powerful web scraping API that manages proxies, headless browsers, and HTML parsing. Simply specify the website and the element you want
Pricing starts at USD 29/month

Web Automation

the largest marketplace to find ready-made no code web scrapers. With only a few clicks and a few seconds you can start extracting data from your favourite site without coding or building from scratch
Pricing starts at USD 49/month

EasyScrape

With EasyScrape you can scrape the articles from any webpage with just one click. You can upload a text file containing a list of URLs or a single URL or a keyword, articles will be scraped from those URLs with just a click.