n8n
This guide explains how to connect your scraper to n8n to automate workflows
n8n Integration
n8n is an open-source workflow automation tool that lets teams connect apps, services, and APIs using a visual, node-based interface. Similar to Zapier or Make, n8n automates repetitive tasks and builds workflows without custom code.
Workflows in n8n run automatically based on triggers such as schedules, webhooks, or events from connected tools.
Overview
The MrScraper n8n integration enables you to:
- Create Scrapers - Set up AI-powered scrapers directly from your workflow
- Rerun Scrapers - Trigger existing scrapers manually, on schedule, or from workflow events
- Get Results - Retrieve scraped data including latest results, paginated results, or specific results by ID
Why Use This Integration?
Integrating MrScraper with n8n enables fully automated data pipelines:
- Automatically create and rerun scrapers on a schedule or trigger
- Fetch and process scraping results programmatically
- Send scraped data to other tools (Google Sheets, databases, APIs, webhooks, notification systems)
- Build end-to-end workflows by connecting MrScraper with hundreds of n8n-supported services
This transforms scraping from a standalone task into a seamless part of broader automation workflows.
Prerequisites
Before you start, ensure you have:
- A MrScraper API key
- A MrScraper scraper with API access enabled (for rerun operations)
- Access to an n8n instance (self-hosted or cloud)
Understanding MrScraper Resources
The MrScraper node in n8n provides different Resources for various scraping operations. Understanding these resources will help you choose the right one for your workflow.
Account
Retrieve your MrScraper account information, including account type, usage limits, and token consumption.
Use Case: Monitor account status and usage in automated workflows.
Agent
Create a new AI-powered scraping agent. This resource creates agents that run and return results immediately.
Available Agent Types:
Extract structured data from single pages or listings
Best for:
- Product detail pages
- Article pages
- Profile pages
- Single-item data extraction
Parameters:
| Parameter | Required | Description |
|---|---|---|
| URL | Yes | The target URL to scrape |
| Prompt | Yes | Instructions for the agent on what data to extract |
| Mode | No | Cheap for weak security, Super for stronger protection. Learn more |
| Proxy Country | No | Country code for proxy (e.g., us, uk, sg) |
Extract data from paginated listings
Best for:
- Product category pages
- Search results
- Multi-page listings
- Directory pages
Parameters:
| Parameter | Required | Description |
|---|---|---|
| URL | Yes | The target URL to scrape |
| Prompt | Yes | Instructions for data extraction |
| Proxy Country | No | Country code for proxy |
Discover and extract all URLs from a website
Best for:
- Site mapping
- URL discovery
- Website crawling preparation
- Building link inventories
Parameters:
| Parameter | Required | Description |
|---|---|---|
| URL | Yes | Starting URL for crawling |
| Max Depth | No | How many levels deep to follow links |
| Max Pages | No | Maximum number of pages to crawl |
| Limit | No | Maximum number of results to return |
| Include Patterns | No | Regex patterns for URLs to include |
| Exclude Patterns | No | Regex patterns for URLs to exclude |
Batch Operation
Run multiple URLs against an existing AI or manual scraper in a single operation.
Use Case: Scrape multiple product pages, profiles, or articles using the same scraper configuration without creating separate workflow nodes.
Tips
Retrieve batch operation results by passing the batch operation ID as a parameter to the Get Result Details action node.
Parameters:
| Parameter | Required | Description |
|---|---|---|
| Mode | Yes | The scraper mode |
| Scraper ID | Yes | ID of your AI or Manual scraper (found in scraper URL or settings) |
| URLs | Yes | List of URLs to scrape |
Create Scraper
Create a persistent scraper in your MrScraper account that can be reused and triggered multiple times.
Usecase: When you need a reusable scraper configuration that you'll run multiple times with different URLs.
Available Scraper Types:
Create a reusable general agent scraper.
Parameters:
| Parameter | Required | Description |
|---|---|---|
| URL | Yes | The target URL to scrape |
| Prompt | Yes | Instructions for data extraction |
| Mode | No | Cheap or Super scraping mode |
| Proxy Country | No | Country code for proxy |
Create a reusable listing agent scraper.
Parameters:
| Parameter | Required | Description |
|---|---|---|
| URL | Yes | The target URL to scrape |
| Prompt | Yes | Instructions for data extraction |
| Proxy Country | No | Country code for proxy |
Create a reusable map agent scraper.
Parameters:
| Parameter | Required | Description |
|---|---|---|
| URL | Yes | Starting URL for crawling |
| Max Depth | No | Link depth to follow |
| Max Pages | No | Maximum pages to crawl |
| Limit | No | Maximum results to return |
| Include Patterns | No | Regex patterns to include |
| Exclude Patterns | No | Regex patterns to exclude |
Rerun Scraper
Trigger an existing scraper to run again with new parameters. This requires an existing scraper created through the Create Scraper resource or in the MrScraper dashboard.
Use Case: Run the same scraper configuration on different URLs or schedules without recreating the scraper.
Available for all agent types:
Parameters:
| Parameter | Required | Description |
|---|---|---|
| Scraper ID | Yes | ID of the scraper to run |
| URL | Yes | Target URL (overrides default) |
| Max Retry | No | Number of retry attempts on failure |
Parameters:
| Parameter | Required | Description |
|---|---|---|
| Scraper ID | Yes | ID of the scraper to run |
| URL | Yes | Target URL (overrides default) |
| Max Retry | No | Number of retry attempts on failure |
Parameters:
| Parameter | Required | Description |
|---|---|---|
| Scraper ID | Yes | ID of the scraper to run |
| URL | Yes | Target URL (overrides default) |
| Max Retry | No | Number of retry attempts on failure |
| Max Pages | No | Maximum number of pages to scrape |
| Timeout | No | Request timeout in seconds |
Parameters:
| Parameter | Required | Description |
|---|---|---|
| Scraper ID | Yes | ID of the scraper to run |
| URL | Yes | Target URL (overrides default) |
| Max Retry | No | Number of retry attempts on failure |
| Max Depth | No | Link depth to follow |
| Max Pages | No | Maximum pages to scrape |
| Limit | No | Maximum results to return |
| Include Patterns | No | Regex patterns to include |
| Exclude Patterns | No | Regex patterns to exclude |
Important
The rerun action must match the agent type used by the scraper. For example, if your scraper uses General Agent, select "Run General Agent Scraper" in the rerun operation.
Results
Retrieve data produced by your scrapers. This is typically the final step in a scraping workflow, where you fetch the data to send to other systems.
Available Operations:
Retrieve paginated results with filtering and sorting
Best for: Large result sets that need pagination or specific sorting
Parameters:
| Parameter | Required | Description |
|---|---|---|
| Scraper ID | Yes | ID of the scraper |
| Page | No | Page number to retrieve |
| Page Size | No | Number of results per page |
| Sort By | No | Field to sort by |
| Sort Order | No | ascending or descending |
Retrieve the most recent results
Best for: Monitoring workflows where you only need the latest data
Parameters:
| Parameter | Required | Description |
|---|---|---|
| Scraper ID | Yes | ID of the scraper |
| Limit (N) | No | Number of latest results to fetch |
Retrieve a specific result by ID
Best for: Fetching a known result or following up on a specific scraping operation
Parameters:
| Parameter | Required | Description |
|---|---|---|
| Scraper ID | Yes | ID of the scraper |
| Result ID | Yes | Unique result ID to retrieve |
Common Use Case
This action is commonly used to pass scraped data to other n8n nodes like Google Sheets, databases, webhooks, or notifications.
Scraping
Quick scraping operations for specific scenarios without creating persistent scrapers. These are pre-built actionss that run and return results immediately.
Available Operations:
Extract all URLs from a website
Parameters:
| Parameter | Required | Description |
|---|---|---|
| URL | Yes | Starting URL for crawling |
| Max Depth | No | Link depth to follow |
| Max Pages | No | Maximum pages to crawl |
| Limit | No | Maximum results to return |
| Include Patterns | No | Regex patterns to include |
| Exclude Patterns | No | Regex patterns to exclude |
Extract data using AI with custom prompts
Parameters:
| Parameter | Required | Description |
|---|---|---|
| URL | Yes | Target URL to scrape |
| Prompt | Yes | Instructions for data extraction |
| Mode | No | Cheap or Super scraping mode |
| Proxy Country | No | Country code for proxy |
Extract data from multi-page listings
Parameters:
| Parameter | Required | Description |
|---|---|---|
| URL | Yes | Target URL to scrape |
| Prompt | Yes | Instructions for data extraction |
| Max Pages | No | Maximum pages to scrape |
Extract data from search result pages
Parameters:
| Parameter | Required | Description |
|---|---|---|
| URL | Yes | Target URL to scrape |
| Prompt | Yes | Instructions for data extraction |
| Proxy Country | No | Country code for proxy |
Extract data using preset schemas
Parameters:
| Parameter | Required | Description |
|---|---|---|
| URL | Yes | Target URL to scrape |
| Structured Data Category | Yes | Preset schema (article, product, hotel, etc.) |
| Mode | No | Cheap or Super scraping mode |
| Proxy Country | No | Country code for proxy |
Available Categories: Article, Product, Hotel, Event, Recipe, Job Posting, and more.
Fetch rendered HTML via stealth browser
Parameters:
| Parameter | Required | Description |
|---|---|---|
| URL | Yes | Target URL to fetch |
| Timeout | No | Maximum seconds to wait for page load |
| Geo Code | No | ISO country code for proxy |
| Block Resources | No | Block images, CSS, and fonts for faster loading |
Web Unblocker
Get raw HTML content from a page, bypassing anti-scraping measures.
Use Case: Troubleshooting scraping issues, retrieving page source, or when you need raw HTML without structured data extraction.
Parameters:
| Parameter | Required | Description |
|---|---|---|
| URL | Yes | Target URL to fetch |
| Timeout | No | Maximum seconds to wait for page load |
| Geo Code | No | ISO country code for proxy |
| Block Resources | No | Block images, CSS, and fonts for faster loading |
Setup Guide
Now that you understand the available resources, let's set up your first MrScraper workflow.
Step 1: Add the MrScraper Node
- Open the n8n workflow editor
- Click the + button to add a new node
- Search for MrScraper
- Select the MrScraper node
Step 2: Configure Credentials
- In Credential to connect with, click Create new credential
- Paste your MrScraper API key
- Click Save
Step 3: Choose Your Resource and Configure
- Select the Resource that matches your use case (see Understanding MrScraper Resources above)
- Fill in the required parameters for your selected resource
- Configure any optional parameters as needed
Step 4: Test and Execute
- Click Test step to verify your configuration
- Review the returned data
- Connect the output to other nodes in your workflow
Quick Tip
Start with the Agent resource for testing and one-off scraping. Once you have a working configuration, use Create Scraper to save it for reuse with the Rerun Scraper resource.
Example Workflows
Create a Scraper
Create a scraper with the MrScraper n8n node and export results to Google Sheets. Use the generated outputs as reusable inputs for building end-to-end scraping workflows.
Listing Agent + General Agent
Automate data extraction from real estate listing websites using a two-agent approach.
Map Agent + General Agent
Scrape entire websites using a two-agent approach for comprehensive data collection.
Map + Listing + General Agent
Comprehensive website scraping combining three powerful agents for complete site coverage.
Prebuilt Workflow Templates
MrScraper provides ready-to-deploy n8n workflow templates for common automation use cases. Each template is built around real-world scenarios and can be deployed in minutes.
Quick Start
Select the template that fits your use case, follow the setup guide, and you'll have a working automation running in minutes.
Monitor Realtor Listings
Scrape Realtor.com listings on a schedule and receive formatted CSV or XLSX reports via Gmail.
Turn Google Sheets into Website Database
Pull structured data from any website and append it automatically to a live Google Sheet.
Extract Product Details from Search URLs
Extract product names, prices, and ratings from a batch of search result URLs into Google Sheets.
Documentation Support Chatbot
Crawl your documentation site and power a GPT-4.1-mini chatbot that answers user questions accurately.
Monitor News Sentiment Trends
Scrape news articles, analyze sentiment with GPT-4o-mini, and receive Slack digests on coverage shifts.
Monitor Ecommerce Reviews
Track platform reviews with GPT-4o-mini and receive Slack alerts when negative patterns emerge.