n8n
This guide explains how to connect your scraper to n8n to automate workflows
n8n Integration
n8n is an open-source workflow automation tool that lets teams connect apps, services, and APIs using a visual, node-based interface. Similar to Zapier or Make, n8n automates repetitive tasks and builds workflows without custom code.
Workflows in n8n run automatically based on triggers such as schedules, webhooks, or events from connected tools.
Overview
The MrScraper n8n integration enables you to:
- Extract data on demand - Run one-off scrapes by prompt, preset schema, or raw rendered HTML
- Discover URLs - Crawl a website for links or pull Google search results
- Create scrapers - Set up reusable AI scrapers directly from your workflow
- Run scrapers - Trigger existing AI or manual scrapers with a single URL or in batch
- Get results - Retrieve scraped data including latest results, paginated results, or a specific result by ID
- Monitor your account - Check token usage and limits
Why Use This Integration?
Integrating MrScraper with n8n enables fully automated data pipelines:
- Automatically create and run scrapers on a schedule or trigger
- Fetch and process scraping results programmatically
- Send scraped data to other tools (Google Sheets, databases, APIs, webhooks, notification systems)
- Build end-to-end workflows by connecting MrScraper with hundreds of n8n-supported services
This transforms scraping from a standalone task into a seamless part of broader automation workflows.
Use MrScraper as an AI Agent tool
The MrScraper node is tool-enabled. You can attach it to an n8n AI Agent node and let the model call MrScraper operations on its own, instead of wiring them into a fixed workflow path.
Prerequisites
Before you start, ensure you have:
- A MrScraper API token
- A MrScraper scraper with API access enabled (for Scraper Run operations)
- Access to an n8n instance (self-hosted or cloud)
The node authenticates with a single API Token credential, sent as an x-api-token header.
Understanding MrScraper Resources
The MrScraper node in n8n groups its actions into six Resources. Understanding these resources will help you choose the right one for your workflow.
Renamed in recent versions
If you built workflows against an earlier version of the node, resource and operation names have changed. See What changed for the full mapping.
| Resource | What it does | Runs immediately? |
|---|---|---|
| Account | Read account details, token usage and limits | Yes |
| Discovery | Find URLs by crawling a site or searching Google | Yes |
| Extraction | One-off scraping by prompt, preset schema, or raw HTML | Yes |
| Result | Fetch data produced by your scrapers | Yes |
| Scraper Creation | Create a reusable scraper in your MrScraper account | Yes |
| Scraper Run | Run an existing scraper on new URLs | Yes |
Account
Retrieve your MrScraper account information, including account type, usage limits, and token consumption.
Operation: Get Account Info
Use Case: Monitor account status and usage in automated workflows. This operation takes no parameters.
Discovery
Find URLs and search results to feed into the rest of your workflow.
Discover URLs by crawling links from a starting website.
Best for: Site mapping, URL discovery, building link inventories before a detail-page scrape.
Parameters:
| Parameter | Required | Default | Description |
|---|---|---|---|
| URL | Yes | — | Starting URL for the crawl |
| Max Depth | No | 2 | How many levels deep to follow links |
| Max Pages | No | 50 | Maximum pages to evaluate during discovery |
| Limit | No | 50 | Maximum number of results to return |
| Include Patterns | No | — | Pipe-separated regex for URLs to include |
| Exclude Patterns | No | — | Pipe-separated regex for URLs to exclude |
Pattern syntax
Include and exclude patterns are regular expressions separated by |, for example ^https://www\.example\.com/blog/|^https://www\.example\.com/products/.
Fetch Google search results as JSON or HTML through the synchronous MrScraper SERP API.
Best for: Keyword monitoring, competitor tracking, seeding a workflow with search results.
Parameters:
| Parameter | Required | Default | Description |
|---|---|---|---|
| Search Query | Yes | — | Google search terms, e.g. best hotels in New York |
| Region | Yes | us | Two-letter country/region code for localized results |
| Language | Yes | en | Two-letter result language code |
| Page | Yes | 1 | Google results page number |
| Format | No | JSON | JSON for parsed results, HTML for the raw results page |
| Render JavaScript | No | false | Render JavaScript before collecting results |
Extraction
Run a scrape and get data back immediately, without creating a persistent scraper first.
Extract data from a single page using your own prompt.
Best for: Product detail pages, article pages, profile pages, single-item extraction.
Parameters:
| Parameter | Required | Default | Description |
|---|---|---|---|
| URL | Yes | — | The target URL to scrape |
| Prompt | No | — | Instructions for what data to extract |
| Expected Output Schema | No | — | JSON describing the expected output shape |
| Mode | No | Super | Cheap for weak security, Super for stronger protection. Learn more |
| Proxy Country | No | — | ISO country code for the proxy, e.g. US, GB, ID, SG |
Expected Output Schema
The schema is not sent as a separate field. It is stringified and appended to your prompt, so the agent returns JSON matching the shape you described. Example: {"name":"string","price":"number","inStock":"boolean"}.
Extract repeated items across one or more pages of a listing.
Best for: Product category pages, search results, directory pages, multi-page listings.
Parameters:
| Parameter | Required | Default | Description |
|---|---|---|---|
| URL | Yes | — | The target URL to scrape |
| Prompt | No | — | Instructions for what to extract from each listing page |
| Expected Output Schema | No | — | JSON describing each expected listing item |
| Max Pages | No | 1 | Maximum pagination pages to scrape |
| Proxy Country | No | — | ISO country code for the proxy |
Extract data using a preset schema instead of writing your own prompt.
Best for: Common page types where a standard set of fields is enough.
Parameters:
| Parameter | Required | Default | Description |
|---|---|---|---|
| URL | Yes | — | The target URL to scrape |
| Structured Data Category | Yes | Article | Preset extraction schema |
| Mode | No | Super | Cheap or Super scraping mode |
| Proxy Country | No | — | ISO country code for the proxy |
Available Categories: Article, Forum Thread, Hotel, Job Posting, Post, Product, Property, Restaurant, Social Media Profile, Tour / Attraction.
Fetch the rendered HTML of a page through the MrScraper stealth browser, with JavaScript execution, bot evasion, and optional geo proxy.
Best for: Troubleshooting scraping issues, retrieving page source, or when you need raw HTML or Markdown rather than structured extraction.
Parameters:
| Parameter | Required | Default | Description |
|---|---|---|---|
| URL | Yes | — | Target URL to fetch |
| Max Retries | No | 3 | Retry attempts when the request fails |
| Timeout | No | 300 | Maximum seconds to wait for the page to load |
| Geo Code | No | us | Country code used for geolocation |
| Proxy Country | No | us | Country code for the proxy location |
| Screenshot | No | false | Capture and return a screenshot |
| Screenshot Mode | No | Full | Full for the entire page or Top for the top only. Appears once Screenshot is enabled |
| Return HTML | No | true | Include the rendered HTML in the response |
| Return Markdown | No | false | Include Markdown converted from the rendered page |
Advanced Options:
Click Add Option to reach these.
| Option | Default | Description |
|---|---|---|
| Token Cap | 30 | Maximum token allowance for processing the scraped content |
| Wait for Selector | — | CSS selector to wait for before returning the page |
| Wait Until | DOM Content Loaded | Browser lifecycle event to wait for: DOM Content Loaded, Load, or Network Idle |
| Block Resources | true | Block images, fonts, and stylesheets for faster, cheaper loads |
| Home Page | false | Navigate via the site's home page before the target URL |
| Return Cookie | true | Include browser cookies in the response |
| Super | true | Use a real device for sites requiring stronger capabilities |
Browser rendering is always on
This operation always renders the page in a browser, so JavaScript content loads. There is no toggle to disable it.
Result
Retrieve data produced by your scrapers. This is typically the final step in a scraping workflow, where you fetch the data to send to other systems.
Retrieve paginated results with sorting.
Best for: Large result sets that need pagination or specific sorting.
Parameters:
| Parameter | Required | Default | Description |
|---|---|---|---|
| Scraper ID | Yes | — | ID of the scraper whose results to fetch |
| Page | Yes | 1 | Page number for pagination |
| Page Size | Yes | 10 | Number of results per page |
| Sort By | Yes | Created At | Field used to sort results |
| Sort Order | Yes | Descending | Ascending or Descending |
Retrieve the most recent results for a scraper.
Best for: Monitoring workflows where you only need the latest data.
Parameters:
| Parameter | Required | Default | Description |
|---|---|---|---|
| Scraper ID | Yes | — | ID of the scraper whose latest results to fetch |
| N | Yes | 10 | Number of latest results to fetch |
Retrieve a specific result by its ID.
Best for: Fetching a known result, or following up on a batch run.
Parameters:
| Parameter | Required | Default | Description |
|---|---|---|---|
| Result ID | Yes | — | Unique result ID to retrieve |
Tips
Retrieve batch run results by passing the batch operation ID to this operation. A Scraper ID is not required here.
Common Use Case
Result operations are commonly used to pass scraped data to other n8n nodes like Google Sheets, databases, webhooks, or notifications.
Scraper Creation
Create a persistent scraper in your MrScraper account that can be reused and triggered multiple times with the Scraper Run resource.
Use Case: When you need a reusable scraper configuration that you'll run repeatedly with different URLs.
Create an AI scraper from a URL, an extraction prompt, and an expected JSON output schema.
Parameters:
| Parameter | Required | Default | Description |
|---|---|---|---|
| URL | Yes | — | The target URL to scrape |
| Prompt | No | — | Instructions for what data to extract |
| Expected Output Schema | No | — | JSON describing the expected output shape |
| Mode | No | Super | Cheap or Super scraping mode |
| Proxy Country | No | — | ISO country code for the proxy |
Create an AI scraper for repeated listing data using a prompt and an expected JSON output schema.
Parameters:
| Parameter | Required | Default | Description |
|---|---|---|---|
| URL | Yes | — | The target URL to scrape |
| Prompt | No | — | Instructions for what to extract from each item |
| Expected Output Schema | No | — | JSON describing each expected listing item |
| Max Pages | No | 1 | Maximum pagination pages to scrape |
| Proxy Country | No | — | ISO country code for the proxy |
Create a scraper that discovers URLs by crawling a website.
Parameters:
| Parameter | Required | Default | Description |
|---|---|---|---|
| URL | Yes | — | Starting URL for the crawl |
| Max Depth | No | 2 | How many levels deep to follow links |
| Max Pages | No | 50 | Maximum pages to evaluate during discovery |
| Limit | No | 50 | Maximum number of results to return |
| Include Patterns | No | — | Pipe-separated regex for URLs to include |
| Exclude Patterns | No | — | Pipe-separated regex for URLs to exclude |
Scraper Run
Run an existing scraper again with new URLs. Requires a scraper created through Scraper Creation or in the MrScraper dashboard.
Operations:
- Run Existing Scraper - run one URL
- Run Existing Scraper in Batch - run multiple URLs in a single request
Instead of a separate operation per agent type, you pick the scraper's shape with two selectors:
| Selector | Values | Shown when |
|---|---|---|
| Scraper Type | AI, Manual | Always |
| Agent Type | General, Listing, Map | Scraper Type is AI and operation is Run Existing Scraper |
Important
The selectors must match how the scraper was built. Choosing Manual routes the request to the manual-scraper endpoint, and each Agent Type exposes a different set of run settings.
Run Existing Scraper
Every single run takes these fields:
| Parameter | Required | Default | Description |
|---|---|---|---|
| Scraper ID | Yes | — | ID of the existing scraper, from the scraper detail page |
| URL | Yes | — | Full URL to process in this run |
| Max Retry | No | 3 | Maximum retry attempts if the run fails |
| Proxy Country | No | — | Proxy country code, e.g. us or uk |
Additional fields depend on the selected type:
No extra required fields beyond the common ones.
Options:
| Option | Default | Description |
|---|---|---|
| Bypass Proxy | false | Block images, fonts, and stylesheets to speed up scraping |
| HTML | false | Include HTML in the result |
| Markdown | false | Include Markdown in the result |
| Render JavaScript | false | Render JavaScript before extracting content |
| Return Cookies | false | Include browser cookies in the result |
| Screenshot | false | Capture a screenshot during the run |
| Use Home Page | false | Visit the site's home page first. Improves success rate on some sites, but adds latency — enable only if the scraper is blocked |
| Wait for Selector | — | CSS selector to wait for before extraction |
Extra parameters:
| Parameter | Required | Default | Description |
|---|---|---|---|
| Max Pages | No | 5 | Maximum pagination pages to scrape |
| Timeout | No | 300 | Maximum seconds to wait for the listing run |
Options: the same set as AI · General, plus:
| Option | Default | Description |
|---|---|---|
| Stream | false | Stream listing results as they become available |
Extra parameters:
| Parameter | Required | Default | Description |
|---|---|---|---|
| Max Depth | No | 2 | Maximum link depth to crawl from the starting URL |
| Max Pages | No | 50 | Maximum pages to evaluate during URL discovery |
| Limit | No | 50 | Maximum number of results to return |
| Include Patterns | No | — | Pipe-separated regex for URLs to include |
| Exclude Patterns | No | — | Pipe-separated regex for URLs to exclude |
Map runs do not expose an Options collection.
Manual scrapers have their own Options set, covering browser session control and pagination.
Options:
| Option | Default | Description |
|---|---|---|
| Bypass Proxy | true | Block images, fonts, and stylesheets to speed up scraping |
| Cookie Jar | — | Cookie jar identifier or serialized cookie jar value |
| Cookies | [] | JSON array of browser cookie objects for this run |
| Home Page | false | Visit the site's home page before the target URL |
| Home Page Timeout | 10 | Maximum seconds to wait for the home page |
| HTML | false | Include HTML in the result |
| Markdown | false | Include Markdown in the result |
| Paginator | {} | JSON pagination configuration, e.g. {"selector":"a.next","maxPages":5} |
| Proxy | — | Proxy URL used for this run |
| Record | false | Record the browser session |
| Return Cookie | false | Include browser cookies in the result |
| Screenshot | false | Capture a screenshot and return it Base64-encoded |
| Stream | false | Stream results as they become available |
| Timeout | 600 | Maximum seconds to wait for the run |
| Token Cap | 0 | Maximum token count for the result; 0 means no explicit limit |
Run Existing Scraper in Batch
Run multiple URLs against one existing scraper in a single operation.
Use Case: Scrape many product pages, profiles, or articles with the same scraper configuration without creating separate workflow nodes.
Parameters:
| Parameter | Required | Description |
|---|---|---|
| Scraper Type | Yes | AI or Manual, matching how the scraper was built |
| Scraper ID | Yes | ID of your AI or manual scraper |
| URLs | Yes | The URLs to scrape |
URL formats accepted
The URLs field accepts a JSON array (["https://example.com/a", "https://example.com/b"]), or a comma- or newline-separated list. The value is normalized to an array before it is sent.
Tips
Retrieve batch results by passing the batch operation ID to the Get Result Detail operation.
What Changed
Resource and operation names were reorganized in recent versions of the node. If you are updating older workflows or older notes, use this mapping:
| Previously | Now |
|---|---|
| Agent resource | Extraction resource |
| Scraping resource | Extraction resource |
| Web Unblocker resource | Extraction → Fetch Rendered HTML |
| Batch Operation resource | Scraper Run → Run Existing Scraper in Batch |
| Create Scraper resource | Scraper Creation resource |
| Rerun Scraper resource | Scraper Run resource |
| Results resource | Result resource |
| Scrape Dynamic Content | Extract Page by Prompt |
| Scrape Paginated Content | Extract Listings and Paginated Content |
| Scrape Structured Data | Extract Structured Data |
| Scrape Web Page | Fetch Rendered HTML |
| Crawl Website Sitemap | Discovery → Crawl Website URLs |
| Scrape Search Results | Discovery → Search Google SERP (now a real Google SERP API) |
| Run General / Listing / Map / Manual Agent Scraper (4 operations) | Run Existing Scraper + Scraper Type / Agent Type selectors |
New in the current node:
- Discovery resource, including the synchronous Google SERP API
- Expected Output Schema on prompt-based and listing operations
- Options collections on Scraper Run (screenshot, HTML, Markdown, cookies, streaming, wait-for-selector, and more)
- An Advanced Options collection on Extraction → Fetch Rendered HTML
- The node is usable as an AI Agent tool
Fetch Rendered HTML was also reshuffled. If you configured it on an earlier node version:
Token Cap,Block Resources,Wait Until,Return Cookie, andSupermoved into Advanced OptionsBrowser Renderingwas removed — the page is always rendered in a browserScreenshotis now a toggle plus a separateScreenshot Mode(Full/Top), instead of a text fieldWait Untilis now a dropdown rather than free textReturn Markdownnow defaults tofalse
Understanding n8n Upstream Node Execution
When you execute a node in n8n, all upstream nodes automatically re-execute silently. With MrScraper, this means hidden token costs.
Common example workflow:
Create Scraper → Run Scraper → Get ResultsWhat happens when you only want to test the Get Results node:
You click "Test" on Get Results, but n8n silently executes ALL upstream nodes:
- Scraper Creation runs → Creates new scraper
- Scraper Run runs → Runs the scraper
- Result runs → Retrieves results
Every test click on Get Results creates a brand new scraper. Testing 10 times = 10 new scrapers created and more tokens burned.
Setup Guide
Now that you understand the available resources, let's set up your first MrScraper workflow.
Add the MrScraper Node
- Open the n8n workflow editor
- Click the + button to add a new node
- Search for MrScraper
- Select the MrScraper node
Configure Credentials
- In Credential to connect with, click Create new credential
- Paste your MrScraper API token
- Click Save
Choose Your Resource and Configure
- Select the Resource that matches your use case (see Understanding MrScraper Resources above)
- Select the Operation
- Fill in the required parameters, then add optional ones as needed
Test and Execute
- Click Test step to verify your configuration
- Review the returned data
- Connect the output to other nodes in your workflow
Quick Tip
Start with the Extraction resource for testing and one-off scraping. Once you have a working configuration, use Scraper Creation to save it for reuse with the Scraper Run resource.
Example Workflows
Create a Scraper
Create a scraper with the MrScraper n8n node and export results to Google Sheets. Use the generated outputs as reusable inputs for building end-to-end scraping workflows.
Listing Agent + General Agent
Automate data extraction from real estate listing websites using a two-agent approach.
Map Agent + General Agent
Scrape entire websites using a two-agent approach for comprehensive data collection.
Map + Listing + General Agent
Comprehensive website scraping combining three powerful agents for complete site coverage.
Prebuilt Workflow Templates
MrScraper provides ready-to-deploy n8n workflow templates for common automation use cases. Each template is built around real-world scenarios and can be deployed in minutes.
Quick Start
Select the template that fits your use case, follow the setup guide, and you'll have a working automation running in minutes.
Monitor Realtor Listings
Scrape Realtor.com listings on a schedule and receive formatted CSV or XLSX reports via Gmail.
Turn Google Sheets into Website Database
Pull structured data from any website and append it automatically to a live Google Sheet.
Extract Product Details from Search URLs
Extract product names, prices, and ratings from a batch of search result URLs into Google Sheets.
Documentation Support Chatbot
Crawl your documentation site and power a GPT-4.1-mini chatbot that answers user questions accurately.
Monitor News Sentiment Trends
Scrape news articles, analyze sentiment with GPT-4o-mini, and receive Slack digests on coverage shifts.
Monitor Ecommerce Reviews
Track platform reviews with GPT-4o-mini and receive Slack alerts when negative patterns emerge.