Create a Scraper
This guide walks you through creating a scraper using n8n node and exporting results to Google Sheets
This example workflow demonstrates how to:
- Create a scraper: Set up a new Scraper in MrScraper directly in n8n
- Create a spreadsheet: Automatically create a new Google Sheets spreadsheet and store the scraper ID and URL of the newly created scraper
Note
This workflow is designed as a foundational building block. By saving the scraper ID and URL to a sheet, you can later reference those values to build a full end-to-end scraping pipeline without needing to manually look up or re-enter scraper details.
Workflow Setup
Set Up the Manual Trigger
- Add a Manual Trigger node called "When clicking 'Execute workflow'".
- This allows you to run the workflow on demand whenever you need to create a new scraper.
Create a Scraper
- Add the MrScraper node and select, "Create general agent" or other agent that you need.
- In Credential to connect with, add your MrScraper API credentials:
- Click Create new credential
- Paste your MrScraper API key
- Save the credential
- Configure the scraper settings:
- Resource: Select Create Scraper
- Operation: Select the Agent you want to use
- URL: Enter the URL you want to scrape (e.g.,
https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html) - Prompt: Describe what data you want to extract (e.g., "please get all details data")
- Mode: Choose scraping mode:
- Select Cheap for websites with weak security
- Select Super for websites with strong protection (learn more)
- Proxy Country (optional): Choose a proxy country if needed (e.g.,
us,uk,sg)
This creates the Scraper along with its scraper ID and target URL.
Create a New Google Spreadsheet
- Add a Google Sheets node called "Create spreadsheet".
- Authenticate with your Google account:
- Click Create new credential
- Follow the OAuth flow to authorize n8n
- Configure the node:
- Resource: Select Spreadsheet
- Operation: Select Create
- Title (optional): Give your spreadsheet a name, or leave blank for auto-generated name
This creates a new Google Sheets spreadsheet to store the scraper ID and target URL.
Append Data to the Spreadsheet
- Add another Google Sheets node called "Append row in sheet".
- Use the same Google credentials from Step 3.
- Configure the node:
- Resource: Select Sheet
- Operation: Select Append
- Document: Select the spreadsheet created in the previous step
- Sheet: Select the sheet (typically "Sheet1")
- Columns: Choose Auto-map Input Data to automatically map all fields from the scraper data
The scraper data will be automatically appended to your new spreadsheet.