Python SDK

The mrscraper-sdk is a typed Python client for the MrScraper API. It exposes three methods: scraper_api to fetch page content (HTML, Markdown, or screenshots), web_search to retrieve real-time web search results, and account to check your subscription and token usage. All client methods are async and must be used with asyncio or another async runtime.

Requirements

• Python 3.9 or later
• Basic familiarity with async/await syntax in Python

Installation

Install the SDK from PyPI using pip:

pip install --upgrade mrscraper-sdk

Import the client in your Python code:

from mrscraper import MrScraper

Authentication

The client reads your API token from the MRSCRAPER_API_TOKEN environment variable automatically. You can get your API token from the MrScraper dashboard.

export MRSCRAPER_API_TOKEN="MRSCRAPER_API_TOKEN"

from mrscraper import MrScraper

client = MrScraper()  # uses MRSCRAPER_API_TOKEN

You can also pass the token explicitly:

client = MrScraper(token="MRSCRAPER_API_TOKEN")

import httpx
from mrscraper import MrScraper

async with httpx.AsyncClient() as http:
    client = MrScraper(http_client=http)
    result = await client.scraper_api("https://example.com")

For more information on generating API tokens, see the Generate Token guide.

Quick Start

import asyncio
from mrscraper import MrScraper

async def main():
    client = MrScraper()

    # 1. Fetch page content
    page = await client.scraper_api("https://example.com")
    print(page["data"])

    # 2. Search the web
    search = await client.web_search("python web scraping", country="us")
    print(search["data"]["data"])

    # 3. Check remaining tokens
    account = await client.account()
    print(account["data"]["token_remaining"])

asyncio.run(main())

Response Format

All methods return a dictionary with the same structure:

Core Methods

Scraper API

Use scraper_api to fetch a page through the MrScraper Scraper API. The service handles JavaScript rendering, bot detection, and optional geolocation proxying.

import asyncio
from mrscraper import MrScraper

async def main():
    client = MrScraper()

    # HTML only (the default) returns the page as a string
    page = await client.scraper_api("https://example.com/product")
    print(page["data"])

    # Markdown or a screenshot returns a dictionary with one entry per output
    result = await client.scraper_api(
        "https://example.com/product",
        html=True,
        markdown=True,
        screenshot=True,
        geo_code="US",
        timeout=120,
    )
    print(result["data"]["markdown"])

asyncio.run(main())

Requesting only HTML returns the page as a string. Requesting Markdown or a screenshot returns a dictionary with an entry for each requested output, such as html, markdown, and screenshot. At least one of html, markdown, or screenshot must be True.

Parameters:

For more examples of what you can fetch, see Scrape HTML, Scrape Markdown, and Get Screenshot.

Web Search

Use web_search to search the web and retrieve real-time search results, including relevant website URLs, titles, and descriptions. The first argument is the search query.

import asyncio
from mrscraper import MrScraper

async def main():
    client = MrScraper()

    result = await client.web_search(
        "python web scraping",
        country="us",
        num_results=10,
    )
    for item in result["data"]["data"]:
        print(item["rank"], item["title"], item["url"])

asyncio.run(main())

The SDK sends this request:

curl --location 'https://api.mrscraper.com' \
--header 'x-api-token: {MRSCRAPER_API_TOKEN}' \
--header 'Content-Type: application/json' \
--data '{
    "action": "websearch",
    "query": "python web scraping",
    "country": "us",
    "numResults": 10
}'

Parameters:

Response (result["data"]):

Account

Use account to check the identity, subscription, token quota, and rate limits of the account that owns the API token.

import asyncio
from mrscraper import MrScraper

async def main():
    client = MrScraper()

    result = await client.account(timeout=30)
    print(result["data"]["subscription_status"])
    print(result["data"]["token_remaining"])
    print(result["data"]["user"]["email"])

asyncio.run(main())

Parameters:

Account summary (result["data"]):

Error Handling

The SDK raises typed exceptions, which you can import from mrscraper or mrscraper.exceptions.

Exception Types

Invalid arguments, such as a timeout outside 1-300 or a num_results outside 10-100, raise ValueError before any request is sent.

Example Error Handling

from mrscraper import MrScraper, AuthenticationError, APIError, NetworkError

client = MrScraper()

try:
    result = await client.scraper_api("https://example.com")
except AuthenticationError:
    print("Authentication failed. Check your API token at https://app.mrscraper.com")
except APIError as e:
    print(f"API error {e.status_code}: {e}")
except NetworkError as e:
    print(f"Network problem: {e}")

Best Practices

• Always wrap SDK calls in try-except blocks
• Log errors with appropriate context for debugging
• Implement retry logic for NetworkError exceptions
• Verify your API token if you encounter AuthenticationError

Migrating from Earlier Versions

From 0.7.x

Version 0.9.0 returns to three public methods. The 0.7.x format-specific methods are no longer public; use scraper_api with output flags instead:

From 0.6.0

• web_search now calls the MrScraper API with action: "websearch" and returns JSON. Its first argument is query, and it accepts country and num_results (10 to 100). The raw, format, region, language, page, and render_js options and search URL input were removed. Results are at result["data"]["data"].
• scraper_api adds the wait_until option. At least one of html, markdown, or screenshot must be True. HTML-only requests return a string, and requests with Markdown or a screenshot return a dictionary with one entry per output.
• account is unchanged.

From 0.5.0

Version 0.6.0 adds the account method. No existing methods changed.

From 0.4.0

Version 0.5.0 renames a public method. Arguments and response formats are unchanged.

The previous method names have been removed.

From 0.3.x and Earlier

fetch_html and the AI scraper, manual scraper, results, and account methods (create_scraper, rerun_scraper, bulk_rerun_ai_scraper, rerun_manual_scraper, bulk_rerun_manual_scraper, get_all_results, get_result_by_id, get_analytic_statuses) are no longer part of the SDK. get_subscription_account is replaced by account.

• Replace fetch_html with scraper_api. It returns a dictionary with status_code, data, and headers.
• For scrapers, results, and analytics, call the REST API directly.

Migration from Firecrawl

If you're migrating from the Firecrawl Python SDK, use this mapping to find equivalent MrScraper methods.