Web Unblocker

MrScraper's Web Unblocker is a powerful feature that helps you bypass anti-scraping measures, render JavaScript-heavy websites, and access geo-restricted content. It combines browser rendering, residential proxies, and intelligent retry mechanisms to ensure reliable data extraction from even the most protected websites.

Key Features

Browser Rendering

Execute JavaScript and render dynamic content just like a real browser.

Use Cases:

• Single Page Applications (SPAs) built with React, Vue, or Angular
• Websites that load content via AJAX
• Pages with lazy-loaded images or infinite scroll
• Dynamic pricing or availability information
• Client-side rendered content

Geo-Targeting with Proxies

Access geo-restricted content by routing requests through residential proxies in specific countries.

Use Cases:

• Scraping region-specific pricing
• Accessing country-locked content
• Testing localized versions of websites
• Bypassing IP-based restrictions
• Market research across different regions

Smart Waiting

Wait for specific elements to appear in the DOM before extracting content.

Use Cases:

• Waiting for lazy-loaded product information
• Ensuring dynamic filters have applied
• Waiting for search results to populate
• Synchronizing with page animations
• Handling delayed content rendering

Homepage Navigation

Navigate through a website's home page before visiting the target URL to better simulate normal user behavior.

Use Cases:

• Websites that expect visitors to arrive from the homepage
• Reducing bot detection on protected websites
• Improving access to sites with navigation-based validation

Intelligent Retry Control

Automatically retry failed scrapes while limiting retry costs with configurable retry settings.

Use Cases:

• Recovering from temporary network failures
• Handling intermittent anti-bot challenges
• Preventing excessive token usage with retry limits
• Balancing reliability and scraping cost

Resource Blocking

Block non-essential resources such as images, fonts, and media to improve scraping speed and reduce bandwidth usage.

Use Cases:

• Faster page rendering
• Lower bandwidth consumption
• Scraping text-heavy pages
• Large-scale scraping jobs

Configurable Timeout

Control how long to wait for page loading and rendering.

Use Cases:

• Slow-loading websites
• Complex pages with many resources
• Sites with large media files
• Pages with extensive JavaScript execution
• Unreliable network conditions

How It Works

The Web Unblocker follows this process:

1. Your request is routed through residential proxies in the specified region (if geo-targeting is enabled)
2. A headless browser instance is created with appropriate fingerprints and headers
3. The browser navigates to the target URL and begins rendering
4. All JavaScript on the page executes, including AJAX calls and dynamic content loading
5. If specified, the scraper waits for the target CSS selector to appear in the DOM
6. The process completes when the selector appears or the timeout is reached
7. The fully rendered HTML is extracted and returned
8. Browser resources are released

Example Request

Example request for scraping using the web unblocker endpoint:

curl -X GET "https://api.app.mrscraper.com/?url=https%3A%2F%2Fwww.amazon.com%2Fstores%2Fluxurystores%2Fpage%2FB6BC6264-7221-424B-9191-DAE2BCF963A2&token=atk_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx&timeout=120&geoCode=pt&browserRendering=true&waitForSelector=div%5Bdata-testid%3D%22grid-item-info%22%5D&homePage=false&blockResources=true&maxRetries=3&tokenCap=10" \
  -H "x-api-token: MRSCRAPER_API_KEY"

Parameters