Troubleshoot URL Access
Diagnose and fix proxy access issues when a target website blocks or fails to respond to requests through MrScraper's Residential Proxy.
When a target website doesn't load through your proxy integration, it can be hard to tell whether the issue lies in the proxy configuration, or the target site's defenses. This guide walks you through a structured process to isolate the cause and resolve it.
Why target websites block proxy requests
Websites use several techniques to detect and block non-human traffic. Even with residential proxy IPs, a request can still fail if:
- Web Application Firewalls (WAF) like Cloudflare, Akamai, or AWS WAF inspect request patterns and block suspicious traffic.
- CAPTCHA challenges are triggered by rate limiting or behavioral analysis.
- Bot detection systems flag requests that lack proper browser fingerprints (headers, cookies, JavaScript execution).
- Geo-restrictions block access from certain countries or regions.
- Rate limiting blocks IPs that send too many requests in a short period.
- TLS fingerprinting identifies non-browser HTTP clients by their TLS handshake characteristics.
Understanding these causes helps you pick the right fix. The fastest way to start is to use Proxy Scraper to test whether the issue is on your side or the proxy side.
Test with Proxy Scraper
Proxy Scraper is a standalone scraping tool built into MrScraper. It sends requests through the same residential proxy network. This isolates the proxy path from your code path.
Go to app.mrscraper.com, sign in, and select Proxy from the left navigation menu. Then select the Proxy Scraper tab.
Enter a simple, reliable URL in the Target URL field. For example, https://www.google.com. Keep all advanced settings at their defaults. Click Run.
If Proxy Scraper returns an HTTP 200 response with page content, the proxy connection is working. Proceed to the next step.
If it returns an error, see When the connectivity test fails below.
Now enter the real target URL. Click Run with the same default settings.
Interpret the results
Use the table below to determine your next action based on the Proxy Scraper results:
| google.com result | Target URL result | What it means | Next action |
|---|---|---|---|
| ✅ Success | ✅ Success | The proxy path works. The website target detecting the scraping script. | You can check your proxy configuration or continue using our Proxy Scraper to access your website target. |
| ✅ Success | ❌ Failure | The proxy path works, but the target website is blocking the request. | Go to Bypass target website defenses. |
| ❌ Failure | ❌ Failure | The proxy path itself has an issue. | Go to When the connectivity test fails. |
| ❌ Failure | ✅ Success | Unlikely, but possible if google.com is blocked in the selected proxy country. Try a different proxy country. | Change the proxy country and retest. |
Check your configuration
If Proxy Scraper succeeds but your own code or tool fails, the issue is in how your integration sends requests. Check the following:
Proxy credentials
- Verify your username and password are correct and match the format described in the Authentication guide.
- Check for typos, extra spaces, or missing segments (e.g.,
-country-,-sessid-).
Request format
- Ensure you're using the correct proxy endpoint:
proxy.mrscraper.com:10000. - Verify the proxy protocol. MrScraper supports HTTP. Make sure your client is configured to use the HTTP protocol.
- Check that the target URL includes the protocol (
http://).
HTTP headers
- Some websites block requests that lack a proper
User-Agentheader. Make sure your client sends a realistic browser User-Agent. - Include standard headers like
Accept,Accept-Language, andAccept-Encodingto reduce the chance of being flagged as a bot.
Network and firewall
- Confirm your network allows outbound connections to
proxy.mrscraper.comon port10000. - If you're behind a corporate firewall or VPN, check whether proxy traffic is being intercepted or blocked.
Bypass target website defenses
If Proxy Scraper can reach google.com but fails on your target URL, the target website is actively blocking the request. Try the following escalation steps in order:
Try a different proxy country
Some websites only serve traffic from specific regions. Change the proxy country in Proxy Scraper to one that matches the target website's expected audience (e.g., us for a US-based website).
Enable Browser rendering
Toggle Browser rendering to On in the Proxy Scraper request options. This renders the page in a headless browser, executing JavaScript before extracting content. This is required for:
- Single-page applications (SPAs)
- Websites that load content dynamically via JavaScript
- Pages behind JavaScript-based challenges
Adjust timeout and retries
Increase the Timeout beyond the default 90 seconds, and set Max retries to 5 or higher. Some heavily protected websites are slow to respond or require multiple attempts.
Contact support
If none of the above steps resolve the issue, the target website may require custom handling. Contact us with the following details:
- The target URL that fails
- The Proxy Scraper result (screenshot or response)
- The proxy country you tested with
- The settings you've tried (Super, Browser rendering, mobile, timeout)
When the connectivity test fails
If Proxy Scraper cannot reach even a simple URL like google.com, check the following:
| Possible cause | What to check |
|---|---|
| Account or subscription issue | Verify your account is active and has remaining proxy bandwidth in the Analytics tab. |
| Service outage | Check if MrScraper is experiencing an outage. Try again in a few minutes. |
| Proxy country unavailable | Some proxy countries may have limited availability. Try a different country code from the supported countries list. |
| Browser or network issue | Try accessing app.mrscraper.com in an incognito/private window to rule out browser cache or extension interference. |
If the issue persists, contact support at support@mrscraper.com.