Usage Analytics

The Usage Analytics dashboard provides real-time insights into your scraping operations, performance metrics, and token consumption. Monitor your scraper activity, identify trends, and optimize your scraping workflows with comprehensive analytics.

Overview

Access the Usage Analytics dashboard from your MrScraper account by navigating to the Analytics section in the sidebar. The dashboard displays key performance indicators and historical data to help you understand your scraping patterns and resource usage.

Dashboard Tabs

The analytics dashboard is organized into three main sections:

Filtering Options

Standard Filters

All users have access to the following filter options:

Time Range Filter

Select the time period for your analytics:

• Last 1 Hour
• Last 24 Hours
• Last 7 Days
• Last 30 Days
• Custom Date Range

Timezone Filter

View analytics in your preferred timezone:

• WIB (GMT+7) - Asia/Jakarta
• UTC
• Your local timezone
• Other timezones

API Token Filter

Filter analytics by specific API tokens to track usage per integration or application.

Enterprise Filters

Action Filter (Enterprise Only)

Filter analytics by specific scraping actions:

• Create Scraper - Track scraper creation operations
• Rerun Scraper - Monitor scraper rerun activity
• Bulk Operations - Analyze bulk scraping jobs
• Fetch HTML - View raw HTML fetch requests
• All Actions - See combined activity across all action types

This allows enterprise customers to:

• Identify which operations consume the most tokens
• Optimize workflows based on action-specific metrics
• Track usage patterns by operation type
• Create detailed cost allocation reports

Domain/Target Filter (Enterprise Only)

Filter by specific domains or targets:

• View metrics for individual websites
• Track success rates per domain
• Monitor domain-specific latency
• Analyze which targets require the most resources

Key Metrics

The dashboard displays four primary performance indicators:

Average Latency

What it measures: The average response time for your scraping requests.

Display: Clock icon with response time in milliseconds or seconds

Use cases:

• Identify slow-performing targets
• Optimize scraper configurations
• Monitor infrastructure performance
• Set realistic timeout values

Request Per Minute

What it measures: The number of scraping requests processed per minute.

Display: Lightning bolt icon with request count

Use cases:

• Monitor API rate limits
• Track scraping velocity
• Identify peak usage periods
• Plan capacity requirements

Rate Limits by Plan:

Success Rate

What it measures: The percentage of successful scraping operations.

Display: Green checkmark icon with percentage

Formula: (Successful Requests / Total Requests) × 100

Use cases:

• Evaluate scraper reliability
• Identify problematic targets
• Monitor configuration effectiveness
• Track improvements over time

Typical Success Rates:

Run Status Breakdown

Track the status of all your scraping operations:

Total Runs

Display: Chart icon with total count

What it shows: Total number of scraping operations in the selected time period

Includes: All runs regardless of status (success, pending, failed)

Success

Display: Green checkmark icon with count

What it shows: Number of successfully completed scraping operations

Definition: Requests that:

• Completed without errors
• Returned valid data
• Met all scraping criteria
• Consumed expected tokens

Pending

Display: Orange clock icon with count

What it shows: Number of scraping operations currently in progress

Includes:

• Queued requests waiting to start
• Active scraping operations
• Processing results
• Bulk operations in progress

Failed

Display: Red X icon with count

What it shows: Number of failed scraping operations

Common Failure Reasons:

• Target website returned errors (404, 500, etc.)
• Timeout exceeded
• Authentication failed
• Invalid scraper configuration
• Anti-scraping measures blocked the request
• Network connectivity issues

Troubleshooting Failed Requests:

1. Check the error details in the Results page
2. Verify the target URL is accessible
3. Review scraper configuration
4. Test with different proxy settings
5. Contact support if issues persist

Usage Analytics Chart

The Usage Analytics section displays a time-series graph showing your scraping activity over time.

What the Chart Shows

Scraping Activity Over Time

• X-axis: Time period (hours, days, weeks)
• Y-axis: Number of scraping operations
• Data points: Individual scraping requests

Color Coding:

• Green areas: Successful requests
• Red areas: Failed requests
• Orange areas: Pending requests

Using the Chart

Identify Patterns:

• Peak usage hours
• Recurring scraping schedules
• Unusual activity spikes
• Downtime periods

Optimize Operations:

• Schedule bulk operations during off-peak hours
• Identify the best times for scraping
• Avoid peak hours for rate-limited targets
• Plan capacity based on usage trends

Monitor Health:

• Spot sudden drops in success rates
• Identify failing patterns
• Track recovery after issues
• Validate configuration changes

Accessing Analytics via API

Enterprise customers can access analytics data programmatically using our API endpoints.

Status Endpoint

GET https://api.app.mrscraper.com/api/v1/analytic/statuses?token=MRSCRAPER_API_KEY

Returns:

• Total runs
• Success count
• Pending count
• Failed count
• Success rate percentage

Timeline Endpoint

GET https://api.app.mrscraper.com/api/v1/analytic/timeline?token=MRSCRAPER_API_KEY&startDate=2024-01-01&endDate=2024-12-31

Returns:

• Time-series data for the specified period
• Request counts by hour/day/week
• Success/failure breakdown
• Latency metrics

See our Authentication documentation for details on analytics API authentication.

Best Practices

Monitor Regularly

• Daily: Check success rates and pending requests
• Weekly: Review usage trends and optimize configurations
• Monthly: Analyze token consumption and plan capacity

Set Up Alerts

Configure notifications for:

• Success rate drops below threshold
• Unusual spike in failed requests
• Token usage approaching limits
• Pending requests backlog

Use Filters Effectively

• Time Range: Start broad, then narrow down to specific periods
• API Tokens: Track usage per integration separately
• Actions (Enterprise): Identify high-cost operations
• Domains (Enterprise): Monitor per-target performance

Optimize Based on Data

If success rate is low:

• Review failed request details
• Adjust proxy settings
• Update scraper configurations
• Consider using Super mode for difficult targets

If latency is high:

• Check target website performance
• Use closer proxy regions
• Reduce page complexity
• Optimize extraction patterns

If token usage is high:

• Review scraper efficiency
• Eliminate unnecessary requests
• Use caching where possible
• Consider bulk operations

Understanding Token Consumption

Token Usage by Operation

Different operations consume different amounts of tokens:

Tracking Token Usage

Monitor your token consumption in the Token Usage tab:

• Daily usage trends
• Breakdown by scraper
• Breakdown by API token
• Projection to month-end
• Remaining balance