PDP Cache Agent
Pre-built agents that scrape Product Detail Pages (PDP) with lower cost, faster response, and more complete data from cached sources.
A PDP Cache Agent is a specialized agent created by MrScraper to scrape Product Detail Pages (PDP)—single-page content such as product details, articles, hotel pages, job postings, property listings, restaurant pages, social profiles, tours, and more.
PDP Cache agents exist because URLs are frequently requested by users. Those requests are logged. When a domain or URL pattern is requested often enough, MrScraper intelligently generates code to scrape that domain using common fields that users frequently ask for (by category). That agent is then uploaded to the Marketplace, so it becomes available for everyone. Because the data is pre-collected and cached, a PDP Cache Agent provides lower cost, faster response, and more complete data, with standardized fields by category (e.g., e-commerce products, articles, hotels).
Why Use a PDP Cache Agent
- Lower cost — Data is extracted with dedicated scrapers instead of AI, so you avoid AI parsing costs.
- Faster response — No AI parsing is used, and some steps are bypassed depending on domain complexity, so results are delivered faster.
- More complete data — Data is aggregated from multiple sources (HTML, APIs, structured data), not only from the page HTML.
- Standardized output — Common fields per category (e.g., Article, Product, Hotel) make integration and parsing easier.
When to Use the PDP Cache Agent
Use a PDP Cache Agent when:
- You need to scrape detail pages (product, article, hotel, job, property, restaurant, social profile, tour) from a supported domain.
- You want standardized fields and predictable output for a given category.
- Cost and speed matter and the target site has a pre-built PDP Cache agent in the Marketplace.
If the site is not yet supported by a PDP Cache agent, use the General Agent for single-page extraction or the Listing Agent for listing pages.
How It Works
- Frequent requests are logged — When users request certain URLs or domains (e.g., via the General Agent), those requests are logged.
- Intelligent agent creation — When a domain is requested frequently enough, MrScraper intelligently creates code to scrape that domain, using common fields that users in that category typically request (e.g., product name, price, description; or hotel name, rating, reviews).
- Upload to Marketplace — The new PDP Cache agent is uploaded to the Marketplace, where it becomes available for all users.
- Multiple data sources — PDP Cache retrieves data not only from HTML but also from other available sources (e.g., APIs, structured data), which improves completeness and consistency.
- Category-based schemas — Each PDP type (article, hotel, job posting, product, property, restaurant, social media, tour) follows common fields. See the API reference for each:
General Agent vs. PDP Cache Agent
| Aspect | General Agent | PDP Cache Agent |
|---|---|---|
| Cost | Higher (AI parsing per request) | Lower (no AI; dedicated scrapers + cache) |
| Speed | Slower (fetch + AI parse per request) | Faster (no AI; bypass steps by domain; cache lookup) |
| Completeness | Depends on current HTML | Often higher (multiple sources) |
| Availability | Any URL | Only supported domains in the Marketplace |
| Output | Prompt-dependent | Common fields by category |
PDP Cache agents are only available for commonly used websites that have been added to the marketplace. For other domains, use the General Agent.
Example Usage
- Go to the Marketplace.
- Browse or search for the dataset or website you want (e.g., product, article, hotel, restaurant).
- Choose the PDP Cache agent that matches your target site and category.
- Use it from your dashboard or via the API as you would other MrScraper agents.
For field-level details and example domains per category, see the PDP API reference (e.g., Article, Hotel, Product, Property, Restaurant, Job Posting, Social Media, Tour).