Listing Agent

The Listing Agent is designed specifically for extracting data from listing pages that display multiple items in a repeating format—such as product grids, job listings, search results, or directory pages. It automatically handles complex loading behaviors including infinite scroll, pagination buttons, load-more interactions, and dynamic content rendering to collect all available listing items and their detail page URLs.

When to Use Listing Agent

Use the Listing Agent when you need to:

• Extract all listing items from pages displaying multiple products, jobs, properties, or other repeated entries
• Scrape data that's visible on the listing page without visiting individual detail pages
• Automatically handle pagination through next buttons, page numbers (1, 2, 3...), or dynamic loading
• Navigate infinite scroll where content loads continuously as you scroll down
• Click "Load More" buttons to reveal additional items progressively
• Collect detail page URLs from each listing item for further processing
• Get a quick overview of all available listings with basic information visible on the listing page

Example URLs:

• https://www.amazon.com/s?k=laptops (search results with pagination)
• https://www.walmart.com/browse/electronics/laptops (category page with load more)
• https://www.zillow.com/homes/for_sale (property listings with infinite scroll)
• https://books.toscrape.com/ (product grid with next page buttons)

How Listing Agent Works

The Listing Agent intelligently analyzes how websites load and display their content, then adapts its behavior accordingly:

1. Traditional Pagination: Automatically clicks "Next" buttons or numbered page links (1, 2, 3...) to navigate through all pages
2. Infinite Scroll: Scrolls down progressively to trigger automatic content loading until all items are revealed
3. Load More Buttons: Identifies and clicks "Load More" or "Show More" buttons to append additional batches of data

The agent continues navigating until it has collected all available listing items from the page, then extracts the visible data along with detail page URLs.

Limitations

The Listing Agent has the following limitations:

• Listing Page Data Only: Can only extract information that's visible on the listing page itself
• No Detail Page Navigation: Cannot click into individual product/item links to access detail pages
• Surface-Level Data: Limited to basic information displayed in listing cards (title, price, thumbnail, brief description)
• No Deep Product Details: Cannot extract specifications, full descriptions, or other data that only appears on detail pages
• Browser Automation for Listings Only: Navigates only to load all listings (scroll, pagination, load-more) but won't enter detail pages

General vs Listing Agent

Example Usage

Follow these steps to use the Listing Agent from your dashboard:

1. Log in to MrScraper, then click Scraper in the left sidebar
2. Click New AI Scraper + at the top to create a new scraper
3. Select the Listing Scraper Agent and enter the listing page URL
4. The AI will automatically detect the page structure and navigation pattern
5. Wait for the agent to navigate through all pages/scrolls and collect all listings
6. Review the extracted data or export as JSON/CSV

Example: Scraping E-Commerce Product Listings

Example URL:
https://books.toscrape.com/

Example Prompt:

Extract all book titles, prices, ratings, and availability from this page

Listing Agent Output:

{
    "response": [
        {
        "page_num": 0,
        "data": {
            "mode": "direct",
            "data": [
            {
                "id": "74301965",
                "address": "80 Emerald Street, Manchester, NH 03103",
                "agent_company": "Listing provided by PrimeMLS",
                "price": "$364,900",
                "bedrooms": "3",
                "bathrooms": "2",
                "sqft": "1",
                "status": "Active",
                "url": "https://www.zillow.com/homedetails/80-Emerald-St-Manchester-NH-03103/74301965_zpid/",
                "highlight": "14 hours ago"
            },
            {
                "id": "121734329",
                "address": "21 School Street, New Portland, ME 04961",
                "agent_company": "KELLER WILLIAMS REALTY",
                "price": "$449,000",
                "bedrooms": "2",
                "bathrooms": "1",
                "sqft": "1",
                "status": "Active",
                "url": "https://www.zillow.com/homedetails/21-School-St-New-Portland-ME-04961/121734329_zpid/",
                "highlight": "18 hours ago"
            },
            ...
            ]
        }
        },
        {
        "page_num": 1,
        "data": {
            "mode": "direct",
            "data": [
            {
                "id": "458007902",
                "address": "Lot 59&60 Starks Road, New Sharon, ME 04955",
                "agent_company": "COLDWELL BANKER SANDY RIVER REALTY",
                "price": "$420,000",
                "bedrooms": null,
                "bathrooms": null,
                "sqft": "92 acres lot",
                "status": "Active",
                "url": "https://www.zillow.com/homedetails/LOT-5960-Starks-Rd-New-Sharon-ME-04955/458007902_zpid/",
                "highlight": "Open fields"
            },
            {
                "id": "91905999",
                "address": "3 Long Cove Road, York, ME 03909",
                "agent_company": "LEGACY PROPERTIES SOTHEBY'S INTERNATIONAL REALTY",
                "price": "$1,795,000",
                "bedrooms": "3",
                "bathrooms": "2",
                "sqft": "2",
                "status": "Active",
                "url": "https://www.zillow.com/homedetails/3-Long-Cove-Rd-York-ME-03909/91905999_zpid/",
                "highlight": "Modern comforts"
            }
            ...
            ],
            "link": "https://www.zillow.com/me/?searchQueryState=%7B%22isMapVisible%22%3Atrue..."
        }
        }
    ]
}

Comparison: Listing Agent vs General Agent

Below is a comparison showing how each agent handles the same listing page:

{
"response": [
    {
    "page_num": 0,
    "data": {
        "mode": "direct",
        "data": [
        {
            "id": "74301965",
            "address": "80 Emerald Street, Manchester, NH 03103",
            "agent_company": "Listing provided by PrimeMLS",
            "price": "$364,900",
            "bedrooms": "3",
            "bathrooms": "2",
            "sqft": "1",
            "status": "Active",
            "url": "https://www.zillow.com/homedetails/80-Emerald-St-Manchester-NH-03103/74301965_zpid/",
            "highlight": "14 hours ago"
        },
        {
            "id": "121734329",
            "address": "21 School Street, New Portland, ME 04961",
            "agent_company": "KELLER WILLIAMS REALTY",
            "price": "$449,000",
            "bedrooms": "2",
            "bathrooms": "1",
            "sqft": "1",
            "status": "Active",
            "url": "https://www.zillow.com/homedetails/21-School-St-New-Portland-ME-04961/121734329_zpid/",
            "highlight": "18 hours ago"
        },
        ...
        ]
    }
    },
    {
    "page_num": 1,
    "data": {
        "mode": "direct",
        "data": [
        {
            "id": "458007902",
            "address": "Lot 59&60 Starks Road, New Sharon, ME 04955",
            "agent_company": "COLDWELL BANKER SANDY RIVER REALTY",
            "price": "$420,000",
            "bedrooms": null,
            "bathrooms": null,
            "sqft": "92 acres lot",
            "status": "Active",
            "url": "https://www.zillow.com/homedetails/LOT-5960-Starks-Rd-New-Sharon-ME-04955/458007902_zpid/",
            "highlight": "Open fields"
        },
        {
            "id": "91905999",
            "address": "3 Long Cove Road, York, ME 03909",
            "agent_company": "LEGACY PROPERTIES SOTHEBY'S INTERNATIONAL REALTY",
            "price": "$1,795,000",
            "bedrooms": "3",
            "bathrooms": "2",
            "sqft": "2",
            "status": "Active",
            "url": "https://www.zillow.com/homedetails/3-Long-Cove-Rd-York-ME-03909/91905999_zpid/",
            "highlight": "Modern comforts"
        }
        ...
        ],
        "link": "https://www.zillow.com/me/?searchQueryState=%7B%22isMapVisible%22%3Atrue..."
    }
    }
]
}

{
"1": {
    "id": "1",
    "name": "Maine, Vermont, New Hampshire Real Estate & Homes For Sale",
    "rows": [
    {
        "address": "16 Thompson Crossing Road #231-1-3, Antrim, NH 03440",
        "price": "$599,999",
        "beds": "3",
        "baths": "2",
        "sqft": "1,613",
        "status": "Active"
    },
    {
        "address": "80 Emerald Street, Manchester, NH 03103",
        "price": "$364,900",
        "beds": "3",
        "baths": "2",
        "sqft": "1,240",
        "status": "Active"
    },
    ...
    ],
    "source": "table",
    "headers": ["Address", "Price", "Beds", "Baths", "Sqft", "Status"]
}
}

Tips and Best Practices

• Let the Agent Navigate: The Listing Agent handles pagination automatically—you don't need to specify how the site loads content
• Combine with General Agent: Use Listing Agent to collect URLs, then use General Agent to scrape detailed information from each detail page
• Review Extracted URLs: Always check that all detail page URLs are captured correctly before proceeding to detail page scraping
• Be Patient with Large Listings: Sites with hundreds of items may take longer as the agent navigates through all pages
• Understand Data Limitations: Only extract fields visible on the listing page. For detailed specs or descriptions, you'll need to visit detail pages
• Test with Small Samples First: Verify the extraction works correctly on the first page before processing entire catalogs

Real-World Use Cases

For practical examples of how to use the Listing Agent in production scenarios, check out our guides: