In this example, we’ll demonstrate how to leverage our AI Scraping API to extract key details from a social media website.

​
Requirements

  • MrScraper console account.
  • MrScraper API token that you can get by following the steps here.

​
Social Media Example

In this example, we’ll retrieve data from Reddit, returning results based on the defined schema.

Follow the steps below to use our AI Scraper API to extract Reddit data:

  1. Use the request body below:
curl --location 'https://app.mrscraper.com/api/ai' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer API_TOKEN' \
--data '{
    "urls": [
        "https://www.reddit.com/r/technology/"
    ],
    "min": 100,
    "max": 200,
    "timeout": 180,
    "schema": {
        "type": "array",
        "description": "List of posts",
        "items": {
            "type": "object",
            "description": "post information",
            "properties": {
                "post_title": {
                    "type": "string",
                    "description": "post title"
                },
                "post_url": {
                    "type": "string",
                    "description": "post valid reddit.com only url"
                },
                "post_date": {
                    "type": "string",
                    "description": "post date"
                },
                "post_username": {
                    "type": "string",
                    "description": "post username"
                }
            },
            "required": [
                "post_title",
                "post_url",
                "post_date",
                "post_username"
            ]
        },
        "required": [
            "post"
        ]
    }
}'
  1. Replace Authorization with your API token and Set the URL to the Reddit.
  2. The above request body will return the following JSON response:
{
    "result": [
        {
            "post_title": "Trump calls for prosecution of Google over search results he says favor Harris",
            "post_url": "https://www.reddit.com/r/technology/comments/1fqxltl/trump_calls_for_prosecution_of_google_over_search/",
            "post_date": "11 hr. ago",
            "post_username": "u/Puginator"
        },
        {
            "post_title": "Meta has been fined €91M ($101M) after it was discovered that to 600 million Facebook and Instagram passwords had been stored in plain text.",
            "post_url": "https://www.reddit.com/r/technology/comments/1fqmjko/meta_has_been_fined_91m_101m_after_it_was/",
            "post_date": "20 hr. ago",
            "post_username": "u/a_Ninja_b0y"
        },
        {
            "post_title": "Flaw in Kia’s web portal let researchers track, hack cars | Bug let researchers track millions of cars, unlock doors, and start engines at will.",
            "post_url": "https://www.reddit.com/r/technology/comments/1fqlf1q/flaw_in_kias_web_portal_let_researchers_track/",
            "post_date": "21 hr. ago",
            "post_username": "u/chrisdh79"
        }
    ],
    "tokenUsage": 4,
    "totalDone": 1
}
Real EstateJob Search Platform