In this example, we’ll demonstrate how to leverage our Google scraper API to extract data based on a keyword.

​
Requirements

  • MrScraper console account.
  • MrScraper API token that you can get by following the steps here.

​
Google Example

In this example, we’ll retrieve a keyword sentiment data from Google, returning results based on the defined schema.

Follow the steps below to use our Google scraper API:

  1. Use the request body below:
curl --request POST \
  --url https://app.mrscraper.com/api/scrapers/leads-generator/google/create-and-run \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
    "name": "google",
    "search_type": "url",
    "url": "https://calendly.com",
    "limit_pages": "5"
}'
  1. Replace Authorization with your API token.
  2. The above request body will return the following JSON response:
{
    "results": [
            {
                "id": 1152473,
                "scraper_id": 2963,
                "scraping_run_id": 368634,
                "scraper_name": "try google from docs",
                "scrapped_url": "Default",
                "scraped_url": "Default",
                "status": "succeeded",
                "content": {
                    "sites": [
                        {
                            "url": "https://mrscraper.com/",
                            "index": 1,
                            "title": "MrScraper - ChatGPT for Scraping.",
                            "emails": [],
                            "snippet": "MrScraper is a tool that helps you collect information from websites quickly and easily. Instead of fighting annoying captchas, MrScraper does the work for you.",
                            "socialProfiles": {
                                "Twitter": [
                                    "https://twitter.com/MrScraper_"
                                ]
                            }
                        },
                        {
                            "url": "https://chromewebstore.google.com/detail/mrscraper/bhjedfpgiobfaojogegbmcbocfnbkimh?hl=en",
                            "index": 2,
                            "title": "MrScraper - Chrome Web Store",
                            "emails": [],
                            "snippet": "MrScraper selector helper tool. This is not a scraper extension. This extension is ONLY a tool to help you easily find the CSS selectors for scrapers in ...",
                            "socialProfiles": []
                        },
                        {
                            "url": "https://twitter.com/mrscraper_",
                            "index": 3,
                            "title": "MrScraper (@MrScraper_) / X",
                            "emails": [],
                            "snippet": "MrScraper - Scraping Tool & Resources To Unlock Data At Scale. ChatGPT for Scraping.",
                            "socialProfiles": {
                                "Twitter": [
                                    "https://twitter.com/MrScraper_",
                                    "https://twitter.com/tos",
                                    "https://twitter.com/privacy"
                                ]
                            }
                        },
                        {
                            "url": "https://news.ycombinator.com/item?id=34736691",
                            "index": 4,
                            "title": "Show HN: MrScraper – A visual web-scraping tool | Hacker News",
                            "emails": [],
                            "snippet": "With MrScraper, you won't be blocked. We use real browser instances to perform fast but human web scrapings, resulting in a much lower block ratio.",
                            "socialProfiles": []
                        },{
                            "url": "https://m.youtube.com/watch?v=IPIF9yxOGQg&pp=ygUKI21yc2NyYXBlcg%3D%3D",
                            "index": 53,
                            "title": "Web scraping with MrScraper.com - YouTube",
                            "emails": [],
                            "snippet": "Building a web scraper is not easy MrScraper is a visual web scraper to extract data, easily and ...",
                            "socialProfiles": []
                        }
                    ],
                    "keywords": "mrscraper"
                },
                "created_at": "2024-09-26T03:34:49.000000Z",
                "updated_at": "2024-09-26T03:40:07.000000Z"
            }
    ],
    "created_at": "2024-09-26T03:34:49.000000Z"
}
To use this endpoint for your use case, please refer to this section.