In this example, we’ll demonstrate how to leverage our X scraper API to extract data based on a keyword.

Requirements

  • MrScraper console account.
  • MrScraper API token that you can get by following the steps here.

X Sentiment Example

In this example, we’ll retrieve a keyword sentiment data from X, returning results based on the defined schema.

Follow the steps below to use our X scraper API:

  1. Use the request body below:
curl --request POST \
  --url https://app.mrscraper.com/api/scrapers/leads-generator/twitter/create-and-run \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
    "name": "twitter",
    "keywords": "@Calendly",
    "sentiment_type": "all",
    "expected_data": "10"
}'
  1. Replace Authorization with your API token.
  2. The above request body will return the following JSON response:
{
    "results": [
            {
                "id": 1152496,
                "scraper_id": 2966,
                "scraping_run_id": 368641,
                "scraper_name": "mrscraper in twitter",
                "scrapped_url": "Default",
                "scraped_url": "Default",
                "status": "succeeded",
                "content": {
                    "tweets": [
                        {
                            "id": "1838949276069835224",
                            "bio": "Data Extraction Made Easy. Built in public by @heykaiyo",
                            "name": "MrScraper",
                            "text": "Facts",
                            "website": "http://MrScraper.com",
                            "link_bio": "",
                            "username": "MrScraper_",
                            "sentiment": "neutral",
                            "created_at": "Wed Sep 25 14:30:48  2024"
                        },
                        {
                            "id": "1838909623094423906",
                            "bio": "Data Extraction Made Easy. Built in public by @heykaiyo",
                            "name": "MrScraper",
                            "text": "Unlock the power of Airbnb data with MrScraper!  \n\nOur new blog guides you through the process, step-by-step.  \n\nStart scraping today!\n\n#buildinpublic \nhttps://t.co/RqXCf9i7ZC",
                            "website": "http://MrScraper.com",
                            "link_bio": "",
                            "username": "MrScraper_",
                            "sentiment": "positive",
                            "created_at": "Wed Sep 25 11:53:14  2024"
                        },
                        {
                            "id": "1838532969440403746",
                            "bio": "Data Extraction Made Easy. Built in public by @heykaiyo",
                            "name": "MrScraper",
                            "text": "Stuck between Wget and Curl for your web scraping projects?\n\nOur new blog breaks down the pros and cons of each tool to help you make an informed decision.\n\n#buildinpublic\n\nhttps://t.co/rxkZd751Vt",
                            "website": "http://MrScraper.com",
                            "link_bio": "",
                            "username": "MrScraper_",
                            "sentiment": "neutral",
                            "created_at": "Tue Sep 24 10:56:32  2024"
                        },
                        {
                            "id": "1838510745882939808",
                            "bio": "+10 SaaS built | 1 exit | Focus on building SaaS in public :)",
                            "name": "Louis | SaaS & Build in public",
                            "text": "@heykaiyo @MrScraper_ @heykaiyo makes scraping much easier. congrats on the new launch.",
                            "website": "",
                            "link_bio": "",
                            "username": "thegrowthcoder",
                            "sentiment": "positive",
                            "created_at": "Tue Sep 24 09:28:14  2024"
                        },
                        {
                            "id": "1838447002348364141",
                            "bio": "Data Extraction Made Easy. Built in public by @heykaiyo",
                            "name": "MrScraper",
                            "text": "@TsunamiStables @parsehub Hmm that sounds tricky, Sir! \n\nWhy not consider using @MrScraper?",
                            "website": "http://MrScraper.com",
                            "link_bio": "",
                            "username": "MrScraper_",
                            "sentiment": "neutral",
                            "created_at": "Tue Sep 24 05:14:56  2024"
                        },
                        {
                            "id": "1838446580107743512",
                            "bio": "Data Extraction Made Easy. Built in public by @heykaiyo",
                            "name": "MrScraper",
                            "text": "@ShyliMadhala @parsehub Why not give @MrScraper a try, Miss?\n\nIt could simplify your process and make scraping a lot easier for you!",
                            "website": "http://MrScraper.com",
                            "link_bio": "",
                            "username": "MrScraper_",
                            "sentiment": "positive",
                            "created_at": "Tue Sep 24 05:13:16  2024"
                        },
                        {
                            "id": "1838446152271958421",
                            "bio": "Data Extraction Made Easy. Built in public by @heykaiyo",
                            "name": "MrScraper",
                            "text": "@dominiconorton @parsehub Hmm that sounds frustrating, Sir. \n\nWhy not consider using @MrScraper? It might provide a smoother experience!",
                            "website": "http://MrScraper.com",
                            "link_bio": "",
                            "username": "MrScraper_",
                            "sentiment": "negative",
                            "created_at": "Tue Sep 24 05:11:34  2024"
                        },
                        {
                            "id": "1679454970737836032",
                            "bio": "💡 3x founder\n🚀 eng @ 8-figure hypergrowth startup (techstars/vc) \n🌱 free startup mentorship w/ https://t.co/ReAE8Jon4U\n🏆 building ppl w/ omw\n🛠️ gaudmire @_buildspace",
                            "name": "Tom",
                            "text": "@philkellr @kai_developer @MrScraper_ Not Kai but have much experience doing the founder-support thing.\n\nIt's not a \"fun\" thing to do, but it builds strong relationships with your early customers and really helps dial in your product because feedback goes directly to you.\n\nMy advice? Don't fear it, it's a good thing!",
                            "website": "https://plzsend.help",
                            "link_bio": "http://plzsend.help",
                            "username": "atunasashimi",
                            "sentiment": "neutral",
                            "created_at": "Thu Jul 13 11:37:22  2023"
                        }
                    ],
                    "keywords": "mrscraper"
                },
                "created_at": "2024-09-26T04:14:13.000000Z",
                "updated_at": "2024-09-26T04:17:40.000000Z"
            }
    ],
    "created_at": "2024-09-26T04:14:13.000000Z"
}
To use this endpoint for your use case, please refer to this section.