​

About Webhooks

Webhooks in MrScraper let you send HTTP requests to an external API when certain scraper events occur. This guide will show you how to set up webhooks, choose the events that trigger them, and link them to specific scrapers.

​

Setting Up Webhooks

To set up a webhook, follow these steps:

1. Open the Mrscraper dashboard and go to the Webhooks section.

2. Click the New Webhook button to create a new webhook.

3. Fill in the necessary details for the webhook:

• Name: A user-friendly name for your webhook.
• Post URL: The endpoint of the external API where the request will be sent.
• Event: The event that will trigger the send operation of the payload. In for the case of ‘send the results anytime the scraper’s result is processed’, we use result.processed event.

4. Click Create to create the webhook.

Example configuration:

{
     "name": "My Webhook",
     "post_url": "https://example-api.com/webhook-endpoint",
     "event": "result.processed",
}

6. Toggle the Activate button to make sure the webhook is activated.

​

Webhook Result Payload

For further use of the Webhook request, here are some example payload for a webhook:

• Success Payload
• Failed Payload

  {
      "id":1668,
      "scraper_id":489,
      "scraper_name":"Luma",
      "scrapped_url":"https://lu.ma/seattle",
      "scraped_url":"https://lu.ma/seattle",
      "status":"succeeded",
      "content": {
          ...
          //here is where the result payload of the scraping operation is saved
      },
      "created_at":"2024-10-18T02:25:13.000000Z",
      "updated_at":"2024-10-18T02:35:47.000000Z"
  }

​

More examples

For a multi-URL scraping operation result, the result will be processed individually for every scrapped_url, so the webhook is going to sent multiple payload for every scraping URL result payload.

• 1st result from URL https://lu.ma/sf
• 2nd result from URL https://lu.ma/dc

  {
  "id":1669,
  "scraper_id":489,
  "scraper_name":"Luma",
  "scrapped_url":"https://lu.ma/sf",
  "scraped_url":"https://lu.ma/sf",
  "status":"failed",
  "content":null,
  "created_at":"2024-10-18T02:25:13.000000Z",
  "updated_at":"2024-10-18T02:35:47.000000Z"
}