Start a Scrape job

Scrape up to 10,000 URLs in one background job; each item takes the same parameters as a single call.

POSThttps://rankfield.dev/v1/scrape/jobs

curl -X POST 'https://rankfield.dev/v1/scrape/jobs' \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -H 'Idempotency-Key: scrape-2026-10-26' \
  -d '{"items":[{"url":"https://rankfield.dev/blog/what-is-site-strength"},{"url":"https://example.com/"}]}'

import requests

response = requests.post(
    "https://rankfield.dev/v1/scrape/jobs",
    headers={"Authorization": "Bearer YOUR_API_KEY", "Idempotency-Key": "scrape-2026-10-26"},
    json={"items": [{"url": "https://rankfield.dev/blog/what-is-site-strength"}, {"url": "https://example.com/"}]},
)
print(response.json())

const response = await fetch("https://rankfield.dev/v1/scrape/jobs", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json",
    "Idempotency-Key": "scrape-2026-10-26",
  },
  body: JSON.stringify({"items":[{"url":"https://rankfield.dev/blog/what-is-site-strength"},{"url":"https://example.com/"}]}),
});
console.log(await response.json());

{
  "job": {
    "id": "8f14e45fceea167a5a36dedd4bea2543",
    "status": "queued",
    "total": 2,
    "done": 0,
    "failed": 0,
    "createdAt": "2026-10-26T18:00:00Z",
    "finishedAt": null,
    "expiresAt": null
  }
}

{
  "error": {
    "code": "invalid_input",
    "message": "This Idempotency-Key was used for different items"
  }
}

Parameters

Headers

Name	Type	Required	Default	Description
`Authorization`	string	Yes	–	`Bearer` followed by your API key. See Authentication.
`Idempotency-Key`	string	Yes	–	1–200 characters you choose for this job. Sending the same key with the same body again returns the same job instead of a new one.

Body

Name	Type	Required	Default	Description
`items`	object[]	Yes	–	1–10,000 items. Each takes the same parameters as Scrape a page, as a JSON object.

Every item is checked before the job starts. If any is invalid, the whole request is a 400 listing the first 20 (index and message).

Response

202 with job. A byte-identical body with the same Idempotency-Key replays the job (200, Idempotency-Replayed: true); any other body with that key is 409.

Field	Type	Description
`job.id`	string	The job id.
`job.status`	string	`queued`, `running`, `done` or `canceled`.
`job.total`	integer	URLs in the job.
`job.done`	integer	Items that succeeded.
`job.failed`	integer	Items that failed.
`job.createdAt`, `job.finishedAt`	string \| null	When it was created and when it finished (UTC).
`job.expiresAt`	string \| null	When its results are deleted: 24 hours after it finishes.

A job runs 16 items at a time. An item counts as usage once the job accepts it, whatever its outcome. Items use the endpoint's default timeout rather than their own, and retry busy by themselves (up to 10 times). Follow it with Get a Scrape job and read items with Get Scrape job results.

Errors

Status	Code	When
400	`invalid_input`	No `Idempotency-Key`, no items, too many items, or an invalid item.
401	`unauthorized`	No valid API key.
409	`invalid_input`	The `Idempotency-Key` was used before with a different body.
429	`busy`	The batch queue is full: 200,000 items from all keys are already waiting. Retry after `Retry-After` (60 seconds).
503	`unavailable`	Batch jobs are paused (storage). Retry after `Retry-After` (300 seconds).

Limits

Send up to 10,000 URLs in one job, and as many jobs as you like. No per-key limits. Jobs from all keys share one queue: when 200,000 items are already waiting, a new job answers 429 busy with Retry-After (60 seconds).

MCP

On the MCP server, the tool for this is start_scrape_job.

Start a Scrape job

#Parameters

#Headers

#Body

#Response

#Errors

#Limits

#MCP