Complete collection workflow through the API

All requests below use the /api/v1 base path and a write-enabled Bearer API key. The IDs in examples are placeholders; use IDs returned by your installation.

1. Create a site

POST /sites

{"name":"Product catalog","base_url":"https://shop.example.invalid","rps":1,"concurrency":1}

name is the label; base_url is a complete URL with http:// or https://. The example domain does not work: enter your own source. A site code is generated automatically. Through the API you can optionally provide a code of up to 64 lowercase Latin letters, digits, _, or -.

Optional settings include render_mode, respect_robots, timeout_ms, proxy_pool_id, max_run_seconds, execution_target, and agent_id. Speed settings still apply to API-triggered runs. A site code cannot be changed after creation.

With multipart/form-data, you can optionally upload an icon file (PNG, JPEG, or WebP up to 4 MB). The platform stores it as a static image. To replace an existing site's icon, send the same icon field to POST /sites/{site}/icon.

2. Create a scenario

POST /scripts

{"site_id":1,"code":"catalog-main","name":"Catalog collection","entry":"run"}

Scenario codes contain lowercase Latin letters, digits, and hyphens. Keep the returned id.

3. Save and test a version

Send PHP source and an optional comment to POST /scripts/2/versions. The maximum source length is 500,000 characters. Saving a version neither publishes it nor proves it will run successfully.

POST /scripts/2/validate checks the source. GET /sandbox/sdk returns the available SDK contract. For methods and working examples, see the SDK section.

Start POST /script-versions/3/trial-run and inspect its log and results. Publish with POST /scripts/2/publish and {"version_id":3} only after a successful check.

4. Start collection

POST /runs

{"site_id":1,"script_id":2,"params":{}}

HTTP 202 means the run is queued, not finished. Poll GET /runs/{id} every 3–5 seconds. Read /log for messages, /http-log for requests, and /item-errors for rejected records. When the run finishes, read GET /sites/1/records.

To stop it, use POST /runs/{id}/cancel. Supported runs can resume through /resume. /replay repeats against saved responses and /replay-report compares results. A replay may become unavailable after snapshots are cleaned up.

5. Schedule future runs

POST /schedules

{"script_id":2,"name":"Twice daily","kind":"cron","timezone":"Europe/Moscow","preset":{"cron":"0 2,12 * * *"},"is_active":true}

This runs at 02:00 and 12:00 Moscow time. For midnight, use 0 0 * * *. Cron belongs inside preset, not in a separate field. Other schedule kinds include interval, daily, weekly, and monthly. Check the next occurrences with POST /schedules/preview before saving.

Use /pause to pause, /resume to enable, /run-now for an extra run, and GET /schedules/{id}/events for the event history. A schedule cannot guarantee an immediate start when workers are busy or the budget is exhausted.

Avoid duplicate actions

There is no universal Idempotency-Key header yet. If a run request times out, check recent runs before retrying: a blind second POST could start another run. Update an existing resource with PATCH where appropriate.