Ingest Url
Queue a website crawl (same site, robots.txt honored, page-limited) into the knowledge base. The URL is checked now (422 for an invalid, private or unresolvable address); the crawl and embedding run as a background job (RO-068). Poll the returned document with `GET /kb/documents/{id}` until it is `ready` or `failed`. A second request for a URL whose crawl is still pending or running returns that crawl.
API key access
kb:write scope with an API key (ro_pat_...). A signed-in session uses the caller's role./api/v1/kb/documents/urlAccess token from /api/v1/auth/login, or an API key (ro_pat_..., RO-176) from /api/v1/workspaces/current/api-keys: the key implies its workspace and needs the route's scope. MCP connectors (/api/v1/mcp only) may use an OAuth access token (ro_oat_..., RO-284).
In: header
Header Parameters
Id of the workspace to act on (must be a member).
uri1 <= lengthDefaults to (and is capped by) KB_CRAWL_MAX_PAGES
Response Body
curl -X POST "http://localhost:8000/api/v1/kb/documents/url" \ -H "X-Workspace-Id: string" \ -H "Content-Type: application/json" \ -d '{ "url": "http://example.com" }'{
"document": {
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"source_type": "url",
"source_url": "string",
"title": "string",
"content_type": "string",
"status": "pending",
"error": "string",
"char_count": 0,
"chunk_count": 0,
"embedding_model": "string",
"created_at": "2019-08-24T14:15:22Z",
"updated_at": "2019-08-24T14:15:22Z"
},
"job_id": "453bd7d7-5355-4d6d-a38e-d9e7eb218c3f",
"status": "string",
"deduplicated": true
}