Extract Structured Website Data
Crawl a website, use the provided JSON Schema and instructions to prioritize relevant internal links, and extract structured data from the selected pages.
Example code
// Using @orth/sdk - The simplest way to call APIs
// Install: npm install @orth/sdk
import Orthogonal from "@orth/sdk";
const orthogonal = new Orthogonal({
apiKey: process.env.ORTHOGONAL_API_KEY,
});
const output = await orthogonal.run({
api: "context-dev",
path: "/web/extract",
body: {
"url": "example",
"schema": {}
}
});
console.log(output);Body parameters
The starting website URL to crawl and extract from. Must include http:// or https://.
JSON Schema for the returned data object.
Optional extraction guidance, such as which facts to prioritize or how to interpret fields.
When true, every returned value must be grounded in facts stated on the page.
When true, follow links on subdomains of the starting URL's domain.
Maximum number of pages to analyze for extraction. Hard cap: 50. Defaults to 5.
Optional maximum link depth from the starting URL (0 = only the starting page).
When true, iframe contents are included in Markdown before extraction.
Return cached scrape results if younger than this many milliseconds.
Optional browser wait time in milliseconds after initial page load for each crawled page.
Soft time budget for the crawl in milliseconds. Min: 10000. Max: 110000. Default: 80000.
Response