Tavily Crawl
Tavily Crawl is a graph-based website traversal tool that can explore hundreds of paths in parallel with built-in extraction and intelligent discovery.
Dynamic pricing
ceil((body.limit ? body.limit : 50) / 10) * (body.instructions ? 0.02 : 0.01) + ceil((body.limit ? body.limit : 50) / 5) * (lower(body.extract_depth) == 'advanced' ? 0.02 : 0.01)
Example code
// Using @orth/sdk - The simplest way to call APIs
// Install: npm install @orth/sdk
import Orthogonal from "@orth/sdk";
const orthogonal = new Orthogonal({
apiKey: process.env.ORTHOGONAL_API_KEY,
});
const output = await orthogonal.run({
api: "tavily",
path: "/crawl",
body: {
"url": "example"
}
});
console.log(output);Body parameters
The root URL to begin the crawl.
Natural language instructions for the crawler. When specified, the mapping cost increases to 2 API credits per 10 successful pages instead of 1 API credit per 10 pages.
Chunks are short content snippets (maximum 500 characters each) pulled directly from the source. Use chunks_per_source to define the maximum number of relevant chunks returned per source and to control the raw_content length. Chunks will appear in the raw_content field as: <chunk 1> [...] <chunk 2> [...] <chunk 3>. Available only when instructions are provided. Must be between 1 and 5.
Max depth of the crawl. Defines how far from the base URL the crawler can explore.
Max number of links to follow per level of the tree (i.e., per page).
Total number of links the crawler will process before stopping.
Regex patterns to select only URLs with specific path patterns (e.g., /docs/.*, /api/v1.*).
Regex patterns to select crawling to specific domains or subdomains (e.g., ^docs\.example\.com$).
Regex patterns to exclude URLs with specific path patterns (e.g., /private/.*, /admin/.*).
Regex patterns to exclude specific domains or subdomains from crawling (e.g., ^private\.example\.com$).
Whether to include external domain links in the final results list.
Whether to include images in the crawl results.
Advanced extraction retrieves more data, including tables and embedded content, with higher success but may increase latency. basic extraction costs 1 credit per 5 successful extractions, while advanced extraction costs 2 credits per 5 successful extractions.
The format of the extracted web page content. markdown returns content in markdown format. text returns plain text and may increase latency.
Whether to include the favicon URL for each result.
Maximum time in seconds to wait for the crawl operation before timing out. Must be between 10 and 150 seconds.
Response