POST/crawl/batch

Batch Scrape

Scrape up to 100 URLs in parallel. Returns markdown, metadata, and tech stack per URL. 1 credit per URL.

Bytemine·Dynamic pricing

Dynamic pricing

0.03 * max(count(get(body, 'urls', [])), 1)

Example code

// Using @orth/sdk - The simplest way to call APIs
// Install: npm install @orth/sdk
import Orthogonal from "@orth/sdk";

const orthogonal = new Orthogonal({
  apiKey: process.env.ORTHOGONAL_API_KEY,
});

const output = await orthogonal.run({
  api: "bytemine",
  path: "/crawl/batch",
  body: {
    "urls": []
  }
});

console.log(output);

Body parameters

urlsrequired

array

Array of URLs to scrape (max 100)

Response

data.0.url

string

data.0.links.0

string

data.0.fallback

string

data.0.markdown

string

data.0.metadata.url

string

data.0.metadata.ICBM

string

data.0.metadata.title

string

data.0.metadata.favicon

string

data.0.metadata.indexId

string

data.0.metadata.DC.title

string

data.0.metadata.language

string

data.0.metadata.scrapeId

string

data.0.metadata.timezone

string

data.0.metadata.viewport

string

data.0.metadata.proxyUsed

string

data.0.metadata.sourceURL

string

data.0.metadata.cacheState

string

data.0.metadata.statusCode

number

data.0.metadata.contentType

string

data.0.metadata.creditsUsed

number

data.0.metadata.concurrencyLimited

boolean

total

number

success

boolean