Web scraping API

Start with the output you need

If you want the page text for an AI app, try a Markdown or text response first. For a page whose content is already present in the HTML, a basic fetch may be enough.

When a page loads its content with JavaScript, choose browser rendering. Proxy options can also help with sites that restrict automated requests. Both can affect the cost, so use them when your target pages need them.

For product details or other specific fields, you can use extraction rules or describe the fields with a JSON schema. Schema-based extraction can reduce the need to maintain selectors, but check the returned values against your source pages.

The example below uses Firecrawl to return Markdown. Other providers use their own request formats; your NativePort key stays the same when you try them.

Try it step by step

Extract product details from a web page

Give Firecrawl a URL and the fields you need, then read the result as JSON. This example extracts a product name, price, and stock status.

View guide

Collect a website for your AI knowledge base

Start with a website URL and collect the pages a crawl discovers as Markdown. Use the content for document search, a knowledge base, or your AI app.

View guide

Try a request

This Firecrawl request returns a page as Markdown. Replace the example URL with a page you want to read.

curl

curl https://api.nativeport.ai/firecrawl/v2/scrape \
  -H "Authorization: Bearer $NATIVEPORT_API_KEY" \
  -H "Content-Type: application/json" \
  --data '{"url": "https://example.com", "formats": ["markdown"]}'

example response

{
  "success": true,
  "data": {
    "markdown": "# Example Domain\n\nThis domain is for use in...",
    "metadata": {
      "title": "Example Domain",
      "sourceURL": "https://example.com",
      "statusCode": 200
    }
  }
}

Compare providers

Compare starting prices and the tasks each provider supports. Features and usage affect the total cost. For measured results, see the benchmarks and how we test.

Provider	Best suited for	Starting price
Firecrawl	Read pages as Markdown or request fields with a JSON schema.	$0.0032 per call
Jina	Read the main content of a page from its URL.	$0.0001 per call
Spider	Collect content from a page or a website crawl.	metered by Spider
Tavily	Retrieve page content for a search or research workflow.	$0.008 per call
Exa	Find pages and retrieve content for your app.	metered by Exa
ScrapingBee	Render JavaScript and extract fields with rules.	$0.00098 per call
Scrapfly	Render a page and extract the content you need.	$0.00015 per call
ZenRows	Fetch pages with optional browser rendering.	metered by ZenRows
ScraperAPI	Fetch page HTML through managed proxies.	$0.00049 per call
Crawlbase	Fetch page content, with a JavaScript option.	metered by Crawlbase
Serper	Look up places and request their reviews.	$0.001 per call

Build on an example

Set your NativePort API key and adapt an example to your own data. Outputs below illustrate the response format.

Start collecting a documentation site for your AI app

Starts a crawl job. Check Firecrawl pricing and set a page limit before running it.

python

# Standard library only — nothing to pip install.
import json, os, urllib.request

def post(path, payload):
    req = urllib.request.Request(
        "https://api.nativeport.ai" + path,
        data=json.dumps(payload).encode(),
        headers={
            "Authorization": f"Bearer {os.environ['NATIVEPORT_API_KEY']}",
            "Content-Type": "application/json",
        },
    )
    with urllib.request.urlopen(req) as r:
        return json.load(r)

job = post("/firecrawl/v2/crawl", {
    "url": "https://docs.example.com",
    "limit": 100,
    "scrapeOptions": {"formats": ["markdown"]},
})
print(job["id"])

example output

$ python crawl_docs.py
8f2a1c04-6b3e-4f77-9a15-2c0d9e7b4411

Let your agent read a web page

One request per URL. Check the Jina provider page for current pricing.

curl

curl https://api.nativeport.ai/jina/reader/https://example.com \
  -H "Authorization: Bearer $NATIVEPORT_API_KEY"

More ways to use these APIs

Convert a page to Markdown

Use Jina Reader to turn a URL into readable content, or request Markdown from Firecrawl when you also need its page metadata. This is a useful starting point when your next step is to pass the text to a model.

Jina · Firecrawl · /jina/reader/<url>

Scrape prices and listings from a marketplace

If a marketplace loads listings with JavaScript, enable rendering before extracting prices and other fields. The ZenRows route is a general scraper, so you need to maintain the extraction for your target site. A site-specific Apify actor is another option to investigate where one is available.

ZenRows · /zenrows?url=…

Scrape reviews and ratings

For Google reviews, look up the place with /serper/places, then use its identifier with /serper/reviews. A rating shown on another website may need to be extracted from that page. Check whether you need the review text, the rating, or both before choosing a request.

Serper · /serper/places → /serper/reviews

Capture a screenshot of a webpage

Capture a page as a PNG with Browserless. A screenshot can help you inspect what a page displayed when its extracted text looks unexpected. Steel and Spider also offer screenshot options to compare.

Browserless · /browserless/screenshot

Extract an article and its details

Zyte can return article fields such as the headline, body, author, and date. If you only need readable text, compare a Markdown response from Firecrawl or Jina. Choose the format that fits what your app will do next.

Zyte · /zyte

Discover the URLs on a website

Use Firecrawl’s map endpoint to discover URLs before collecting page content. You can review the list, choose the relevant paths, and fetch only the pages your project needs.

Firecrawl · /firecrawl/v2/map

Collect pages across a website

Start a crawl to discover pages and collect their content. Choose Markdown for text-based workflows or an HTML format when you need the markup. The website crawling guide shows how to start the job, check progress, and set a page limit.

Firecrawl · /firecrawl/v2/crawl

Find sources when you do not have a URL

Start with search when you have a question or topic. Tavily can return results with page content that a model can read. Your application can run additional searches if it needs more information. Compare options in the web search guide.

Tavily · /tavily/search

Look up search volume and ranking for a keyword

Use DataForSEO’s live endpoints to request keyword metrics or search results for rank tracking. These use separate requests. The SEO data guide explains which data to request and how to use it in a report.

DataForSEO · /dataforseo/v3/…/live

Convert a curl command into code

An AI model can help turn a curl example into code in the language you use. Send the example and the language you want in a chat-completions request, then review the generated code before running it. See the AI model guide for the available models.

NativePort Inference · /inference/v1/chat/completions

Before you start

Do I need a subscription to each provider?

No. You can use the listed providers with your NativePort key and shared balance. Check the pricing page for how usage is billed.

What happens if the website blocks a request?

NativePort returns the provider’s error and status code. Your app can then decide whether to retry with different options or use another provider. Retries are not automatic.

How should I compare providers?

Try the same representative URLs with each option. Check whether you get the content and format you need, then compare the cost of the complete request, including rendering or extraction features.

Try it with $5 in credits

Get an API key Compare providers