Give your agent a website in readable Markdown

The problem

A website is spread across links, and its HTML mixes useful content with menus, scripts, and repeated page furniture. An ordinary agent with one fetched page does not automatically discover the rest or assemble it into a usable reading collection.

How NativePort helps

NativePort connects your agent to Firecrawl to discover pages and return their content in a readable format. You choose a limit, wait for the collection, and give the resulting pages to the agent with their source links. Your application decides what to store or index.

Technical implementation

Use Python 3 with requests installed (python -m pip install requests). Set NATIVEPORT_API_KEY in your environment to your NativePort key. Run the snippets on your server, where your key stays private.

Set SITE_URL to the site to read. This example limits the crawl to ten pages and polls the job using the same account.

python

import os
import requests

BASE = "https://api.nativeport.ai"
HEADERS = {"Authorization": f"Bearer {os.environ['NATIVEPORT_API_KEY']}"}

import json
import time
from pathlib import Path

response = requests.post(BASE + "/firecrawl/v2/crawl", headers=HEADERS,
    json={"url": os.environ["SITE_URL"], "limit": 10,
          "scrapeOptions": {"formats": ["markdown"]}}, timeout=90)
response.raise_for_status()
job_id = response.json()["id"]
status_url = BASE + "/firecrawl/v2/crawl/" + job_id
for attempt in range(60):
    response = requests.get(status_url, headers=HEADERS, timeout=90)
    response.raise_for_status()
    job = response.json()
    if job["status"] == "completed":
        break
    if job["status"] in ("failed", "cancelled"):
        raise RuntimeError(job)
    time.sleep(5)
else:
    raise TimeoutError(f"Crawl {job_id} is still running; resume polling this ID")

pages = job.get("data", [])
if job.get("next"):
    raise RuntimeError("More pages are available; paginate before saving a complete collection")
Path("pages.json").write_text(json.dumps([
    {"url": page.get("metadata", {}).get("sourceURL"),
     "markdown": page.get("markdown", "")}
    for page in pages
], ensure_ascii=False, indent=2), encoding="utf-8")

The crawl may still run after the local polling deadline. Resume with the same job ID rather than starting a duplicate crawl. For large responses, follow the crawl pagination contract through the NativePort route; never send your NativePort credential to an upstream URL. For a single known page, Jina Reader is another option.

Tool costs

Tool used in the example	Price per call
Firecrawl — website crawl	$0.003376 / credit (usage-based)

Prices in USD. Usage-based tools have no fixed per-call price. View pricing.

Let your agent build it

You don’t need to write this code yourself. Copy this page’s link and paste it into your agent. Ask it to follow the guide and implement the feature for you.

https://nativeport.ai/use-cases/website-to-markdown/

The problem

How NativePort helps

Technical implementation

Tool costs

Let your agent build it

Keep exploring

Prepare PDFs for your agent to read

Put the most useful passages in front of your agent

Explore all use cases