NativePort
Sign in Get a key
← How-to

How to use LangChain with NativePort

Point langchain-openai's ChatOpenAI at NativePort's unified inference endpoint: one key and one balance for invoke, async, streaming and structured tool calls.

LangChain’s ChatOpenAI class, from the langchain-openai partner package, already documents the exact pattern this tutorial uses: pass a custom base_url and api_key to reach an OpenAI-compatible endpoint that isn’t OpenAI itself. Point that same class at NativePort’s unified inference endpoint and every LangChain call, sync, async, streaming, or tool calling, runs on one NativePort API key and one balance instead of a separate account and credential for each model provider you want to reach.

This tutorial covers installation, a minimal call, async and streaming variants, and a structured tool calling example with bind_tools(), along with the handful of things about this setup that behave differently from calling OpenAI directly.

What you’ll need

  • A NativePort API key. Sign up to get a key and $5 in credits, so you can run every example below without adding a card first.
  • Python 3.10 or later.
  • pip install langchain langchain-openai. This tutorial is written against langchain 1.3.14 and langchain-openai 1.4.1. ChatOpenAI lives in the langchain-openai partner package, not in core langchain, so both are needed even for a single chat call.
  • Your key exported as an environment variable, never hardcoded in source:
export NATIVEPORT_API_KEY="<your NativePort API key>"

A minimal call

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="openai/gpt-5.4-mini",
    base_url="https://api.nativeport.ai/inference/v1",
    api_key=os.environ["NATIVEPORT_API_KEY"],
    max_completion_tokens=600,
)

response = llm.invoke("In one sentence, what does a gateway API do?")
print(response.content)

A few things here are load-bearing, not stylistic choices:

  • model="openai/gpt-5.4-mini" is NativePort’s canonical model id for this endpoint. The openai/ prefix selects the route; everything after it is OpenAI’s own model name. langchain-openai forwards this string to the gateway byte for byte, so there’s no prefix stripping to work around here.
  • base_url="https://api.nativeport.ai/inference/v1" is NativePort’s unified inference path. The same request shape works for OpenAI, Anthropic, Grok and Hugging Face models behind this one endpoint by swapping the model string.
  • max_completion_tokens, not max_tokens. The gpt-5.4-mini model family rejects the older max_tokens field outright, the same way OpenAI’s own current reasoning-capable model families do. max_completion_tokens is the field this route accepts, and ChatOpenAI passes it straight through, so set it explicitly rather than relying on a client default.

api_key and base_url are the constructor kwargs current LangChain docs and examples use. The underlying field names openai_api_key and openai_api_base still work, since they’re the same Pydantic fields under an alias, but treat them as legacy rather than reaching for them in new code.

Async

ChatOpenAI exposes an async counterpart to every sync method, ainvoke() for invoke(), with the same parameters:

import asyncio

async def main():
    response = await llm.ainvoke("Name one advantage of an async HTTP client.")
    print(response.content)

asyncio.run(main())

Nothing about routing through NativePort changes this surface. ainvoke() is the same method you’d call against OpenAI directly, on the same llm instance built with the custom base_url above.

Streaming

for chunk in llm.stream("List three uses for a message queue, one per line."):
    print(chunk.content, end="", flush=True)
print()

astream() is the async equivalent, used the same way inside an async def with async for.

One default worth knowing before you rely on it: stream_usage, LangChain’s flag for including token usage in the final streamed chunk, defaults to off whenever ChatOpenAI is constructed with a non-default base_url. That’s a deliberate LangChain default, not a NativePort restriction, because many OpenAI-compatible endpoints don’t support streaming token usage at all. Ask for it explicitly if you need it:

for chunk in llm.stream("Say hello in five words.", stream_usage=True):
    if chunk.usage_metadata:
        print(chunk.usage_metadata)
    print(chunk.content, end="", flush=True)
print()

Requesting stream_usage=True tells the client to ask for usage in the stream; whether a given backend actually populates it can still vary, so check chunk.usage_metadata on the response rather than assuming it’s present just because you asked. response.usage_metadata on a non-streaming invoke() call doesn’t carry this caveat, since usage there comes back with the full response body rather than trailing a stream.

Tool calling with bind_tools()

bind_tools() attaches a tool schema to the model and returns a new runnable that can emit structured tool calls instead of, or alongside, plain text. Define a tool with a JSON schema, bind it, and call the model with a prompt that should trigger it:

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Look up the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

llm_with_tools = llm.bind_tools(tools)
response = llm_with_tools.invoke("What's the weather in Boston?")
print(response.tool_calls)

response is still an AIMessage, and response.tool_calls is a list of dicts, each with name, args (already parsed into a Python dict, not a raw JSON string), and id. For the prompt above, that list holds one entry naming get_weather with args={"city": "Boston"}. Dispatch on name to route the call to your own function, then run it:

def get_weather(city: str) -> str:
    return f"It's sunny in {city}."

available_tools = {"get_weather": get_weather}

for call in response.tool_calls:
    fn = available_tools[call["name"]]
    result = fn(**call["args"])
    print(call["id"], result)

That’s the scope of the tool-call handling shown here: the model returns a correctly named, correctly argued structured call, and your code dispatches it. A complete tool loop normally continues from there by appending a ToolMessage (carrying that same id and the tool’s return value) back onto the message list and calling invoke() again so the model can incorporate the result into a final reply. That continuation follows LangChain’s own documented ToolMessage pattern and isn’t specific to NativePort, so build and test it against your own tools rather than treating the snippet above as the whole loop.

bind_tools() also accepts tool_choice, the same way the underlying chat completions API does, if you want to force a call rather than leave the choice to the model:

llm_with_tools = llm.bind_tools(tools, tool_choice="required")

Troubleshooting

  • 401 on every request: NATIVEPORT_API_KEY is missing, malformed, or was passed to the wrong kwarg. Check it lands in api_key, not left for OPENAI_API_KEY to pick up a different value, since api_key always takes precedence when both are set.
  • 402: the NativePort balance backing this key is at zero. Every request fails closed here rather than degrading in quality; top up and retry.
  • 404 or “model not found”: the model id is missing its route prefix. Use the full canonical id, openai/gpt-5.4-mini, not a bare gpt-5.4-mini. langchain-openai doesn’t rewrite or validate this string; it sends exactly what you pass in model, so a typo or a missing prefix reaches the gateway unchanged.
  • A model rejects max_tokens with a 400 about an unsupported or invalid parameter: that field is the legacy one. Use max_completion_tokens instead, which is what gpt-5.4-mini and similarly current model families accept; passing both at once is redundant and only one is required.
  • A tool-bearing request fails with a schema-related 400: bind_tools() accepts an optional strict argument, and OpenAI’s own chat completions API applies stricter validation to tool schemas whenever structured output constraints are turned on for the request. A schema with optional or default-valued arguments is more likely to trip this than one where every property is required. If you hit this, simplify the schema (mark every field required, drop defaults) before assuming the gateway or the model is at fault.
  • A field you expected from a provider-specific response shows up as missing or None: ChatOpenAI only parses the fields defined by OpenAI’s own response schema. Anything outside that shape is silently dropped rather than surfaced on the returned message, regardless of what the underlying backend actually returned. This is a langchain-openai parsing boundary, not a sign the field wasn’t in the raw response.

What this costs

openai/gpt-5.4-mini through this endpoint is metered at $0.75 per 1M input tokens and $4.50 per 1M output tokens, NativePort’s real, pass-through rate with no per-call markup. Check response.usage_metadata if you’re tracking spend per call. See pricing for the full billing model.

Where to go next

  • OpenAI provider page: scope and pricing range for the models behind this route, including the /inference/v1 contract this tutorial runs on.
  • Pricing: the full billing model.
  • Coming from an OpenRouter integration? How NativePort compares covers what carries over and what doesn’t.