LangSmith Integration

LangSmith is LangChain's platform for tracing, evaluating, and monitoring LLM applications. It helps developers debug AI chains, run evaluations against datasets, and track performance metrics --- all while supporting custom model providers.

LangSmith works alongside LangChain, which supports OpenAI-compatible endpoints. By configuring LLM API in your LangChain application, LangSmith will automatically trace all interactions.

Prerequisites

An LLM API account with an API key
LangSmith installed or accessible

Setup

Get Your LLM API Key

Log in to your LLM API dashboard
Click Create Key to Start
Copy your new API key immediately — it will only be shown once
Store the key securely (e.g., in a password manager or .env file)

LLM API is an OpenAI-compatible gateway that gives you access to dozens of AI models through a single API key and endpoint.

Use LLM API with LangChain and LangSmith

Install LangChain and configure LangSmith tracing:

pip install langchain-openai langsmith
export LANGSMITH_API_KEY="your-langsmith-api-key"
export LANGSMITH_TRACING=true

Configure LLM API as the model provider in your LangChain code:

from langchain_openai import ChatOpenAI
llm = ChatOpenAI(
model="openai/gpt-4o",
api_key="your-llm-api-key-here",
base_url="https://api.llmapi.ai/v1"
)
response = llm.invoke("Hello!")

All LLM API requests will now appear in your LangSmith dashboard for tracing and evaluation.

Test the Integration

Verify that LangSmith can successfully communicate with LLM API by sending a test request. All requests will now be routed through LLM API.

LangSmith captures every request and response from LLM API, giving you full observability over your AI application's behavior.

Benefits of Using LLM API with LangSmith

Multi-Provider Access: Use models from OpenAI, Anthropic, Google, and more through a single API
Cost Control: Track and limit your AI spending with detailed usage analytics
Unified Billing: One account for all providers instead of managing multiple API keys
Caching: Reduce costs with response caching for repeated requests

View all available models on the models page.