Skip to main content

Track LLM Costs

This guide walks you through setting up cost tracking for Large Language Model (LLM) API calls in your Python application. By the end, you'll see per-request token counts and costs appear in Beakpoint Insights automatically.

Beakpoint supports cost tracking for LLM calls made through direct provider APIs (OpenAI, Anthropic) and cloud gateways (AWS Bedrock, Azure OpenAI, Azure AI Foundry). The setup is the same across all platforms — the only differences are which instrumentation package to install and which client to use.

Language

Examples use Python. Beakpoint works with any language that OpenTelemetry supports.

For a complete, runnable example of everything described here, see Quill — Beakpoint's reference application for multi-model LLM cost attribution.

Prerequisites​

Before you begin, ensure you have:

Install Dependencies​

Install the OpenTelemetry SDK and the OTLP exporter (common to all platforms):

pip install opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp-proto-http

Then install the auto-instrumentation package for your platform(s):

PlatformPackage
OpenAI (direct)pip install opentelemetry-instrumentation-openai-v2
Anthropic (direct)pip install opentelemetry-instrumentation-anthropic
AWS Bedrockpip install opentelemetry-instrumentation-botocore
Azure OpenAIpip install opentelemetry-instrumentation-openai-v2 (same as OpenAI)
Azure AI Foundrypip install azure-ai-inference[opentelemetry] azure-core-tracing-opentelemetry

Configure the Exporter​

Set your Beakpoint API key and OTLP endpoint as environment variables before running your application:

export OTEL_EXPORTER_OTLP_ENDPOINT="https://otel.beakpoint.io/api/traces"
export OTEL_EXPORTER_OTLP_HEADERS="x-bkpt-key=YOUR_API_KEY"
export OTEL_SERVICE_NAME="my-llm-service"

Replace YOUR_API_KEY with your actual Beakpoint Insights API key.

Instrument Your Application​

The setup below is adapted from Quill's tracing.py. This initialization function:

  1. Creates a Resource with service.name, service.namespace, and service.version, attaching these identifiers to every exported span.
  2. Registers a custom SpanProcessor that copies gen_ai.provider.name → gen_ai.system and cloud.provider on every LLM span. Beakpoint requires gen_ai.system for pricing lookup.
  3. Adds a BatchSpanProcessor with an OTLPSpanExporter to send traces to Beakpoint.
import os

from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import Span, SpanProcessor, TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor


class _GenAiSystemProcessor(SpanProcessor):
    """Copy gen_ai.provider.name → gen_ai.system and cloud.provider."""

    def on_start(self, span: Span, parent_context=None):
        attrs = span.attributes
        if attrs is None:
            return
        provider_name = attrs.get("gen_ai.provider.name")
        if provider_name:
            provider_str = str(provider_name)
            if not attrs.get("gen_ai.system"):
                span.set_attribute("gen_ai.system", provider_str)
            if not attrs.get("cloud.provider"):
                span.set_attribute("cloud.provider", provider_str)

    def on_end(self, span):
        pass

    def shutdown(self):
        pass

    def force_flush(self, timeout_millis=30000):
        return True


def init_tracing():
    resource = Resource.create({
        "service.name": os.environ.get("OTEL_SERVICE_NAME", "my-llm-service"),
        "service.namespace": "my-app",
        "service.version": "0.1.0",
    })

    provider = TracerProvider(resource=resource)
    provider.add_span_processor(_GenAiSystemProcessor())

    endpoint = os.environ.get("OTEL_EXPORTER_OTLP_ENDPOINT")
    if endpoint:
        exporter = OTLPSpanExporter(endpoint=endpoint)
        provider.add_span_processor(BatchSpanProcessor(exporter))

    trace.set_tracer_provider(provider)

    return provider
Why the _GenAiSystemProcessor?

The auto-instrumentors set gen_ai.provider.name but not gen_ai.system. Beakpoint requires gen_ai.system to look up per-model pricing. The processor above copies the value automatically so you don't need to set it on every span yourself. See Quill's tracing.py for the full implementation.

Platform-Specific Setup​

After calling init_tracing(), activate the instrumentor for your platform and use the provider's client as you normally would. The instrumentor automatically captures a span for every API call, including token counts and model information.

OpenAI​

from opentelemetry.instrumentation.openai_v2 import OpenAIInstrumentor
import openai

OpenAIInstrumentor().instrument(tracer_provider=provider)

client = openai.OpenAI()
response = client.chat.completions.create(
    model="gpt-4.1-mini",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Summarise the key points of this document."},
    ],
)
print(response.choices[0].message.content)

Anthropic​

from opentelemetry.instrumentation.anthropic import AnthropicInstrumentor
import anthropic

AnthropicInstrumentor().instrument(tracer_provider=provider)

client = anthropic.Anthropic()
response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=4096,
    system="You are a helpful assistant.",
    messages=[{"role": "user", "content": "Summarise the key points of this document."}],
)
print(response.content[0].text)

AWS Bedrock​

from opentelemetry.instrumentation.botocore import BotocoreInstrumentor
import boto3

BotocoreInstrumentor().instrument(tracer_provider=provider)

client = boto3.client("bedrock-runtime")
response = client.invoke_model(
    modelId="anthropic.claude-sonnet-4-20250514-v1:0",
    contentType="application/json",
    accept="application/json",
    body='{"messages": [{"role": "user", "content": "Hello!"}], "max_tokens": 4096, "anthropic_version": "bedrock-2023-05-31"}',
)

Azure OpenAI​

Uses the same OpenAIInstrumentor as direct OpenAI. The instrumentor handles both client types:

from opentelemetry.instrumentation.openai_v2 import OpenAIInstrumentor
from openai import AzureOpenAI

OpenAIInstrumentor().instrument(tracer_provider=provider)

client = AzureOpenAI(
    azure_endpoint="https://mydeployment.openai.azure.com",
    api_version="2025-04-01-preview",
)
response = client.chat.completions.create(
    model="gpt-4.1",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hello!"},
    ],
)

Azure AI Foundry​

Azure AI Foundry requires one extra configuration step — enabling Azure SDK tracing:

from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.tracing import AIInferenceInstrumentor
from azure.core.credentials import AzureKeyCredential
from azure.core.settings import settings

settings.tracing_implementation = "opentelemetry"
AIInferenceInstrumentor().instrument()

client = ChatCompletionsClient(
    endpoint="https://myendpoint.models.ai.azure.com",
    credential=AzureKeyCredential("YOUR_KEY"),
)
response = client.complete(
    model="claude-sonnet-4",
    messages=[{"role": "user", "content": "Hello!"}],
)

The instrumentors intercept every SDK call automatically. No additional tracing code is needed.

Adding Cost Attribution​

To slice costs by project, department, or code path, set span attributes on your application spans. This pattern is taken from Quill's analyzer.py:

tracer = trace.get_tracer("my-app.analyzer")

with tracer.start_as_current_span(
    "my-app.analyze",
    attributes={
        "code.function.name": "my_app.analyzer.analyze_document",
        "app.user.org.id": "Finance",  # department for cost attribution
    },
) as span:
    # LLM calls made here become child spans of my-app.analyze
    result = client.messages.create(...)

    # Propagate token counts to the parent span for rollup
    span.set_attribute("gen_ai.usage.input_tokens", result.usage.input_tokens)
    span.set_attribute("gen_ai.usage.output_tokens", result.usage.output_tokens)

See the Cost Attribution Tags concept page for the full list of supported attribution dimensions.

Verify Traces in Beakpoint​

  1. Run your instrumented application and make at least one LLM API call.
  2. Log in to Beakpoint Insights.
  3. Navigate to Traces and search for your service name (the value you set in OTEL_SERVICE_NAME).
  4. Open a trace and confirm you can see a span with a gen_ai.system attribute and non-zero token counts.
  5. Navigate to Costs to see the calculated spend broken down by model and request.
tip

If no traces appear within a minute of running your application, check that your OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS environment variables are set correctly and that outbound HTTPS traffic to otel.beakpoint.io is not blocked by a firewall.

Next Steps​