Skip to main content

Track LLM Costs

This guide walks you through setting up cost tracking for Large Language Model (LLM) API calls in your Python application. By the end you will see per-request token counts and costs appear in Beakpoint Insights automatically.

For a complete, runnable example of everything described here, see Quill — Beakpoint's reference application for multi-model LLM cost attribution.

Prerequisites​

Before you begin, ensure you have:

Install Dependencies​

Install the OpenTelemetry SDK, the OTLP exporter, and the GenAI auto-instrumentation packages for your providers:

pip install opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
pip install opentelemetry-instrumentation-anthropic opentelemetry-instrumentation-openai-v2

Configure the Exporter​

Set your Beakpoint API key and OTLP endpoint as environment variables before running your application:

export OTEL_EXPORTER_OTLP_ENDPOINT="https://otel.beakpoint.io/api/traces"
export OTEL_EXPORTER_OTLP_HEADERS="x-bkpt-key=YOUR_API_KEY"
export OTEL_SERVICE_NAME="my-llm-service"

Replace YOUR_API_KEY with your Beakpoint Insights API key.

Instrument Your Application​

The setup below is adapted from Quill's tracing.py. It does four things:

  1. Creates a Resource with service.name, service.namespace, and service.version — attached to every exported span.
  2. Registers a custom SpanProcessor that copies gen_ai.provider.name → gen_ai.system and cloud.provider on every LLM span (Beakpoint requires gen_ai.system for pricing lookup).
  3. Adds a BatchSpanProcessor with an OTLPSpanExporter pointed at Beakpoint.
  4. Activates auto-instrumentation for Anthropic and OpenAI so every API call automatically produces a span with token counts, model name, and gen_ai.* attributes.
import os

from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.anthropic import AnthropicInstrumentor
from opentelemetry.instrumentation.openai_v2 import OpenAIInstrumentor
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import Span, SpanProcessor, TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor


class _GenAiSystemProcessor(SpanProcessor):
    """Copy gen_ai.provider.name → gen_ai.system and cloud.provider.

    The auto-instrumentors set gen_ai.provider.name, but Beakpoint requires
    gen_ai.system for pricing. This processor bridges the gap on span start.
    """

    def on_start(self, span: Span, parent_context=None):
        attrs = span.attributes
        if attrs is None:
            return
        provider_name = attrs.get("gen_ai.provider.name")
        if provider_name:
            provider_str = str(provider_name)
            if not attrs.get("gen_ai.system"):
                span.set_attribute("gen_ai.system", provider_str)
            if not attrs.get("cloud.provider"):
                span.set_attribute("cloud.provider", provider_str)

    def on_end(self, span):
        pass

    def shutdown(self):
        pass

    def force_flush(self, timeout_millis=30000):
        return True


def init_tracing():
    resource = Resource.create({
        "service.name": os.environ.get("OTEL_SERVICE_NAME", "my-llm-service"),
        "service.namespace": "my-app",
        "service.version": "0.1.0",
    })

    provider = TracerProvider(resource=resource)
    provider.add_span_processor(_GenAiSystemProcessor())

    endpoint = os.environ.get("OTEL_EXPORTER_OTLP_ENDPOINT")
    if endpoint:
        exporter = OTLPSpanExporter(endpoint=endpoint)
        provider.add_span_processor(BatchSpanProcessor(exporter))

    trace.set_tracer_provider(provider)

    AnthropicInstrumentor().instrument(tracer_provider=provider)
    OpenAIInstrumentor().instrument(tracer_provider=provider)

    return provider
Why the _GenAiSystemProcessor?

The OpenAI and Anthropic auto-instrumentors set gen_ai.provider.name but not gen_ai.system. Beakpoint requires gen_ai.system to look up per-model pricing. The processor above copies the value automatically so you don't need to set it on every span yourself. See Quill's tracing.py for the full implementation.

Making LLM Calls​

Once instrumented, use the Anthropic and OpenAI clients as normal. Every API call automatically produces a span with token counts and model info:

import anthropic

client = anthropic.Anthropic()
response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=4096,
    system="You are a helpful assistant.",
    messages=[{"role": "user", "content": "Summarise the key points of this document."}],
)
print(response.content[0].text)
import openai

client = openai.OpenAI()
response = client.chat.completions.create(
    model="gpt-4.1-mini",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Summarise the key points of this document."},
    ],
)
print(response.choices[0].message.content)

No additional tracing code is needed for these calls — the instrumentors intercept every SDK call automatically.

Adding Cost Attribution​

To slice costs by project, department, or code path, set span attributes on your application spans. This pattern is taken from Quill's analyzer.py:

tracer = trace.get_tracer("my-app.analyzer")

with tracer.start_as_current_span(
    "my-app.analyze",
    attributes={
        "code.function.name": "my_app.analyzer.analyze_document",
        "app.user.org.id": "Finance",  # department for cost attribution
    },
) as span:
    # LLM calls made here become child spans of my-app.analyze
    result = client.messages.create(...)

    # Propagate token counts to the parent span for rollup
    span.set_attribute("gen_ai.usage.input_tokens", result.usage.input_tokens)
    span.set_attribute("gen_ai.usage.output_tokens", result.usage.output_tokens)

See the Cost Attribution Tags concept page for the full list of supported attribution dimensions.

What Beakpoint Reads From Each Span​

The auto-instrumentation and the _GenAiSystemProcessor produce these attributes on every LLM span:

AttributeExample valueBeakpoint purpose
gen_ai.systemanthropic / openaiCost calculation — provider pricing table lookup
cloud.provideranthropic / openaiCost calculation — provider identification
gen_ai.request.modelclaude-sonnet-4-6Cost calculation — determines per-token price
gen_ai.response.modelclaude-sonnet-4-6Cost calculation — exact model version for pricing
gen_ai.usage.input_tokens512Cost calculation — input token count
gen_ai.usage.output_tokens128Cost calculation — output token count
service.nameAcme-AcquisitionCost attribution — slice spend by project
app.user.org.idFinanceCost attribution — slice spend by department

Verify Traces in Beakpoint​

  1. Run your instrumented application and make at least one LLM API call.
  2. Log in to Beakpoint Insights.
  3. Navigate to Traces and search for your service name (the value you set in OTEL_SERVICE_NAME).
  4. Open a trace and confirm you can see a span with gen_ai.system = openai or gen_ai.system = anthropic and non-zero token counts.
  5. Navigate to Costs to see the calculated spend broken down by model and request.
tip

If no traces appear within a minute of running your application, check that your OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS environment variables are set correctly and that outbound HTTPS traffic to otel.beakpoint.io is not blocked by a firewall.

Next Steps​