Track LLM Costs: A Complete Guide for Developers

If you're building with LLMs, you need to track LLM costs. Not just total spend—but costs broken down by feature, by user, by model. This is the complete guide to getting that visibility.

Whether you're using OpenAI, Anthropic, Google Gemini, or multiple providers, this guide covers the strategies, tools, and best practices for tracking LLM costs effectively.

Why Track LLM Costs?

Before diving into the how, let's establish the why. LLM cost tracking enables:

• Budget management: Know what you're spending before the bill arrives
• Optimization decisions: Focus engineering effort where it matters most
• Pricing your product: Understand unit economics to price AI features correctly
• Anomaly detection: Catch bugs, abuse, or unexpected usage patterns
• Stakeholder reporting: Answer "why is AI costing so much?" with data

The Three Levels of LLM Cost Tracking

Level 1: Total Spend (Basic)

This is what you get from provider dashboards. You know your total monthly spend across all usage.

Pros: Zero setup, always available

Cons: No actionable insights

Level 2: Model-Level Tracking (Better)

Track costs by model type. You can see how much GPT-4o vs GPT-4o-mini costs.

Pros: Helps with model selection decisions

Cons: Still doesn't tell you which features are expensive

Level 3: Feature-Level Tracking (Best)

Track costs by feature, user segment, and use case. This is where real optimization becomes possible.

How to Implement LLM Cost Tracking

Step 1: Define Your Features

List every place your application calls an LLM. Give each a clear, consistent name:

// Good feature names
'customer-support-chat'
'document-summarizer'
'email-draft-generator'
'code-review-assistant'

// Bad feature names (too vague)
'ai-feature'
'llm-call'
'gpt-request'

Step 2: Choose Your Tracking Approach

You have three options:

Option A: Build Custom Logging

// Wrap every LLM call
async function trackedCompletion(feature, params) {
  const start = Date.now();
  const response = await openai.chat.completions.create(params);

  await trackUsage({
    feature,
    model: params.model,
    inputTokens: response.usage.prompt_tokens,
    outputTokens: response.usage.completion_tokens,
    cost: calculateCost(response.usage, params.model),
    latency: Date.now() - start
  });

  return response;
}

Pros: Full control

Cons: Maintenance burden, need to build dashboards, handle pricing updates

Option B: Use Provider SDKs with Tagging

Some providers support request metadata. Limited, but better than nothing.

Option C: Use a Dedicated Tracking SDK

import { Orbit } from '@with-orbit/sdk';

const orbit = new Orbit({ apiKey: process.env.ORBIT_API_KEY });

// Wrap once per feature
const chatClient = orbit.wrapOpenAI(new OpenAI(), {
  feature: 'customer-support-chat'
});

// Use normally - tracking is automatic
const response = await chatClient.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: userMessage }]
});

Pros: Minimal code, automatic cost calculation, built-in dashboards

Cons: External dependency

Step 3: Set Up Dashboards

Your dashboard should answer these questions at a glance:

• What's my total spend today/this week/this month?
• Which features cost the most?
• Are costs trending up or down?
• Are there any anomalies?

Step 4: Configure Alerts

Don't wait for the monthly bill. Set up proactive alerts:

• Budget threshold: Alert at 50%, 75%, 90% of budget
• Daily anomaly: Alert if daily spend exceeds 2x average
• Feature-specific: Alert if any feature exceeds its budget

Advanced Tracking: Customer Attribution

For SaaS products, you often need to track costs per customer:

// Track customer-level costs
const client = orbit.wrapOpenAI(new OpenAI(), {
  feature: 'ai-assistant',
  customer_id: customer.id  // Attribute to specific customer
});

This enables:

• Usage-based billing
• Identifying high-cost customers
• Per-customer rate limiting
• Understanding customer unit economics

Common Pitfalls in LLM Cost Tracking

1. Inconsistent Feature Tagging

If different developers use different names for the same feature, your data becomes useless. Establish naming conventions early.

2. Forgetting Development/Staging

Test environments can pollute production metrics. Always tag the environment:

const client = orbit.wrapOpenAI(new OpenAI(), {
  feature: 'chat',
  environment: process.env.NODE_ENV // 'production' | 'staging' | 'development'
});

3. Not Accounting for Retries

Failed requests still cost tokens (for the input). If you have retry logic, those retries should be tracked too.

Checklist: LLM Cost Tracking Setup

• ☐ List all LLM-powered features
• ☐ Establish feature naming conventions
• ☐ Implement tracking for all LLM calls
• ☐ Set up dashboards for cost visibility
• ☐ Set budget thresholds to monitor
• ☐ Review costs weekly
• ☐ Document tracking approach for team