Best LLM observability tools compared: Langfuse, Helicone, Datadog

As of November 3, 2026, Langfuse suits self-hosted tracing and evaluations, while Datadog LLM Observability (now called Agent Observability) connects AI workflows with infrastructure. Helicone and Portkey combine monitoring with gateway features, and gateway logs such as Requesty's help inspect routed calls.

Price is harder to compare. A trace can contain several provider calls, observations and scores, and each vendor counts those objects differently. We make Requesty, one of the products covered below.

Start with the operation you need to debug

An application trace groups the operations in one workflow. An LLM span records an operation involving a model call; observations add detail about what happened, and scores can record how an output was evaluated.

Those definitions affect both coverage and the bill. Evaluation coverage below refers only to the features cited in this comparison.

An incorrect agent decision may require evaluations and application-level traces. A provider-latency investigation may get its answer from gateway logs. Neither job tells you which product costs less until you count the objects it records.

What does a million traces cost?

Take a workload with 1 million application traces per month. Our assumptions: each trace has one LLM provider request, every request is recorded, and there are no retries or sampling.

For Langfuse, each trace creates one trace object, five observations and one score. That is seven billable units per trace, or 7 million units in total. Our Datadog calculation uses one billable LLM span per trace; the Portkey calculation uses one recorded request log per trace.

Sources: Langfuse pricing, Datadog pricing, Portkey pricing, Helicone pricing and Requesty billing.

Now, change one input. If every trace creates one more Langfuse observation, the workload rises to 8 million units; at the stated tier rate, that adds $70 per month and takes the estimate to $591. The observation and score counts are the weakest inputs because they depend on how you instrument and evaluate your application.

Model costs sit outside these estimates. Use the LLM API pricing guide to budget inference separately.

When is Langfuse the better fit?

Langfuse fits teams that want tracing, prompt management, evaluations and experiments, with the option to run the open-source core themselves. Its self-hosting configuration lists web and worker containers, PostgreSQL, Redis or Valkey, ClickHouse and blob storage in the minimum footprint.

That leaves your team with infrastructure to operate. On Cloud, the tradeoff is different: Langfuse counts each ingested trace, observation and score, including qualifying objects created by its evaluation features.

Core costs $29 per month, includes 100,000 units and lists 90 days of data access. Pro costs $199 per month, also includes 100,000 units and lists three years of data access.

According to Langfuse's cost guidance, sampling and removing unnecessary observations can lower the Cloud bill. The tradeoff is less detail when a sampled-out workflow is the one you need to debug.

Gateway monitoring points to Helicone or Portkey

Both products belong on the shortlist when monitoring and gateway features are part of the same purchase. Their limits and integration choices deserve a closer look.

If gateway routing is part of the purchase, compare the best AI gateways too. A monitoring trial alone will not tell you whether the routing layer meets your requirements.

Datadog brings in broader operational context

Datadog now calls its LLM observability product Agent Observability. Its trace terminology covers workflow, tool and task operations as well as LLM calls, which is useful when an investigation needs broader operational context.

Datadog bills by ingested LLM spans, and an LLM span is a request to an LLM provider. One workflow can produce several LLM spans. The free tier includes up to 40,000 LLM spans per month with 15-day retention.

Check cost coverage against the model names and providers you use. Datadog says its LLM cost estimates can be incomplete for unsupported model names or providers.

Gateway logs cover the calls the gateway receives

For calls a gateway receives, its logs can answer three common questions: which model was called, how long did the request take and what did it cost? In Requesty, the trace view groups requests that share a trace_id and shows total calls, tokens, cost and latency.

From there, usage analytics can group data by model, provider, user, member or custom field, with calculations including median, P95 and P99. If the investigation points to provider reliability, fallback policies are available for routing subsequent calls.

There is a boundary. A gateway can only provide complete request logs for calls it receives, and grouping those calls does not capture every application operation outside the gateway. Use application instrumentation and evaluation tooling when you need to inspect those steps.

Requesty documents configurable request-body logging. Its data privacy documentation also says operational metadata is stored regardless of logging or zero-data-retention settings, so review that exception before sending sensitive data.

Requesty's billing documentation describes a 5% service fee on credit top-ups: a $100 top-up costs $105 and adds $100 to the balance. That fee is not a per-trace observability price.

Run the same workflow through every candidate

Before committing to a plan, run the same workflow through each candidate. Pick one that includes the provider calls and application steps your team expects to debug in production.

1. 1
   Instrument a representative workflow

   Record the workflow with each candidate and check which operations appear in its trace or logs. Include any evaluation steps your team expects to use.

2. 2
   Count the billable objects

   Count traces, observations, scores, LLM spans and request logs as applicable. Multiply those counts by expected monthly workflow volume, then apply each plan's included allowance and published rates.

3. 3
   Verify retention and data controls

   Check how long traces and logs remain accessible, whether storage has a separate charge, and what happens after a plan limit is reached. Review which prompts, responses and operational metadata are stored.

4. 4
   Debug a known failure

   Give the team a workflow with a known failure and ask it to find the cause. Check whether the recorded data supports the evaluation and debugging work before removing observations or applying sampling.

Use our gateway observability guide to decide what to log, measure and alert on. For hosted deployment tradeoffs, the managed gateway comparison covers another part of the decision.

Choose for the debugging job

Start with one workflow and count what each tool records. Then price the resulting objects at your expected volume and retention period before choosing a plan.