LangSmith alternatives: how to choose by constraint, not by feature list

The short version

• Langfuse is the usual LangSmith alternative when traces must stay on infrastructure you control, because it is open source and self-hostable.
• Braintrust fits teams whose main work is evaluation: datasets, scoring and comparing prompt or model changes.
• Arize fits teams that treat agents as a production monitoring problem, and its Phoenix tool can be run on your own infrastructure.
• A gateway such as Helicone records model calls with little code, but it does not see the tool calls and retrieval steps between them unless you log those separately.
• Instrumenting with OpenTelemetry keeps traces portable, which lowers the cost of the next tool change.

Which LangSmith alternative should you choose?

Choose the tool that removes the specific constraint that made you look. Pick Langfuse to self-host, Braintrust for evaluation-led work, Arize for production monitoring, Helicone for low-effort logging of model calls, and OpenTelemetry if you want traces in the observability stack you already run.

The feature lists overlap heavily, so a feature-by-feature comparison rarely settles anything. What differs is where the data lives, how much you have to operate yourself, and which workflow the product was built around first. Start from those three questions and the shortlist usually shrinks to one or two.

How do the main LangSmith alternatives compare?

When is it not worth leaving LangSmith?

It is not worth leaving when no hard constraint is forcing the move. A mild preference for another interface does not pay for a migration.

Tracing is the cheap part to move. Swapping an SDK or pointing an exporter somewhere else is usually a small job. The expensive parts are the things built on top: evaluation datasets, scoring functions, prompt versions, dashboards, alerts and the habits of the people who read them. Those hold the judgement your team has built up, and they do not move with a configuration change.

Dropping LangChain is not a reason on its own either. LangSmith works without it, so a team that has left the framework can keep the tool. Good reasons to move are a data location requirement LangSmith's plans do not meet, a cost curve that gets worse as volume grows, or a workflow your team keeps working around.

Which LangSmith alternative keeps traces on your own infrastructure?

Langfuse is the usual answer, because it is open source and designed to be self-hosted. Arize Phoenix is the other common option. Before assuming you have to leave at all, check which LangSmith plan includes self-hosting.

This matters because a trace is not just metadata. A useful agent trace holds the user's input, the retrieved documents, the tool arguments and the model's output. In a legal or healthcare system, that can make your tracing store one of the most sensitive databases you own, and it needs the same access rules as the systems the data came from.

Self-hosting moves the risk rather than removing it. You now own the database, the storage, the upgrades, the backups and the retention policy. LangChain's State of Agent Engineering survey took 1,340 responses, and 49% of respondents were at companies of under 100 people. It is a vendor's own survey, so read it as a view of that vendor's audience rather than the whole market. Even so, teams of that size feel the cost of one more stateful system to run. For some of them, a hosted tool with sensitive fields masked before they leave the application is the better decision.

Which LangSmith alternative is best for evaluation?

Braintrust is the natural one to test first if evaluation is your main need, because the product centres on datasets, scoring and comparing runs. Langfuse, Arize and LangSmith all offer evaluation too, so test against your own cases rather than assuming a gap.

Evaluation is what tells you whether a prompt or model change made an agent better or merely different. A trace explains what happened on one request. An evaluation shows what happens across the set of cases you care about. Teams that only trace tend to learn about regressions from their users.

The tool is the smaller part of this. The real asset is the dataset: actual requests, including the ones that went wrong, with an agreed idea of what a good result looks like. Keep that dataset in a portable format whichever product you pick, because it will outlive the tool.

Is a gateway like Helicone enough for tracing agents?

A gateway is enough for a simple app that calls a model, but usually not for an agent. It sits between your code and the model provider, so it records every model call with little effort. It cannot see what your code did between those calls unless you log that separately.

An agent's failures tend to sit in those gaps. The wrong tool got called, a retrieval step returned nothing useful, or a loop ran several times before giving up. A log of model calls shows the symptoms: more tokens and a strange answer. SDK or OpenTelemetry tracing shows the structure: which step called which, in what order, with what arguments.

A gateway also sits in your request path, which makes it a dependency of your production traffic. Check how it fails and what happens to your calls if it is unavailable. Logging done outside the request path can fail without taking requests down with it.

Can you use your existing observability stack instead?

Yes for tracing, latency and cost, and it is often the right first step if you already run Grafana, Datadog or something similar. Evaluation is the gap to check, because that is where general tools vary most.

OpenTelemetry is the neutral layer. It has semantic conventions for generative AI, which are still maturing, and several of the tools in this post can ingest it. Instrumenting with it means your traces are not written in one vendor's format, so the next move costs less than this one.

The honest limit is that you may have to build the views yourself. A long agent trace full of large prompts and tool payloads is not what most dashboards were designed to display, and comparing prompt versions is not a job a metrics tool was built for. For a small team with one agent, that may be fine. With several agents and a review process, a dedicated tool starts to earn its place.

How should you compare pricing across LangSmith alternatives?

Compare on your own trace volume over a real week, not on plan headlines. LangChain, Arize and Braintrust publish pricing pages, listed in the sources for this post, and those pages are the ones to trust. This post quotes no prices, because they change faster than a blog post does.

The units differ, which is why a headline comparison misleads. A plan may combine seats, trace or span allowances, data volume and retention. An agent also produces far more telemetry than a chatbot, because one user request can fan out into many model and tool steps. An allowance that looks generous for a chatbot can run out quickly for an agent.

Retention is the line people miss. Evaluation and audit work need older traces, so the cheapest retention tier may not be one you can actually use. If you self-host, add the infrastructure and the engineer time to run it. A tool with no licence fee still has a bill.

How do you test a LangSmith alternative before committing?

Run it alongside your current tool on one real agent, and judge it on a failure you have already seen. A demo trace proves nothing. A production failure shows whether the tool would actually have helped.

A short trial plan

1. Write down the one constraint driving the search. If you cannot name it, stop here.
2. Send the same traces to both tools from one agent, and keep your current tool as the record.
3. Take a real production failure and try to diagnose it using the new tool alone.
4. Move one evaluation dataset and one scorer, and note what did not survive the move.
5. Price a real week of traffic on the published plans, including retention.
6. Export your traces and datasets once, so you know you can leave.