Introducing Langshark

My experience with implementing long running Langchain multi-agent solutions has become disconnected from the Internet news feed topics that land in my inbox. The news feeds highlight AI agent design solutions for sandboxing, prompt optimization, eval runs and runbooks. My day-to-day experience is less pondering "big think" topics and more about being "the practitioner" tackling provider rate limiting, token credit exhaustion, external API rate limiting and keeping span observability managable. And, not letting those issues rise to a level of complexity where they distract from my core agent design objectives.

It led me to create Langshark, a local-first TUI for monitoring Langchain runtime in real-time. Langshark supports easy inspection of what your under-development multi-agent application is doing or not doing. It optionally generates clickable handoff links to a locally installed Arize Phoenix server. The Phoenix server offers a broader view of the LLM transaction history.

The default view for Langshark displays the status of all the available threads. Select a thread using the up/down arrow keys. The details about the selected thread are displayed in the lower pane.

Langshark is not intended to replace Langchain's cloud-based LangSmith but serve as a instructional bridge for learning and debugging Langchain agents. Where many Langchain tutorials start in the cloud, I am, for better or worse, stubborn about starting locally and staying local for as long as possible. You can call me old-school for wanting to troubleshoot standing up an agent server with minimal infrastructure rather than relying solely on cloud-based tools. I wanted a raw and unfiltered view of what was occurring. That was my motivation for developing a tool that is both informational and instructional.

Before diving deeper into Langshark's features, its worth knowing how the Langchain runtime processes your agent code. The Langchain runtime design is unique among the available agent harnesses. And, being familiar with Langchain runtime concepts will help you build agents that scale from day one.

Langchain is an evolving platform – especially the user facing features – offering multiple facets for developing rich agents. My view is the multitude of agent design patterns will eventually settle, as other AI technologies have.

MCP design is settling. MCP is now stateless, for example. The A2A specification has merged with ACP to be simply A2A. Skills.md and Agent.md file implementation patterns are converging on Anthropic as the golden format standard.

While Langchain's agent coding facets: nodes and graphs, send-API with map reduce and deepagents are evolving, the runtime is shared and common to all Langchain deployments. The runtime is responsible for agent execution and becoming familiar with it is a one-time investment. Being concrete foundational technology, its not likely to change significantly.

Runtime execution is when your agent first begins to consume resources which means there will inevitably be constraints to manage. I am not talking about LLM provider or API gateway constraints, but Langchain runtime resources.

The Langchain tech stack vocabulary is large and a few terms are overloaded which can introduce confusion. Over time, this too will converge on a consistent vocabulary. The most common overloaded term is "thread". In Langchain parlance, a thread refers to a runtime context and should not be confused with OS type threads used in Python and C++. A multi-agent deployment can require the Langchain runtime to manage multiple threads at once.

Langchain threads are non-preemptive and always run to completion. If another agent or user input requires a thread to run again, the thread is rescheduled by Langchain's runtime. The natural lifecycle of a thread is to alternate between running and idle states. A thread is placed in an error state if its not able to be re-scheduled.

Langchain's runtime switches threads only upon a running thread pausing for user input or reaching an idle state. The idle state is triggered when the tools, LLM responses, and middleware functions attached to a thread run to completion.

Every rescheduling of a thread is captured in a step, to use Langchain parlance. Langshark supports inspecting the contents of each step within a thread's life as shown below:

A complex or poorly designed, non-converging, multi-agent deployment can exceed a hundred steps per thread. Whether this occurrence is categorized as an error or intended execution outcome is an engineering decision. This is why Langshark provides this level of agent visibility. By inspecting thread metadata and step data you get an unfiltered view of what a Langchain agent is doing in real-time.

The ability to inspect thread content is the direct result of the Langchain's builtin checkpoint capabilities – where it maintains retrievable slices of runtime context in a database. The checkpoint feature is what gives Langchain agents their durability, repeatability, and inspectability.

Knowing how to navigate threads, steps and checkpoint data is foundational knowledge that accrues in the form of you developing robust troubleshooting skills as your agent server solution evolves. Using Langshark is a low-friction approach to becoming familiar with inner working Langchain's runtime.

To learn more about Langshark please refer to the README.md on my stokomax/langshark repository.