PlantUML in LLM Agents — visualising reasoning and human-in-the-loop

puml.online
  2026-07-30 11:19 2026-07-30 11:19 Created   2026-08-14 21:34:29 2026-08-14 21:34:29 Updated    

LLM agent output is a token stream that’s hostile to humans. PlantUML is the bridge between agents turning their “thinking” into structured, readable diagrams — a practical guide for LangChain / AutoGPT / CrewAI.

The interpretability headache for agents

Today’s pain points:

  • Process opaque: reasoning trace is implicit prompt chain; developers can’t see why a decision was made.
  • Hard to debug: when an agent fails, you don’t know which step broke.
  • Multi-agent complexity: a 20-step agent task → rerun, hope it works.
  • Weak human-in-the-loop: humans want to intervene, but agents don’t expose “I’m currently thinking X”.

PlantUML is the bridge.

Main use cases

1. Decision-tree visualisation (ReAct agent)

ReAct is “Reasoning + Acting” alternating:

1
2
3
4
5
Thought: I need to find X
Action: search(query)
Observation: result of search
Thought: based on result, I now do Y
Action: ...

Convert trace to PlantUML activity:

1
2
3
4
5
6
7
8
9
10
11
@startuml
start
:Thought: search;
:Action: search("plantuml");
:Observation: 100 results;
:Thought: pick top 5;
:Action: rank();
:Observation: top 5;
:Final: 5 results;
stop
@enduml

Value: at a glance, developers see which observation didn’t drive reasoning forward → check the prompt / tool.

2. Multi-agent collaboration (CrewAI / AutoGen)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
@startuml
left to right direction
skinparam rectangle<<agent>> {
  BackgroundColor<<researcher>> #FFE0E0
  BackgroundColor<<coder>>      #E0FFE0
  BackgroundColor<<critic>>     #E0E0FF
}

rectangle "Researcher" <<researcher>> as r
rectangle "Coder"      <<coder>>      as c
rectangle "Critic"     <<critic>>     as crit

r -> c : "research findings"
c -> crit : "draft implementation"
crit -> c : "feedback"
crit -> r : "ask for more research"
@enduml

This is a state machine, not a call stack — it expresses information flow between agents.

3. Tool-call decisions (function calling)

1
2
3
4
5
6
7
8
9
10
11
12
@startuml
(*) --> "Receive user query"
if "Query mentions database?" then
  -->[yes] "Call db_query tool"
  --> "Get records"
  --> "Format"
  --> "Return"
else
  -->[no] "Direct LLM response"
  --> "Return"
endif
@enduml

Engineering implementation

Tracking traces in LangChain

LangChain has langchain.debug = True for detailed traces, but human reading nested arrays is unfriendly.

1
2
3
4
5
6
7
8
import langchain
from langchain.agents import AgentExecutor
from langchain.tools import tool

langchain.debug = True

agent = AgentExecutor(...)
result = agent.invoke({"input": "What is PlantUML?"})

Output:

1
2
3
4
5
6
[chain/start] [1:chain:AgentExecutor]
[llm/start] ...
[tool/start] search
[tool/end] ...
[llm/end] ...
[chain/end]

Custom callback to emit PlantUML

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
from langchain.callbacks.base import BaseCallbackHandler

class PlantUMLCallback(BaseCallbackHandler):
    def __init__(self):
        self.steps = []
    
    def on_agent_action(self, action, **kwargs):
        self.steps.append(f"Action: {action.tool}, Input: {action.tool_input}")
    
    def on_agent_finish(self, finish, **kwargs):
        self.steps.append(f"Finish: {finish.return_values}")
    
    def to_puml(self):
        puml = "@startuml\nstart\n"
        for s in self.steps:
            puml += f":{s};\n"
        puml += "stop\n@enduml\n"
        return puml

# Usage
cb = PlantUMLCallback()
agent.invoke({"input": "..."}, config={"callbacks": [cb]})
print(cb.to_puml())
# Render → see the decision process in the browser

Trace rendering to console

1
2
3
4
5
6
7
8
9
10
11
import plantuml
puml_server = plantuml.PlantUML(url='http://www.plantuml.com/plantuml/svg/')

def render_trace(steps):
    puml = "@startuml\nstart\n"
    for s in steps:
        puml += f":{s};\n"
    puml += "stop\n@enduml\n"
    return puml_server.processes(puml)

# Output to /var/log/agent-trace.svg for devops

Live display in a web UI

1
2
3
4
5
6
7
8
const pumlEncoder = require('plantuml-encoder');

async function fetchAndRender(traceId) {
  const trace = await fetch(`/api/agent/trace/${traceId}`).then(r => r.json());
  const puml = traceToPuml(trace);
  const encoded = pumlEncoder.encode(puml);
  document.getElementById('trace-img').src = `https://www.plantuml.com/plantuml/svg/~1${encoded}`;
}

LangSmith / LangGraph × PlantUML

LangSmith visualises traces but not via PlantUML. Manual export:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# LangGraph
from langgraph.graph import StateGraph

graph = StateGraph(MyState)
graph.add_node("search", search_node)
graph.add_node("rank", rank_node)
graph.add_edge("search", "rank")

# Manually export graph.adjacency as PlantUML
def to_puml(graph):
    puml = "@startuml\n"
    for src, dst in graph.edges:
        puml += f"{src} --> {dst}\n"
    puml += "@enduml"
    return puml

with open("/tmp/agent-graph.puml", "w") as f:
    f.write(to_puml(graph))

plantuml -tsvg /tmp/agent-graph.puml → visualisation.

CrewAI built-in visualisation

CrewAI has crew.plot() for the task graph:

1
2
crew = Crew(agents=[...], tasks=[...])
crew.plot()  # outputs tasks_graph.png

Under the hood: NetworkX + matplotlib. To get PlantUML: regenerate.

1
2
3
4
5
6
7
8
9
def crew_to_puml(crew):
    puml = "@startuml\n"
    for agent in crew.agents:
        puml += f'rectangle "{agent.role}" as {agent.id}\n'
    for task in crew.tasks:
        for agent in task.assigned_agents:
            puml += f"{agent.id} --> {task.id}\n"
    puml += "@enduml"
    return puml

ReWOO / Plan-and-Execute agents

ReWOO agents (plan first, then execute):

1
2
3
4
5
6
7
8
9
10
11
@startuml
(*) --> "Plan:"
--> "Step 1: search"
--> "Step 2: extract"
--> "Step 3: synthesize"
--> "Execute plan:"
--> "Run step 1"
--> "Run step 2"
--> "Run step 3"
--> "Final answer"
@enduml

This is a PlantUML state-machine + activity hybrid.

Multi-agent state-machine implementation

Let the multi-agent system explain itself:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
class MultiAgentSystem:
    def __init__(self):
        self.transitions = []
    
    def transition(self, from_agent, to_agent, reason):
        self.transitions.append({
            "from": from_agent,
            "to": to_agent,
            "reason": reason
        })
    
    def render(self):
        puml = "@startuml\n"
        for agent in set(a["from"] for a in self.transitions) | set(a["to"] for a in self.transitions):
            puml += f'rectangle "{agent}" as {agent}\n'
        for t in self.transitions:
            puml += f"{t['from']} --> {t['to']} : {t['reason']}\n"
        puml += "@enduml"
        return puml

In debugging:

1
2
3
4
5
6
mas = MultiAgentSystem()
mas.transition("user", "router", "ask plan")
mas.transition("router", "researcher", "needs data")
mas.transition("researcher", "critic", "review")
# ...
print(mas.render())

Emit PlantUML → see agent collaboration.

“Human intervention” during debugging

PlantUML exposes → humans can clearly see where to pause, add constraints:

1
2
3
4
5
6
7
8
9
10
11
12
class InterventionHook:
    def __init__(self, mas):
        self.mas = mas
    
    def check(self, agent_output):
        for rule in self.rules:
            if rule.matches(agent_output):
                return HumanInterventionRequest(
                    agent=agent_output.agent,
                    reason=rule.reason,
                    diagram=self.mas.render()  # current state
                )

HumanInterventionRequest attaches the current PlantUML state; humans see a visual snapshot of the current collaboration and decide “approve / modify / abort”.

vs other visualisation tools

Tool Used by Best for
LangSmith LangChain trace timeline, token counts
LangGraph viz LangGraph graph statics
PlantUML Cross-framework Easy doc embed, version control, PDF export
Mermaid LangGraph default Browser rendering
matplotlib CrewAI default Static PNG

PlantUML advantages:

  • Text is diff-friendly → lives in git
  • Multiple output formats (SVG/PNG/PDF) → embed in PPTs
  • CJK readable → Chinese papers
  • !include for sub-diagram reuse

Project case: two ReAct agents collaborating

1
2
3
4
5
6
7
8
9
10
11
12
13
14
@startuml
rectangle "User Query" as UQ
rectangle "Agent A (Researcher)" as A
rectangle "Agent B (Coder)" as B
database "Tool Cache" as TC

UQ --> A : task
A --> TC : search
TC --> A : result
A --> B : handoff to coder
B --> A : ask for more
A --> B
B --> UQ : final output
@enduml

A complete example: debug a failing agent

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# debug_agent.py
from langchain_community.tools import DuckDuckGoSearchRun
from my_callback import PlantUMLCallback

cb = PlantUMLCallback()
tools = [DuckDuckGoSearchRun()]

agent = initialize_agent(
    tools,
    llm,
    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
    verbose=True,
    callbacks=[cb]
)

try:
    result = agent.run("Tell me about plantuml.com")
except Exception as e:
    print("Error:", e)
finally:
    print(cb.to_puml())
    # → write to /var/log/agent-trace.svg

When the agent fails, output the state-machine snapshot for post-mortem.

Recap

  • PlantUML has 4 roles in agent scenarios: debug visualisation, flow narrative, human-intervention snapshot, doc embed.
  • Lowest implementation cost: custom callback outputs puml + render → SVG.
  • Multi-agent framework diagrams are “information flow”, not “call stack” — always distinguish.

Next

  • Title: PlantUML in LLM Agents — visualising reasoning and human-in-the-loop
  • Author: puml.online
  • Created at : 2026-07-30 11:19:00
  • Updated at : 2026-08-14 21:34:29
  • Link: https://puml.online/blog/plantuml-llm-agent-en/
  • License: This work is licensed under CC BY-NC-SA 4.0.
On this page
PlantUML in LLM Agents — visualising reasoning and human-in-the-loop
  1. The interpretability headache for agents
  2. Main use cases
    1. 1. Decision-tree visualisation (ReAct agent)
    2. 2. Multi-agent collaboration (CrewAI / AutoGen)
    3. 3. Tool-call decisions (function calling)
  3. Engineering implementation
    1. Tracking traces in LangChain
    2. Custom callback to emit PlantUML
    3. Trace rendering to console
    4. Live display in a web UI
  4. LangSmith / LangGraph × PlantUML
  5. CrewAI built-in visualisation
  6. ReWOO / Plan-and-Execute agents
  7. Multi-agent state-machine implementation
  8. “Human intervention” during debugging
  9. vs other visualisation tools
  10. Project case: two ReAct agents collaborating
  11. A complete example: debug a failing agent
  12. Recap
  13. Next