Keeping PlantUML diagrams from rotting with unit tests

puml.online
  2026-07-30 16:35 2026-07-30 16:35 Created   2026-08-14 21:34:29 2026-08-14 21:34:29 Updated    

PlantUML lives in .puml files and rots like code does — broken syntax, layout drift, diagrams out of sync with the architecture. This is the three-test approach that makes diagrams a first-class CI citizen.

Why diagrams need tests

Conventional wisdom says “docs don’t need tests.” Wrong.

  • A sequence diagram drawn three months ago: the signature changed but the diagram didn’t → reviewers integrate against the diagram → integration fails
  • Auto-layout changed themes, components overlap → documentation quality drops
  • The architecture was split from monolith to microservices, but the component diagram still shows the monolith → new hires understand the system from a wrong picture

Treat diagrams like code: change the architecture, the test fails, the diagram update gets forced.

Test 1: Syntax test (every PR must pass)

Goal: every .puml renders successfully in CI. Syntax errors, broken !include, missing icons → fail.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# tests/test_plantuml_syntax.py
import subprocess
import pathlib
import pytest

PUML_FILES = list(pathlib.Path("docs/diagrams").rglob("*.puml"))

@pytest.mark.parametrize("puml", PUML_FILES)
def test_puml_renders(tmp_path, puml):
    out = tmp_path / (puml.stem + ".svg")
    result = subprocess.run(
        ["docker", "run", "--rm", "-v", f"{puml.parent}:/data",
         "plantuml/plantuml", f"/data/{puml.name}", "-tsvg", "-o", str(tmp_path)],
        capture_output=True, text=True)
    assert result.returncode == 0, f"{puml} failed: {result.stderr}"
    assert out.exists(), f"{puml} did not produce SVG"

Go version:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
// diagrams_test.go
package diagrams

import (
    "os/exec"
    "path/filepath"
    "testing"
)

func TestPumlRenders(t *testing.T) {
    files, _ := filepath.Glob("docs/diagrams/*.puml")
    for _, f := range files {
        cmd := exec.Command("docker", "run", "--rm",
            "-v", "$(pwd)/docs/diagrams:/data",
            "plantuml/plantuml", "/data/"+filepath.Base(f), "-tsvg")
        if out, err := cmd.CombinedOutput(); err != nil {
            t.Errorf("%s: %v\n%s", f, err, out)
        }
    }
}

What it catches:

  • !include path is wrong → instant fail
  • PlantUML version incompatible → whole CI breaks
  • New hire commits syntactically broken diagram → PR rejected

Test 2: Visual regression test (layout drift)

Goal: when layout changes (components move, arrows get longer) the screenshot diff fires.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
# tests/test_plantuml_visual.py
from PIL import Image, ImageChops
import pathlib
import subprocess

THRESHOLD = 0.05  # 5% pixel diff

def render_puml(puml_path: pathlib.Path) -> pathlib.Path:
    out = puml_path.with_suffix(".png")
    subprocess.run([
        "docker", "run", "--rm", "-v", f"{puml_path.parent}:/data",
        "plantuml/plantuml", f"/data/{puml_path.name}",
        "-tpng", "-o", str(puml_path.parent)
    ], check=True)
    return out

def test_visual_no_regression():
    for puml in pathlib.Path("docs/diagrams").rglob("*.puml"):
        baseline = pathlib.Path("tests/baselines") / puml.with_suffix(".png").name
        new = render_puml(puml)
        if not baseline.exists():
            baseline.parent.mkdir(parents=True, exist_ok=True)
            new.rename(baseline)
            continue
        diff = ImageChops.difference(Image.open(baseline), Image.open(new))
        bbox = diff.getbbox()
        if bbox:
            pixels = sum(1 for px in diff.getdata() if any(c > 5 for c in px))
            ratio = pixels / (new.size[0] * new.size[1])
            assert ratio < THRESHOLD, f"{puml.name}: {ratio*100:.1f}% pixels changed"

Baseline files go into git. Every CI run diffs against them. Over 5% pixel change → fail, forcing the developer to decide: was the layout intentionally changed (update the baseline) or did the theme/border drift (fix the puml).

Test 3: Architecture drift test (diagram vs code in sync)

Goal: every component/interface in the diagram must exist in code; every component in code must appear in the diagram.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# tests/test_architecture_drift.py
import re
import pathlib

PUML_TEXT = pathlib.Path("docs/diagrams/architecture.puml").read_text()

def test_all_services_in_diagram_exist_in_code():
    """Components in diagram must have matching directories in code"""
    services_in_diagram = set(re.findall(r"component\s+\"?(\w+)\"?", PUML_TEXT))
    actual_services = {
        p.name for p in pathlib.Path("services").iterdir() if p.is_dir()
    }
    missing = services_in_diagram - actual_services
    assert not missing, f"In diagram but missing in code: {missing}"

def test_all_services_in_code_are_in_diagram():
    """Code services must appear in the diagram"""
    services_in_diagram = set(re.findall(r"component\s+\"?(\w+)\"?", PUML_TEXT))
    actual_services = {
        p.name for p in pathlib.Path("services").iterdir() if p.is_dir()
    }
    orphans = actual_services - services_in_diagram
    assert not orphans, f"In code but missing in diagram: {orphans}"

How architecture evolution triggers tests:

  • Add new service → second test fails → reminder to add it to the diagram
  • Delete old service → first test fails → reminder to remove from diagram
  • Architecture documentation can never go stale

Force diagrams into ADRs

Michael Nygard’s ADR template recommends every ADR include a “Consequences” diagram.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# ADR-007: Use Redis for session storage

## Status
Accepted

## Context
PHP-FPM sessions are stored locally; multi-host deployment has inconsistent sessions.

## Decision
Introduce Redis for shared sessions.

## Consequences

![session-store](diagrams/adr-007.svg)

```puml
@startuml
component "Web 1" as w1
component "Web 2" as w2
database "Redis" as r
w1 --> r : session
w2 --> r : session
@enduml

Outcome

  • Multi-host session consistency
  • Redis ops overhead
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

Both `adr-007.md` and `adr-007.svg` **go into git together**. The CI syntax test runs the puml block embedded in the ADR too.

## CI integration

```yaml
# .github/workflows/diagrams.yml
name: diagrams
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: syntax test
        run: pytest tests/test_plantuml_syntax.py -v
      - name: visual regression
        run: pytest tests/test_plantuml_visual.py -v
      - name: architecture drift
        run: pytest tests/test_architecture_drift.py -v
      - name: upload visual diffs
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: visual-diffs
          path: tests/diffs/

Reviewers see GitHub Actions status checks:

1
2
✓ syntax test (47 files passed)
✗ visual regression: architecture.puml 12.3% pixels changed

12% change → open the artifact to view the diff → accept or fix.

Cost / payoff

Test Setup Maintenance What it prevents
Syntax 30 min low (only on plantuml upgrade) broken includes, version skew
Visual regression 2 h medium (baseline updates on intentional layout change) theme drift, auto-layout wobble
Architecture drift half day medium (developer keeps diagram in sync) stale docs, new-hire mis-integration

Minimum viable version: start with the syntax test — 30 minutes, blocks 80% of the foot-guns. Add the other two as the team grows.

  • Title: Keeping PlantUML diagrams from rotting with unit tests
  • Author: puml.online
  • Created at : 2026-07-30 16:35:00
  • Updated at : 2026-08-14 21:34:29
  • Link: https://puml.online/blog/plantuml-diagram-as-test-en/
  • License: This work is licensed under CC BY-NC-SA 4.0.
On this page
Keeping PlantUML diagrams from rotting with unit tests
  1. Why diagrams need tests
  2. Test 1: Syntax test (every PR must pass)
  3. Test 2: Visual regression test (layout drift)
  4. Test 3: Architecture drift test (diagram vs code in sync)
  5. Force diagrams into ADRs
  6. Outcome
  7. Cost / payoff