Reading this left me confused, because quantum computers, as they stand today, are not suited for this kind of computation. Yet the motif is popular in science fiction. Adam Fawer’s thriller Improbable has no quantum computer, but there someone turns into a so-called Laplace’s demon: an intelligence, conceived by Pierre-Simon Laplace in 1814, that knows the exact state of every particle in the universe and computes the entire future and past from it. The punchline: long before computing power becomes the issue, the demon fails at his input, because “all the data” is not something that physically exists (more on that below). So the half-sentence “once it has all the data” does not skip some minor detail, it skips exactly the spot where physics says no. Which made me ask, also because I had not followed the state of research for over ten years: are there quantum algorithms for chaotic systems at all, and where exactly does the idea break in practice?

The basic problem: linear versus nonlinear

A quantum computer computes in a fundamentally different way from a classical machine. Instead of bits (0 or 1) it works with qubits, which span huge state spaces through superposition and entanglement. The decisive catch for the weather scenario is the mathematics underneath: quantum mechanical time evolution under the Schrödinger equation is necessarily linear and unitary, probabilities are preserved.

The weather, however, is a highly chaotic system. Chaotic systems are deterministic and still unpredictable in the long run. They are described by nonlinear equations, such as the Navier-Stokes equations for turbulent flows or the simplified Lorenz model. Their defining feature is the butterfly effect: extreme sensitivity to initial conditions. If two initial states differ by as little as a millionth of a degree, their distance grows exponentially over time. This divergence is measured by the positive Lyapunov exponent. Even if the quantum computer were infinitely fast, the smallest measurement error in the input data blows up any exact trajectory prediction after a short time (the Lyapunov time).

So we are forcing a machine that only masters linear operations to solve a nonlinear problem. The currently most studied way to bridge this gap is called Carleman linearization.

Carleman linearization in one equation

The idea is old and elegant. You embed a nonlinear equation into a linear system by treating the powers of the variables as new, independent variables. The clearest way to see it is the simplest nonlinear example, the logistic equation:

Now define \(y_k := x^k\) and differentiate:

That is the whole mechanism. A nonlinear equation becomes a linear system, but one with infinitely many equations: \(y_1\) depends on \(y_2\), \(y_2\) on \(y_3\), and so on. In vector form this is \(\dot{\mathbf{y}} = A\,\mathbf{y}\) with an infinitely large, sparse matrix \(A\). And that is exactly the form a quantum computer likes: linear systems of differential equations can be handled by algorithms like HHL or newer linear solvers in time that grows only logarithmically with the dimension.

The catch: nobody can solve infinitely many equations. You truncate at some order \(N\) and set \(y_{N+1} = 0\). The last row then becomes \(\dot{y}_N = N\,y_N\), a pure growth term. The eigenvalues of the truncated matrix are simply \(1, 2, \dots, N\), all positive. Keep that in mind for a moment.

The experiment

How quickly this truncation breaks down is something an experiment can show. Here is the complete code: it solves the logistic equation exactly (analytically) and compares it with Carleman truncations of different orders, solved via the matrix exponential.

import numpy as np
from scipy.linalg import expm

def carleman_matrix(N):
    """Carleman matrix for dx/dt = x - x^2 with y_k = x^k.
    dy_k/dt = k*(y_k - y_{k+1}). Truncation: y_{N+1} = 0."""
    A = np.zeros((N, N))
    for k in range(1, N + 1):
        A[k-1, k-1] = k
        if k < N:
            A[k-1, k] = -k
    return A

def exact(x0, t):
    return x0 / (x0 + (1 - x0) * np.exp(-t))

def carleman_solve(x0, t, N):
    A = carleman_matrix(N)
    y0 = np.array([x0**k for k in range(1, N + 1)])
    return np.array([(expm(A * tt) @ y0)[0] for tt in t])

t = np.linspace(0, 1.6, 500)
x0 = 0.6
ex = exact(x0, t)
for N in [2, 4, 8, 16]:
    cl = carleman_solve(x0, t, N)
    err = np.abs(cl - ex)
    idx = np.argmax(err > 1e-2)
    print(f"N={N:2d}: breaks down at t={t[idx]:.2f}")

The result as a picture:

Two things stand out.

First: every truncation follows the exact solution perfectly for a while and then tips over abruptly. That is not a rounding error, it is structural. As soon as the truncated order becomes relevant, the growth term \(\dot{y}_N = N\,y_N\) from above takes over and the solution explodes. In the code you see maximum errors of \(10^{30}\) and beyond.

Second: higher order helps, but with clearly diminishing returns. Going from order 2 to 16, eight times the number of variables, only shifts the valid window from \(t \approx 0.2\) to \(t \approx 0.82\).

Why the window does not grow arbitrarily

The second point is the decisive one. There is a parameter that determines whether Carleman converges at all. Liu and colleagues showed in 2021 that the ratio \(R\) of nonlinearity to dissipative linearity decides the situation: for \(R < 1\) there is a genuine quantum advantage, for \(R \ge \sqrt{2}\) the problem is not efficiently solvable in the worst case¹. The logistic equation has a growing linear part instead of a damping one, so it sits deliberately in the hard regime, hence the early breakdown.

In fluid dynamics this \(R\) carries a familiar name. Several works link it to the Reynolds number, which measures precisely the ratio of inertial to viscous forces and thus how turbulent a flow is². Gonzalez-Conde and colleagues tie the convergence to the Kolmogorov scale, the grid resolution needed to resolve the energy cascade of the nonlinear term³. Sanavio and Succi get their lattice-Boltzmann-Carleman variant to work at moderate Reynolds numbers between 10 and 100 already at second order⁴. That is encouraging, but it does not describe real turbulence yet.

From the principle to real chaos: the Lorenz system

The logistic equation demonstrates the mechanism, but it is only one-dimensional and not chaotic. For a well-known chaotic system with several variables, consider the Lorenz system:

with the classic parameters \(\sigma = 10\), \(\rho = 28\), \(\beta = 8/3\). The system has three variables and shows real deterministic chaos with the famous butterfly attractor.

Carleman linearization works analogously: you define monomials \(x^a y^b z^c\) as new variables and derive the linear system in them. At truncation order \(N\) (all monomials with \(a+b+c \leq N\)) the dimension grows quickly: \(N=2\) needs 9 variables, \(N=3\) already 19.

Here is the code:

import numpy as np
from scipy.linalg import expm

def generate_monomials(N):
    """All monomials x^a y^b z^c with a+b+c <= N."""
    monoms = []
    for total in range(1, N+1):
        for a in range(total+1):
            for b in range(total+1-a):
                c = total - a - b
                monoms.append((a, b, c))
    return monoms

def deriv_monom(abc, sigma=10.0, rho=28.0, beta=8.0/3.0):
    a, b, c = abc
    terms = []
    if a > 0:
        terms.append((a * sigma, (a-1, b+1, c)))
        terms.append((-a * sigma, (a, b, c)))
    if b > 0:
        terms.append((b * rho, (a+1, b-1, c)))
        terms.append((-b, (a+1, b-1, c+1)))
        terms.append((-b, (a, b, c)))
    if c > 0:
        terms.append((c, (a+1, b+1, c-1)))
        terms.append((-c * beta, (a, b, c)))
    return terms

def carleman_matrix_lorenz(N, sigma=10.0, rho=28.0, beta=8.0/3.0):
    monoms = generate_monomials(N)
    idx = {abc: i for i, abc in enumerate(monoms)}
    n = len(monoms)
    A = np.zeros((n, n))
    for i, abc in enumerate(monoms):
        for coeff, abc_prime in deriv_monom(abc, sigma, rho, beta):
            if sum(abc_prime) <= N and abc_prime in idx:
                A[i, idx[abc_prime]] += coeff
    return A, monoms

def exact_lorenz_rk4(y0, t, sigma=10.0, rho=28.0, beta=8.0/3.0):
    def rhs(s):
        x, y, z = s
        return np.array([sigma*(y-x), x*(rho-z)-y, x*y-beta*z])
    states = [y0]
    dt = t[1] - t[0]
    for i in range(len(t)-1):
        s = states[-1]
        k1 = rhs(s)
        k2 = rhs(s + 0.5*dt*k1)
        k3 = rhs(s + 0.5*dt*k2)
        k4 = rhs(s + dt*k3)
        states.append(s + (dt/6)*(k1 + 2*k2 + 2*k3 + k4))
    return np.array(states)

def carleman_solve_lorenz(y0, t, N, sigma=10.0, rho=28.0, beta=8.0/3.0):
    A, monoms = carleman_matrix_lorenz(N, sigma, rho, beta)
    idx = {abc: i for i, abc in enumerate(monoms)}
    y0_carleman = np.array([y0[0]**a * y0[1]**b * y0[2]**c for a, b, c in monoms])
    states = []
    for tt in t:
        y_t = expm(A * tt) @ y0_carleman
        states.append([y_t[idx[(1,0,0)]], y_t[idx[(0,1,0)]], y_t[idx[(0,0,1)]]])
    return np.array(states)

# main run
y0 = np.array([1.0, 1.0, 1.0])
t = np.linspace(0, 2.0, 500)
ex = exact_lorenz_rk4(y0, t)
for N in [2, 3]:
    cl = carleman_solve_lorenz(y0, t, N)
    err = np.linalg.norm(cl - ex, axis=1)
    idx = np.argmax(err > 10.0)
    if idx == 0 and err[0] <= 10.0:
        print(f"N={N}: stable until t={t[-1]:.2f}")
    else:
        print(f"N={N}: breaks down (error > 10) at t={t[idx]:.3f}")

The result for the initial value \((1, 1, 1)\):

x(t), y(t), z(t) and the error on a log scale. The exact RK4 solution (black) stays bounded, the Carleman approximations (orange N=2, blue N=3) explode from t ≈ 0.3 on. The x marker shows the respective divergence point, the red line the error threshold of 10.

The difference to the logistic equation is drastic. Higher order barely helps here: from \(N=2\) to \(N=3\) the valid window only shifts from \(t \approx 0.27\) to \(t \approx 0.29\). For real chaos, the Carleman truncation is next to useless.

The exact trajectory forms the butterfly attractor, the color gradient encodes time. The Carleman approximations (orange N=2, blue N=3) follow only briefly and then diverge (x marker).

Fixable or fundamental

Still, the breakdown of the truncated Carleman matrix is not the final verdict on the quantum computer. By now there are workarounds such as pivot switching by Endo and Takahashi⁵ or pivot-shifted Carleman linearization⁶: you shift and rescale the system around a pivot state so that the truncated dynamics no longer inevitably diverges. The exploding matrix is primarily a numerical problem. For regular dynamics it can be defused.

The hard limit sits elsewhere. Lewis and colleagues proved in 2024 that any quantum algorithm that exactly propagates a system with a positive Lyapunov exponent has a complexity that grows at least exponentially with the simulated time⁷. That limit comes from the chaos itself; the Carleman truncation just happens to run into it first. Better code, pivot switching, or more qubits change nothing about it.

For the sci-fi plot this means keeping two things apart. The numerical breakdown of the Carleman method is a bug that clever people can fix. The exponential cost caused by chaos is a structural limit of the mathematics, not of the hardware.

A different line: quantum reservoir computing

The Carleman line tries to solve the differential equation itself. Since around 2023 a second route has established itself that reframes the problem. Quantum reservoir computing (QRC) uses the quantum system as a nonlinear substrate that approximates the attractor from a short observed time series. Only a classical readout layer is trained. The results are remarkable: Ahmed, Tennie and Magri predict extreme events in a turbulent shear flow model with correct timing using a recurrence-free variant⁸, Steinegger and Räth reproduce the short-term behavior and long-term statistics of eight prototypical chaotic systems with four qubits⁹, and Connerty and colleagues ran a quantum echo state network on real IBM hardware, far longer than the coherence times of the chip¹⁰.

What matters is what QRC cannot do. It approximates the attractor and delivers short-term forecasts, not a single trajectory over long horizons. The Lyapunov exponent strikes here just the same, without Carleman and without truncation. QRC shows that quantum computers can indeed take over a certain prediction task. It is just not the one the novel promises: the exact future from complete initial data.

Getting the data in and out

The half-sentence “once it has all the data” hides an assumption that is easy to miss: that the data can get into the machine at all. Algorithms of the HHL family deliver their exponential speedup only if the input is already available as a quantum state. Loading trillions of classical sensor readings into the amplitudes of a quantum register is the state preparation problem, and the standard answer to it, a quantum random access memory (QRAM)¹¹, exists essentially on paper: the circuits it requires are deep and error-prone, and nobody has built one at scale. Aaronson pointed at exactly this fine print years ago: if loading the data costs as much as solving the problem classically, the quantum advantage evaporates¹². So the sci-fi computer does not fail at the computation first, it fails at the loading dock, and by the time the state is prepared, the weather has moved on.

The other end is not free either. If the algorithm produces a quantum state that encodes the solution field, a measurement collapses it and yields only a limited amount of classical information per run. For an expectation value that is fine, for a complete weather field it is not. To be fair, the QRC line has learned to work around the collapse where it hurts most: weak measurement protocols read the reservoir gently enough that the computation can keep running¹³, and feedback-driven QRC performs projective measurements but feeds the results straight back into the circuit as new rotation angles, keeping the memory of the system alive¹⁴. Both tricks keep a running forecast going. Neither hands you the full solution vector of a differential equation.

And then there is the initial data itself. Chaos means sensitive dependence on initial conditions: \(\delta(t) \approx \delta(0)\,e^{\lambda t}\) with a positive Lyapunov exponent. To predict \(T\) time steps ahead you need roughly \(\lambda T\) additional bits of precision in the input. This holds for every computer, classical or quantum. “Having all the data” does not physically exist.

This is exactly the premise Laplace’s demon already hangs on. The demon could own perfect instruments and still fail: the required precision grows without bound with the forecast horizon, so “all the data” would have to mean arbitrarily many decimal places. The premise breaks classically, no quantum mechanics required. With that, the foundation of “once it has all the data” is gone, with or without a quantum computer.

Conclusion

Back to the original question: are there quantum algorithms for chaotic systems, and where does the novel’s idea break? They exist, and they are an active field of research, with Carleman linearization as the common thread. But the gap between “can efficiently simulate other quantum systems” and “predicts the weather once it has all the data” does not hang on the hardware. It hangs on two mathematical facts: linear time evolution meets nonlinear chaos, and the initial conditions would require a precision that does not physically exist. Both are structural, and structure does not yield to better hardware. On top of them sit the practical barriers at both ends, loading the data without a QRAM and reading the solution out through measurements: those are engineering, but engineering nobody has done.

That is also what makes the novel’s idea interesting: it points at real research, but the half-sentence “once it has all the data” quietly defines both problems away. The experiment above makes the first one visible, the second sits in a single exponential function. An afternoon of code is enough to make a whole novel’s premise wobble.

Sources

The answer sits in a 76-line class:

template <typename T>
class Slice {
  public:
    constexpr Slice() noexcept = default;
    constexpr Slice(value_type* data, size_type len) noexcept
        : data_(data), len_(len) {}

    [[nodiscard]] constexpr size_type len() const noexcept { return len_; }
    constexpr value_type* begin() noexcept { return data_; }
    constexpr value_type* end() noexcept { return data_ + len_; }
    constexpr value_type& operator[](size_type index) noexcept { return data_[index]; }

    [[nodiscard]] constexpr SortedSlice<T> sort() {
        // Insertion sort, in place ...
        return SortedSlice<T>(data_, len_);
    }

    constexpr Option<size_type> find(const value_type& value) const noexcept { ... }
    constexpr bool contains(const value_type& value) const noexcept { ... }
};

A Slice<T> owns nothing. It is nothing more than a pair of pointer and length. But this is where all the algorithms live. Not in the containers that hold the data. Exactly one place.

The static_array simply hands out its internal storage as a slice:

constexpr Slice<const T> as_slice() const noexcept {
    return Slice<const T>(data_, N);
}

That is Rust’s &[T] deref coercion in C++ syntax. The consequence: contains, find and sort are written once and usable by any container that can hand out a slice. Add another contiguous container later and it gets search and sort for free, no algorithm code duplicated. static_array::sort() itself is therefore only three lines:

template <typename T, std::size_t N>
constexpr SortedArray<T, N> static_array<T, N>::sort() const {
    auto copy = *this;
    static_cast<void>(copy.as_mut_slice().sort());  // the slice does the work
    return SortedArray<T, N>(std::move(copy));
}

The container makes a copy, hands out a mutable slice over it, and delegates. No sorting algorithm in the container itself. The STL solves this differently: algorithms in <algorithm> as free functions over iterator pairs. A different abstraction with a different trade-off, the length is not bundled in, it follows from end - begin. When you build it yourself you have to commit to one variant and feel in your own code what it costs.

The obvious follow-up: which containers does this even work for? It hangs on a single property, is the memory contiguous or not. From that a clear split follows.

Slice heirs are all containers that allocate contiguously: a fixed-size or growable array, the dense matrices and vectors of a linear-algebra type, the slots of an open-addressing hash table. Whatever the structure, as long as its elements sit back to back in memory it can hand out a slice and inherits every algorithm for free.

Adapters like stack and queue build on a container and deliberately restrict the interface. They do not hand out a slice, because push, pop and top make sense precisely when the container is not sortable and not freely indexable. A stack that hands out a slice is no longer a stack. This is the point where the rule “write once, use everywhere” deliberately stops.

Iterator containers are the structures whose memory is not contiguous: linked lists, trees without an array backing, graphs with adjacency lists, hash maps with separate chaining. Here a fat pointer is pointless, because it would have to point at several scattered regions. Instead of a slice an iterator steps in, closing off the layer downward cleanly. Rust shows the same convention: contiguous containers implement Deref<Target=[T]> and thereby hand out &[T], pointer-backed structures only offer an Iterator. The two are not mutually exclusive: Vec<T> has both, because its iterator is internally the same pointer arithmetic over the same memory.

From that follows the question I ask myself for every new container: array-backed or pointer-backed? Array-backed means slice for free, cache-friendly, with size limits. Pointer-backed means iterator API, more flexible. Both are legitimate, mixing them is the trap: a non-contiguous container that hands out a slice lies about its memory layout. This question has become the first one I ask, before I even start writing a new container.

What surprised me most along the way is how far the principle stretches. static_array::sort() returns no sorted static_array, but a new type:

[[nodiscard]] constexpr SortedArray<T, N> sort() const;

And binary_search is available only on that type:

template <typename T, std::size_t N>
class SortedArray {
  public:
    constexpr BinarySearchResult binary_search(const T& value) const noexcept;
    // sort() does not exist here anymore
};

binary_search on an unsorted array silently violates the precondition of the algorithm: no compiler error, no crash, just a wrong result. The STL solves this by convention: you just sort beforehand, and the compiler does not help. In my implementation binary_search is simply not a member of static_array. It is only available on SortedArray:

arr.binary_search(42);          // compile error: no member
arr.sort().binary_search(42);   // OK

That is exactly the point where the type system makes an assumption of the program logic visible. Rust’s [T]::binary_search() shows the alternative: the method exists on every slice, the precondition lies with the caller, the compiler does not check it. The newtype approach, which enforces the same contract at the type level, you find in Rust as a deliberate design pattern, not as a stdlib feature. Building it yourself, you grasp why that is so. Using the STL, it stays a good tip from a book.

Both observations are ultimately one: build the right abstraction layer, and the algorithms come for free. Not because my 76-line Slice class is better than std::span. But because at every line you make a design decision that stays invisible in the STL. Building the slice myself was the fastest way not just to know Rust’s &[T] deref coercion, but to understand why it exists.

Obsidian, which I use to navigate the notes, only sees the Markdown half. A note about cache line alignment links to a wiki page on CPU architecture, which touches the same topic as a C++ file that implements a memory pool, which is built by a BUILD file that depends on a benchmark library. Obsidian sees two of those connections. The rest are invisible.

I built Synapse to make the whole graph visible. It crawls a repository, extracts references from every file type it understands, resolves them to concrete paths, and serves a force-directed graph over a small HTTP API. No database, no external Go dependencies. D3.js is embedded in the binary; the graph view works offline without any CDN.

What Gets Indexed

The core question for a tool like this is: what counts as a “reference”?

For Markdown files, there are two kinds: standard links [text](path) and Obsidian-style wiki-links [[note name]]. For C and C++ files, local #include "header.h" directives. For Rust, use crate::module and mod submodule declarations. For Bazel BUILD files, the deps, srcs, data, and similar attributes.

The extractor design reflects this. Each file type gets one or more Extractor implementations:

type Extractor interface {
    Extract(content []byte) []RawRef
}

A RawRef is the literal text from the source file, before any path resolution. [[some-note]] in Markdown becomes RawRef{Target: "some-note", Type: EdgeWikiLink}. #include "util/math.h" becomes RawRef{Target: "util/math.h", Type: EdgeCInclude}. The extractor does not know where the file lives; it only reads bytes and returns what it finds.

The dispatch is by extension for most types, and by exact filename for cases where the extension alone is not enough:

var ByExtension = map[string][]Extractor{
    "md":  {markdownLinkExtractor{}, wikilinkExtractor{}},
    "cpp": {cIncludeExtractor{}},
    "rs":  {rustUseExtractor{}, rustModExtractor{}},
    "bzl": {bazelDepExtractor{}},
    // ...
}

// Extension-less files need separate handling.
var ByName = map[string][]Extractor{
    "BUILD":     {bazelDepExtractor{}},
    "WORKSPACE": {bazelDepExtractor{}},
}

The worker pool reads each file and dispatches to both maps:

for _, ex := range extract.ByExtension[ext] {
    refs = append(refs, ex.Extract(content)...)
}
for _, ex := range extract.ByName[filepath.Base(path)] {
    refs = append(refs, ex.Extract(content)...)
}

This turned out to be necessary because BUILD files have no extension, and a lookup in ByExtension[""] would match any extension-less file in the repository.

No Graph Database

The graph itself is a plain in-memory structure. Two maps: paths to node pointers, edge keys to edge pointers. A sync.RWMutex for concurrent ingestion.

type Node struct {
    Path     string   `json:"path"`
    FileType string   `json:"file_type"`
    Tags     []string `json:"tags,omitempty"`
    Outgoing []Edge   `json:"outgoing"`
    Incoming []Edge   `json:"incoming"`
}

type BrainGraph struct {
    nodes map[string]*Node
    edges map[string]*Edge
    mu    sync.RWMutex
}

Each node holds its own adjacency lists. After the crawl, the graph is read-only and the API serves from it directly. For a repository of 1,500 files, this fits comfortably in a few megabytes.

The crawl itself uses a worker pool. Walking the file tree and reading files is I/O bound; multiple goroutines keep the disk busy while others wait. The workers return fileResult structs over a channel; the main goroutine collects results and adds edges to the graph after all workers finish:

jobs    := make(chan fileJob,    len(files))
results := make(chan fileResult, len(files))

for i := 0; i < cfg.Workers; i++ {
    wg.Add(1)
    go crawlWorker(cfg, jobs, results, &wg)
}

// Collect after workers close results channel
for res := range results {
    for _, ref := range res.refs {
        target, resolved := resolver.ResolveRawRef(ref, res.path)
        g.AddEdge(graph.Edge{ Source: res.path, Target: target, ... })
    }
}

Resolution happens centrally because it needs the full file tree. A wiki-link [[Rust Ownership]] needs to be matched against all indexed paths to find zettelkasten/Rust Ownership.md. The extractor only sees one file at a time and cannot do this.

Three Bugs That Only Real Data Reveals

I wrote tests for each extractor. They all passed. Then I ran the crawler against my actual repository and found three classes of errors that no unit test had caught.

C++ attributes look like wiki-links. My Zettelkasten notes contain C++ code examples in fenced blocks. A note about [[nodiscard]] as a design technique wrote it in a code block. The extractor ran the wiki-link regex over the raw bytes and added 113 edges to nonexistent nodes: [[nodiscard]], [[likely]], [[unlikely]], and one memorable [[gnu::target_clones("sse4.2,avx2,avx512f,default")]].

The fix was to strip code regions before running the regex, while preserving all newline characters so that byte offsets for line numbers stayed valid:

func stripCode(src []byte) []byte {
    dst := make([]byte, len(src))
    copy(dst, src)
    // Replace fenced blocks and inline spans with spaces.
    // Newlines are never touched, so line counts stay correct.
    ...
    return dst
}

filepath.Ext has an opinion about book titles. The wiki-link resolver builds a lookup table of lowercase-basename-without-extension -> file path. A note titled Robert C. Martin - Clean Code.md should be resolvable by [[Robert C. Martin - Clean Code]]. It was not.

filepath.Ext("Robert C. Martin - Clean Code") returns . Martin - Clean Code, because “the extension is the suffix beginning at the last dot in the last element of the path.” The last dot in that title is the one after C. The lookup key computed from the link target was "robert c", not "robert c. martin - clean code".

The fix: try the full lowercased basename as a key first, fall back to extension-stripping only if that fails. This handles both note titles with dots and explicit [[note.md]]-style references.

Bazel labels have colons. The Bazel extractor used a regex that excluded colons from label strings: [^":]+. Bazel’s label syntax is //package:target. Every cross-package dependency was silently dropped. The fix was a regex that matches the actual label grammar rather than “everything that is not a quote or colon.”

Tags Need to Be Nodes

The first version of the graph rendered 1,489 nodes with a standard D3 forceSimulation. The result was indistinguishable from a scatterplot. Every node drifted away from every other node with equal force. No clusters, no structure.

The Obsidian graph, visualizing the same notes, shows clear topic regions: a dense area for #rust notes, another for #embedded, a looser cloud for #journal entries. Obsidian treats tags as nodes.

Every note with tags: [rust, embedded] has two edges in Obsidian’s graph: one to a rust node, one to an embedded node. Notes sharing tags end up near each other because they are all connected to the same anchor. The clustering is an emergent property of the graph structure, not a layout algorithm.

Synapse extracts tags from YAML frontmatter for each Markdown file and includes them in the API response. The frontend creates virtual tag nodes from the frequency map and adds note-to-tag edges to the simulation. No tag data is stored server-side beyond the tags field on each node.

The other piece is distanceMax on forceManyBody. Without it, D3’s Barnes-Hut approximation treats distant clusters as combined bodies and generates a global outward pressure that blows every cluster apart. With distanceMax(200), nodes only repel their local neighborhood and distant clusters do not interact at all. [A separate post covers the D3 physics in more detail.][d3-post]

Running against my repository: 1,082 of 1,134 Markdown files have tags, 240 unique tags, top tags are #learning (361 notes), #journal (176), #software-engineering (149), #rust (138), #embedded (123). The graph now shows these as distinct regions with overlap where topics intersect.

What It Shows That Obsidian Does Not

The most useful output is the stats.

My repository has 2,204 edges across 1,489 nodes. 699 of those edges point to files that do not exist: wiki-links to notes I referenced but have not written yet, C++ headers that are system includes, Rust crate references that resolve to external packages. These are knowingly unresolved and that is fine. But examining them surfaces planned notes, gaps in the knowledge base, and occasionally a broken link that used to point somewhere.

There are also 673 orphan nodes: files with no incoming or outgoing edges at all. Most are blog drafts and README files. Some are Zettelkasten notes that have never been linked from anywhere. Seeing them listed is useful: either they should be linked or they should not exist.

The top hub in my repository is zettelkasten/Zettelkasten Methodik.md with degree 87: 71 notes link to it, and it links to 16 others. The second is a wiki page on software architecture. These are the conceptual load-bearing nodes of the second brain, and they are not obvious when browsing the vault file by file.

The connected components count is less reassuring: 877 separate components, meaning the graph is nowhere close to a single connected whole. Most notes live in small clusters of 2 to 10 nodes with no path to the rest.

Running It

go install codeberg.org/lmilz/synapse/cmd/synapse@latest
synapse serve /path/to/repo

The server starts on port 8080. The graph view loads the full index, builds tag nodes in the browser, and lets you filter by file type, minimum degree, or search. All data lives in the Go binary at runtime; there is no configuration file, no schema, no migration.

The source is at githut.com/lmilz/synapse and codeberg.org/lmilz/synapse.

When my second son was born I had a month off and read Tiago Forte’s book: How to Build a Second Brain. PARA and CODE clicked immediately. Finally a framework that made sense. But the longer I worked with it, the clearer the problem became:

All of these approaches treat knowledge as text.

As a developer, my knowledge also lives in code.

PARA structures by purpose, not topic: not “where does this belong?” but “which project needs it?” CODE keeps information moving from capture to expression. Information serves a purpose, or it stays out. Right foundation. But only for text.

The Difference Between Describing and Understanding

PARA answers: “What do I use this for?” When developing, I am often more interested in: “What happens when I change this?”

Code is not an artifact, it is a thinking tool.

I have a note that says: use a tag type to distinguish iterators between containers. But six months later I could not reconstruct why.

// Why not just: using iterator = T*?
// std::iterator_traits<T*> is specialized, all STL algorithms work.
// But then vector<int>::iterator and static_array<int, 5>::iterator
// are the same type — the compiler cannot catch misuse across containers.
//
// Fix: a Tag parameter. Empty struct, private to the container.
// Zero runtime cost, the compiler erases it entirely.
// Each container gets a distinct iterator type. Misuse becomes a compile error.
template <typename T, typename Tag, bool IsConst>
class pointer_iterator { /* ... */ };

// Declared inside static_array — no outside type can share it:
class static_array {
    struct iterator_tag {};
    using iterator = pointer_iterator<T, iterator_tag, false>;
};

The comments are the note. The code is the proof. But that only works if the note and the code can actually live together.

Code as a First-Class Citizen

For a long time I had everything across separate repos: blog, journal, Zettelkasten, small projects. A note about a data structure and the corresponding code lived in different places. Today everything is in a monorepo. A thought and the code that proves it live in the same repository, often in the same commit.

Kepano, the CEO of Obsidian, called it “File over App.” Knowledge does not belong in a tool, it belongs in files you own and can read without anyone running a server. That applies to Markdown just as much as to code, with the difference that code can be executed.

Writing and Seeing

My current setup uses two tools, not one. Neovim for writing: fast, close to the code, with the movements I have known for years. Obsidian for the graph view: I see which notes are connected, which are isolated, where clusters form. That overview I cannot get in Neovim.

For a long time I thought I had to choose: Obsidian or Neovim. Every time the question was: which tool is the right one?

The question was wrong. As long as my notes are plain Markdown in a Git repo, I can use both tools in parallel. Neovim writes, Obsidian shows, both work on the same files. The repo is the source, the tools are views on it.

At the start of this post I wrote that for a long time I could not name why none of the earlier systems worked. Today I can: because they made the tool the source, not the file.

C has no RAII. Every resource you create, you have to release yourself. One function returns a pointer, another takes it back to free it. Between those two calls sits all the code that can go wrong. That leads to three recurring bugs: leaks on early return, when a later call fails and the cleanup code is never reached; double-free, when a pointer gets copied by accident and destroyed twice; and unclear ownership, because the return type alone does not say who is responsible for cleanup.

SDL2 is a typical C library in this situation, and it serves as the concrete example here. std::unique_ptr with a custom deleter solves all three. The trick is connecting the SDL type to its cleanup function:

struct Deleter {
    void operator()(SDL_Window*   p) const noexcept { if (p) SDL_DestroyWindow(p);   }
    void operator()(SDL_Renderer* p) const noexcept { if (p) SDL_DestroyRenderer(p); }
    void operator()(SDL_Texture*  p) const noexcept { if (p) SDL_DestroyTexture(p);  }
    void operator()(SDL_Surface*  p) const noexcept { if (p) SDL_FreeSurface(p);     }
};

using WindowPtr   = std::unique_ptr<SDL_Window,   Deleter>;
using RendererPtr = std::unique_ptr<SDL_Renderer, Deleter>;
using TexturePtr  = std::unique_ptr<SDL_Texture,  Deleter>;
using SurfacePtr  = std::unique_ptr<SDL_Surface,  Deleter>;

One type, four resources. unique_ptr picks the right overload based on the template parameter. The overhead is zero: empty structs with operator() are eliminated via Empty Base Optimization. The noexcept makes explicit what SDL already guarantees through its C ABI, and gives the compiler room for better optimizations.

Direct construction is not quite enough, though. When SDL_CreateWindow fails, it returns nullptr and sets an internal error string. The caller has to check afterward, and SDL_GetError() must be read immediately after the failing call, because later SDL calls overwrite it. A factory function handles this:

[[nodiscard]] inline WindowPtr make_window(std::string title,
                                           uint32_t x, uint32_t y,
                                           uint32_t width, uint32_t height,
                                           uint32_t flags) {
    auto* raw = SDL_CreateWindow(title.c_str(), x, y, width, height, flags);
    if (!raw) throw SDLError("SDL_CreateWindow");
    return WindowPtr(raw);
}

Either make_window returns a valid object, or it throws. No null check at the call site, no chance of an invalid unique_ptr making it into the rest of the code.

SDL_Init and SDL_Quit are a special case: they have process-wide effect and do not fit into unique_ptr. A dedicated class is more direct:

class Subsystem {
  public:
    explicit Subsystem(uint32_t flags) {
        if (SDL_Init(flags) != 0) throw SDLError("SDL_Init");
    }
    ~Subsystem() { SDL_Quit(); }

    Subsystem(const Subsystem&)            = delete;
    Subsystem& operator=(const Subsystem&) = delete;
    Subsystem(Subsystem&&)                 = delete;
    Subsystem& operator=(Subsystem&&)      = delete;
};

Subsystem is not moveable because SDL_Quit affects the whole process. A moved-from object could call SDL_Quit again on destruction while other parts of the program are still using SDL. Deleting the move operations means there is exactly one instance in exactly one place.

The result looks like this:

int main() {
    sdl2::Subsystem sdl{SDL_INIT_VIDEO};
    auto window   = sdl2::make_window("Demo", 100, 100, 640, 480, 0);
    auto renderer = sdl2::make_renderer(window.get(), -1, SDL_RENDERER_ACCELERATED);

    // Cleanup is automatic, in the right order: renderer, then window, then SDL_Quit.
}

The order follows from the declaration order: C++ destroys local variables in reverse. That is exactly the order SDL requires, and the language guarantees it rather than relying on discipline.

The pattern is generic. Any C library that manages resources with create and destroy functions benefits from the same approach: a Deleter struct with one overload per resource type, using aliases for unique_ptr, factory functions that catch nullptr and throw immediately, and for process-wide init and cleanup, a class with deleted copy and move operations. Write it once and you can forget that SDL2 is a C library.

I started with a first simple example, the kind everyone knows and has done before: the famous Hello World. It’s simple, and that’s exactly why it’s useful. You don’t learn a language’s syntax with it, it’s too small for that. You learn how to actually use the language. What file format, how to compile, how to link, how to run the result.

That’s why I always reach for Hello World first whenever I pick up a new language or a new environment. It forces me to run the whole build system once before I do anything else. Two things I learned this way that weren’t obvious to me at the start. First: you can learn a surprising amount from a simple example if you take it seriously. Second: simple is almost never really simple. Most things that look easy are easy because someone else hid the complexity for you.

The embedded world has a Hello World too: the blinking LED. Most microcontroller boards have an onboard LED that you turn on and off at some frequency. Sounds trivial. And it is, if you use a Hardware Abstraction Layer (HAL) and some ready-made project template.

I’ve been working in automotive software for years, and before that on physics simulations at university. In my head I’d always been “close to the hardware”, I write embedded software after all, not frontend. A while back I wrote about the roles of C, C++, and Rust in automotive and quietly took for granted that “embedded equals close to the hardware”. At some point it hit me that this was a delusion. MCAL, AUTOSAR OS, RTE: there are more layers between me and the silicon than between a web app and the kernel. I wanted to actually get down to the bottom for once. No HAL, no framework, no vendor black box. Just the reference manual and the compiler.

A Blinky in Rust

In the Rust ecosystem the example quickly ends up looking like this:

#![no_std]
#![no_main]

use cortex_m_rt::entry;
use panic_halt as _;
use stm32f4xx_hal::{pac, prelude::*};

#[entry]
fn main() -> ! {
    let dp = pac::Peripherals::take().unwrap();
    let rcc = dp.RCC.constrain();
    let clocks = rcc.cfgr.sysclk(48.MHz()).freeze();

    let gpiob = dp.GPIOB.split();
    let mut led1 = gpiob.pb0.into_push_pull_output();
    let mut led2 = gpiob.pb7.into_push_pull_output();
    let mut delay = dp.TIM1.delay_ms(&clocks);

    loop {
        led1.toggle();
        delay.delay_ms(400u32);
        led2.toggle();
        delay.delay_ms(100u32);
    }
}

It’s short, type-safe, and it works. The compiler keeps me from toggling an input pin. The clock config goes through a builder pattern. Pin types carry their configuration in the type system, so toggle() on an input pin is a compile error. The delay is timed off SYSCLK. And everything you don’t see, the vector table, the reset handler, the copy loop for .data, the zeroing of .bss, all of that comes from the cortex-m-rt crate. The linker just gets a small memory.x that tells it where flash and RAM are.

That’s exactly the problem I wanted to dig into. Not in the “the HAL is bad” sense (I actually came away appreciating it more), but: I wanted to see what the HAL does for me. So the same thing again, but in C, no HAL, no CMSIS, just register addresses straight out of the reference manual.

Hello World, embedded

The board I picked was the Nucleo-F446ZE, and I started reading the docs (Reference Manual RM0390, chapter 6 for RCC and chapter 8 for GPIO).

The blinky itself is quickly explained. Enable the GPIOB clock, configure PB0 as an output, in a loop toggle the output register. In C, with no abstraction, it looks like this. First the registers as macros, then main():

#include <stdbool.h>
#include <stdint.h>

#define RCC_BASE   0x40023800UL
#define GPIOB_BASE 0x40020400UL

#define RCC_AHB1ENR  (*(volatile uint32_t*)(RCC_BASE  + 0x30UL))
#define GPIOB_MODER  (*(volatile uint32_t*)(GPIOB_BASE + 0x00UL))
#define GPIOB_ODR    (*(volatile uint32_t*)(GPIOB_BASE + 0x14UL))

#define RCC_AHB1ENR_GPIOBEN (1UL << 1)
#define LED_PIN             0U

static void delay(volatile uint32_t n) {
    while (n--) {
        __asm__("nop");
    }
}

int main(void) {
    RCC_AHB1ENR |= RCC_AHB1ENR_GPIOBEN;

    GPIOB_MODER &= ~(3UL << (LED_PIN * 2));
    GPIOB_MODER |=  (1UL << (LED_PIN * 2));

    while (true) {
        GPIOB_ODR ^= (1UL << LED_PIN);
        delay(500000);
    }
}

Three things about this code need explaining.

First: *(volatile uint32_t*)(...). That’s memory-mapped I/O in its purest form. The hardware exposes certain addresses that don’t point to ordinary RAM cells, but to registers of the peripherals. Writing to RCC_AHB1ENR doesn’t mean “write into a memory cell”, it means “tell the RCC block which clocks to enable”. The volatile cast isn’t a style choice, it’s mandatory. Without volatile, the compiler wouldn’t care how often you write, it would optimize the accesses away as dead stores, and the blinky would silently do nothing. volatile is the contract with the compiler: “hands off, every access has a side effect you can’t see.”

Second: initializing GPIOB_MODER. I clear the two mode bits for PB0 first, then set them to 01 (General Purpose Output). Read-modify-write with &= and |=, so that other pins in the same register stay untouched. On Cortex-M, by the way, this is not atomic, that’s three instructions (LDR, ORR/BIC, STR), and an ISR could fire in between. It works here because no interrupts are active during init. If you actually need atomicity, you use the bit-band region (where available, it’s gone on the Cortex-M7) or LDREX/STREX. For pure set-or-clear on GPIO output pins there’s also the BSRR register, which is specifically designed to let you set or reset individual bits atomically in one write, no read-modify-write required.

Third: delay(). The combination of volatile on the parameter and the explicit nop isn’t decoration. Without volatile, and depending on the optimization level, the compiler may simply skip decrementing the counter, because nobody reads the value. Without the nop, it’s free to collapse the loop body. Together they force the loop to actually run. The comment “500 ms at 16 MHz” is wishful thinking, since the real duration depends on the optimizer, flash wait states, and the pipeline. For a blinky that’s fine, in production you’d use SysTick.

So much for the functionality. The really interesting question isn’t what’s in main(), it’s: how does main() ever get called in the first place? On a PC the operating system does that. On a microcontroller there is no operating system, no loader, no process, nothing that reads in code, allocates memory, or prepares a runtime. Someone has to do all of this by hand. That’s where it got interesting for me.

The hardware doesn’t know about main()

When the ARM Cortex-M4 in the STM32 powers on, it does something very concrete. It reads 4 bytes from address 0x08000000 and loads them as the initial stack pointer. Then it reads the next 4 bytes from 0x08000004, interprets them as an address, and jumps there. That’s not a software instruction, that’s circuit logic, set in silicon. Everything that happens after that is software.

One detail that can cost you hours if you don’t know it: bit 0 of the reset vector address has to be set. The Cortex-M4 only knows the Thumb instruction set, and the CPU uses bit 0 of the jump address as a mode bit. If it’s zero, you get a HardFault right after reset. The linker usually takes care of this for you, but anyone who builds the vector table by hand and has to cast a function pointer symbol themselves will learn this one the hard way.

Which gives us a clear requirement: at address 0x08000000 exactly the right thing has to be sitting there. This structure is called the vector table, and it’s really just an array of function pointers. First entry is the stack pointer (cast as a function pointer, the hardware doesn’t care about the type, it just reads 4 bytes). Second entry is the address of the reset handler. After that come NMI, HardFault, and the other handlers. On an interrupt, the hardware looks into this table, reads the address, jumps there. It’s a hardware jump table, not a software dispatch.

In code, heavily shortened, it looks like this. The full table also has MemManage, BusFault, UsageFault, SVCall, PendSV, SysTick, and then the roughly 80 STM32-specific IRQs:

__attribute__((section(".isr_vector")))
void (*const vector_table[])(void) = {
    (void (*)(void))(&_estack),
    Reset_Handler,
    Default_Handler, /* NMI */
    Default_Handler, /* HardFault */
};

The section(".isr_vector") attribute matters. It tells the compiler: this data belongs in a specially named section. Where that section ends up in memory, though, isn’t decided here. That was the first moment I realized the compiler and the hardware don’t talk to each other directly. Something’s missing in between.

Since the Cortex-M4 is a licensed ARM core, none of this is STM-specific. It works the same way on boards from NXP, Microchip, or TI. Once you’ve understood it once, you can dive right in on a different board.

Sections floating in nothing

The STM32 has two memory regions. Flash, non-volatile, starting at 0x08000000. RAM, volatile, starting at 0x20000000. Both on the same 32-bit address bus. From the CPU’s point of view both regions are equally addressable; which addresses point to flash and which to RAM is decided by how the chip is wired.

The C compiler knows none of this. It takes main.c, produces machine code, puts it into a section called .text. Constants go into .rodata, initialized variables into .data, uninitialized variables into .bss. These are all just names. The compiler has no idea that .text is supposed to end up in flash later and .bss in RAM. It doesn’t even know that flash and RAM exist. The sections have no absolute addresses. They’re just floating in nothing.

So someone has to decide which section ends up at which physical address. That’s the job of the linker script.

The linker script is the floor plan

A linker script is a text file with a .ld extension. It describes two things: which memory regions exist, and which section goes where.

The line

ENTRY(Reset_Handler)

tells the linker where the entry point is.

The MEMORY block lists the physical regions. The numbers come straight from the chip’s datasheet:

MEMORY
{
    FLASH (rx)  : ORIGIN = 0x08000000, LENGTH = 512K
    RAM   (xrw) : ORIGIN = 0x20000000, LENGTH = 128K
}

In the SECTIONS block every section gets assigned to a region. .isr_vector goes at the beginning of flash, because that’s where the hardware reads its first 4 bytes. The KEEP(*(.isr_vector)) keeps the linker from throwing the vector table away, since nothing in the C code explicitly references the vector_table symbol.

.text and .rodata go into flash, because they’re supposed to be non-volatile. .bss goes into RAM, because it gets filled with zeros at runtime.

My favorite part is .data. These are variables with an initial value: int baud_rate = 9600;. At runtime they need to live in RAM, otherwise they aren’t writable. But the initial value has to be stored somewhere before the board ever gets power. So the initial value has to live in flash and get copied into RAM at startup.

The linker script solves this by giving .data two addresses. A virtual address (VMA) in RAM, that’s the address the code expects the variable at. And a load address (LMA) in flash, that’s where the initial values physically sit. Who actually does the copying, the linker script doesn’t say. It only emits boundary markers as symbols: _sidata (start of the initial values in flash), _sdata and _edata (start and end in RAM), _sbss and _ebss for the .bss section.

These symbols aren’t variables in the usual sense. They don’t occupy any memory. They’re just numbers that the linker stamps in at the end. In C you access them by declaring them extern and then taking their address:

extern uint32_t _sidata;
extern uint32_t _sdata;
extern uint32_t _edata;

That confused me for a minute, because they look like variables but aren’t. You never read the value of _sdata, you always use &_sdata. The “variable” is its own address.

The startup code is a mini OS

The linker script sets the boundaries. Filling everything in is the job of the startup code. Traditionally a file called startup.s in assembly, nowadays often startup.c or inline in the same file. I kept mine inline in the blinky, for maximum transparency.

The startup code is the reset handler. It does three things. It copies .data from flash to RAM so that initialized variables have their initial values. It fills .bss with zeros so the C standard for uninitialized variables is upheld. Then it calls main(). With C++ there’s a fourth step: calling global constructors, which runs through a section called __init_array. In an AUTOSAR project this exact work lives inside the supplier’s startup code and runs before EcuM_Init ever sees a register.

In the blinky it looks like this:

void Reset_Handler(void) {
    uint32_t* src = &_sidata;
    uint32_t* dst = &_sdata;
    while (dst < &_edata) {
        *dst++ = *src++;
    }

    dst = &_sbss;
    while (dst < &_ebss) {
        *dst++ = 0;
    }

    main();

    while (true) { }
}

Two things stood out to me when I wrote this for the first time.

First: the startup code uses the linker symbols directly as loop bounds. That’s the contract between the two files. The linker script promises that the symbols are there and point to the right addresses. The startup code just trusts it blindly. If you rename the symbols in the linker script, the startup code breaks silently. Nothing warns you. As an aside: the copy loop runs over uint32_t*, not uint8_t*. That’s faster, one word per bus transaction instead of four bytes, and it works because the linker aligns the sections to 4 bytes. Unaligned 32-bit accesses can trigger a fault on Cortex-M depending on the configuration.

Second: the while (true) {} after main(). On a PC, main() returns to the operating system. Here there is no operating system. If main() accidentally returns, the processor has to go somewhere. The infinite loop is insurance against it running wild through memory.

The only reason the startup code can run at all, before the C environment is set up, is that it uses only stack-local variables. And the stack already works because the hardware loaded the stack pointer from the vector table at reset. The whole sequence is a domino chain. Each step has exactly the one precondition the previous step just created.

From .c to bytes in flash

What happens between source and a blinking LED is also not one step, but several. The preprocessor resolves includes and macros. The compiler translates each .c file individually into assembly, architecture-specific via -mcpu=cortex-m4 -mthumb. The assembler produces object files in ELF format, with relative addresses and unresolved symbols. The linker collects all sections of the same name from all object files, assigns them absolute addresses via the linker script, and resolves the symbols. What used to say “jump to delay” now carries the concrete flash address of delay.

The final product is an ELF file. It contains the machine code, but also program headers (which bytes go where in memory), the symbol table (function and variable names with their addresses, for the debugger), and optionally debug information (the mapping of machine code to source lines). A .bin file, which you produce via objcopy -O binary, is just the raw bytes without any metadata.

To flash it, you use a tool like probe-rs or OpenOCD. The tool talks over a debug adapter (ST-Link, J-Link, or CMSIS-DAP) to the Cortex-M4’s SWD port (Serial Wire Debug). The debug port has direct access to the entire address space of the chip, regardless of whether the CPU is running. The tool reads the ELF file, extracts the program headers, writes the bytes into flash, and triggers a reset. After which the whole cycle starts over. Hardware reads the vector table, jumps to the reset handler, startup initializes, main() runs, LED blinks.

I want to stress again that this is a little learning project of mine, not production code. In production you’d put an abstraction on top, either a vendor HAL (for example ST’s) or the Cortex Microcontroller Software Interface Standard (CMSIS), which is vendor-agnostic.

What I took away

Two things stuck with me.

The first: a Hello World is very much not a waste of time if you take it seriously. I could have written the blinky with a HAL in five lines, done. Instead I wrote the vector table myself, the reset handler, read the linker script, understood the boundary markers. And I learned more about the system than I would have in weeks of framework tutorials.

The second: simple is almost never actually simple. The blinky with a HAL isn’t any easier than the blinky with registers, it’s just further away from what’s actually happening. Between make flash and a blinking LED sit the linker script, the startup code, the ELF, the flashing, the hardware reset. All of this is there, even when you don’t see it.

For those of us in automotive this is particularly interesting. In a classical AUTOSAR project we never see the vector table, never see the reset handler, never see the linker script. The MCAL, the supplier’s startup code, the OS init, BswM scheduling: all of that arrives as a black box, and we write runnables that get called from the RTE. Between power-on and Rte_MainFunction_* there’s the exact same chain as here. Hardware reads 4 bytes, jumps to the reset handler, someone initializes .data and .bss, someone calls the OS init, someone starts the tasks. It’s just that each of those steps is buried inside an AUTOSAR configuration we normally don’t open. Once you’ve built this foundation yourself, you read an MCAL doc differently. It also sharpens the language question. In the earlier post I argued that C, C++, and Rust each have their layer, but the layer we usually work on in AUTOSAR is several steps above the one where that choice really matters. The reset handler, the .data copy, the linker script, those are all C, regardless of what we write on top. The supplier picks the language down there, we don’t.

Not seeing it is comfortable as long as everything works. The moment it doesn’t, you have to go a level deeper. And then it’s good to have been there before.

Over the past few weeks I ran into a familiar problem: my personal projects started to feel like work.

Family, work, and then spending evenings on side projects slowly drained my motivation.

So I asked myself: what is actually bothering me, and where does it make sense to invest personal energy? The answer came quickly: my homelab. Not because I didn’t have one, but because I had been neglecting it. A few Docker containers were running on the Raspberry Pi that I had tried out and then forgotten about. The NAS, which was supposed to handle backups, never really worked as planned. And my self-hosted Git instance was only reachable within my home network, which made it almost useless for day-to-day work.

The idea was clear: build a simple setup that automates a lot, is as open source as possible, and takes privacy seriously.

The Stack

The goal was a small, practical stack that solves real problems rather than hosting everything under the sun.

As a foundation I rented a VPS from a German provider for a few euros a month. Everything runs behind Caddy as a reverse proxy, which also handles HTTPS automatically without any configuration on my part.

The first thing I set up was Forgejo as a Git server, reachable via its own subdomain. That sounds small, but for me it made a real difference: I can now work on private projects from anywhere, whether on a train using a hotspot or somewhere else. At the same time I moved my website from GitHub Pages to the new repo and set up a deploy workflow so that the site gets automatically built and deployed on every push. That actually turned out to be smoother than using GitHub Pages.

After that came CommaFeed as an RSS reader and Readeck as a read-it-later app. Both were set up quickly, and the value was immediately noticeable: I can now read anywhere, save articles, and add notes directly.

Up to this point: solid, but unremarkable self-hosting. The interesting part came after.

The Problem with Notes

I read a lot. And like most developers, I have a familiar problem: I save articles, read them, take notes, and then they disappear into the void. The highlights collect dust, the insights fade. I do use the Zettelkasten method for this. But sometimes I simply lacked the focus and time to turn my notes into actual Zettelkasten entries.

While thinking about which other containers I wanted to host, I remembered n8n. I had watched several YouTube videos about what n8n can do.

So I hosted that too and started experimenting. My goal was clear: everything I read and find valuable should flow automatically into my Zettelkasten. Not as a bland summary, but as a thoughtful note that reflects my own thinking and highlights.

The Idea

My setup consists of three components connected through n8n:

• Readeck: where I save, read, and highlight articles
• Claude (via the Anthropic API): which generates a Zettelkasten note from my highlights
• Forgejo: where my Zettelkasten lives as Markdown files

The key idea is that the workflow never sends the entire article to the language model. Instead it only sends my highlights, notes, and a short summary. This keeps it privacy-friendly, saves tokens, and most importantly the generated note actually reflects my understanding rather than a generic summary.

The Reading Process

Before the workflow can do its thing, I need to do my part. My reading process in Readeck looks like this: I read an article and mark the most important passages as highlights. For each highlight I write a short note with my thoughts. At the end I write an overall summary on one of the highlights, prefixed with “Zusammenfassung:”. Finally I assign the label Done, which signals to the workflow: this article is ready.

This convention is essential:

• the Done label triggers the workflow
• the Zusammenfassung: prefix tells the LLM what the central idea is

The Readeck API

Readeck offers a REST API and since version 0.22 there are also annotations (highlights with notes) and a Markdown export endpoint that is worth its weight in gold for this workflow.

The endpoint GET /api/bookmarks?labels=Done returns all articles I have marked for processing. For each bookmark I then fetch the content as Markdown via GET /api/bookmarks/{id}/article.md. The nice thing about this: the endpoint delivers not just the article text, but also the highlights as ==highlighted text== and my notes as footnotes [^1], all in a clean Markdown document with YAML frontmatter.

The n8n Workflow

The workflow consists of a loop that processes each article individually. The full chain looks like this:

A Schedule Trigger starts the workflow every night at 2am. An HTTP Request then fetches all bookmarks with the label Done from Readeck. A Loop Over Items node iterates over each bookmark. Inside the loop another HTTP Request fetches the article as Markdown. A Code Node parses the Markdown: it extracts the frontmatter, separates the highlights from the regular text, looks for my Zusammenfassung:, and assembles the input for the language model. A Basic LLM Chain sends everything to the Anthropic API and gets a Zettelkasten note back. A second Code Node encodes the LLM response as Base64 for the Forgejo API and generates a slugified filename. Three HTTP Requests against the Forgejo API create a feature branch, commit the file, and open a pull request. A final HTTP Request updates the label in Readeck from Done to processed.

Parsing: Extracting Highlights

The core is the Code Node that takes the Markdown article apart. It extracts the highlights (everything between == characters), maps them to the footnote notes, and specifically looks for my summary.

// Extract highlights with footnote number
const highlightRegex = /==([^=]+)==(?:\[?\^(\d+)\]?)?/g;

// Extract footnotes
const footnoteRegex = /\[\^(\d+)\]:\s*(.*?)$/gm;

// Find summary
for (const h of highlightPairs) {
  const note = h.footnoteNum ? (footnoteMap[h.footnoteNum] || '') : '';
  if (note.toLowerCase().startsWith('zusammenfassung:')) {
    zusammenfassung = note.replace(/^[Zz]usammenfassung:\s*/i, '');
  }
}

The output then contains only the frontmatter, my summary, and the individual highlights with notes, no full article text.

The Prompt

The prompt is designed to generate an atomic Zettelkasten note that reflects my understanding. It took a few iterations before the results felt right.

Key aspects: the LLM should use my summary as a guide and weight it more heavily than individual highlights. The note should distill one central idea, not retell the article. Tags should be thematic and connections to other areas of knowledge should be written as running text. The output format is a fixed Markdown template with YAML frontmatter that fits directly into my Zettelkasten.

A typical note generated by the workflow looks like this:

---
title: "Deliberate Practice Is the Bottleneck of Learning Software Development"
date: "2025-11-12"
tags: [learning, deliberate-practice, software-craftsmanship]
source: "https://example.com/deliberate-practice-programming"
author: "Jane Doe"
type: literature
---

## Core Idea
Improvement in software engineering does not primarily come from accumulated experience but from deliberate practice with feedback. Writing code every day is not the same as intentionally improving one’s skills.

## Note
The central idea of the article is that many developers spend most of their time performing rather than practicing. Day-to-day work usually focuses on delivering features or fixing bugs, not on deliberately improving specific programming skills.

Deliberate practice requires isolating a skill and working on it intentionally. In software engineering this could mean refactoring a small piece of code multiple times, implementing a known algorithm from memory, or experimenting with different design approaches for the same problem.

Another important aspect is feedback. Without feedback it is difficult to know whether a mental model is actually improving. Code reviews, pair programming, and discussions about architecture decisions can provide exactly this kind of feedback loop.

This aligns with my own experience. Many side projects feel productive but often do not improve specific skills. Short, focused exercises such as refactoring a function or re-implementing a concept from scratch tend to lead to deeper learning.

## Highlights
> Expertise is not the result of experience alone, but of structured and deliberate practice.

*Experience alone does not automatically improve programming skills. Improvement requires consciously focusing on specific aspects of the craft.*

> Most professionals spend their time performing, not practicing.

*In daily work the focus is on delivering results rather than improving individual skills.*

> Feedback is the mechanism that turns repetition into learning.

*Practices like code reviews or pair programming create the feedback loops that deliberate practice depends on.*

## Connections
The idea of deliberate practice originates from learning psychology and is widely applied in areas such as music or sports. In software engineering it aligns closely with ideas from the software craftsmanship movement, which emphasizes continuous improvement of programming skills. It also connects to practices such as coding katas and focused refactoring sessions, where the primary goal is learning rather than delivering features.

The Forgejo PR Flow

Instead of committing directly to the main branch, the workflow creates a feature branch and a pull request. This gives me the chance to review and adjust the generated note before merging, in the morning over coffee, without rushing.

The three Forgejo API calls are straightforward: POST /api/v1/repos/{owner}/{repo}/branches creates a branch like zettelkasten/2026-03-15-atomic-habits. Then POST /api/v1/repos/{owner}/{repo}/contents/{path} commits the Markdown file to that branch. Finally POST /api/v1/repos/{owner}/{repo}/pulls opens the pull request.

Cost and Performance

The Anthropic API is billed through prepaid credits. Running one article through Claude Sonnet costs a few cents, since I only send the highlights and metadata rather than the full article text. A budget of 5 euros per month is more than enough.

Stumbling Block

n8n is marketed as a no-code automation platform. But here comes the catch. It works great with mainstream tools like GitHub, which can be connected quickly and easily.

For tools like Forgejo and Readeck you basically have to model the REST calls by hand, exactly as in my workflow. You can import curl commands, but still need to adjust a few things. That raised a question for me: would a script not have been a better solution here? Because apart from the Anthropic API call, I essentially modelled the REST calls directly in n8n.

That said, I will keep using n8n because I am still just getting started with it. There might be better approaches I have not discovered yet. And it gives me a lot of flexibility without having to wrestle with bash syntax.

Conclusion

The entire workflow took one afternoon, from the first Readeck API call to the finished pull request in Forgejo. Most of the time actually went into refining the prompt and parsing the highlights correctly.

The result is a system that connects my reading process seamlessly to my Zettelkasten, without manually transferring notes.

What I like most: the workflow forces me to read deliberately. I have to set highlights, write notes, and compose a summary before an article gets processed. The language model amplifies my thinking, it does not replace it.

The complete stack is self-hosted: Readeck, n8n, and Forgejo run on my own infrastructure. Only the LLM call goes to the Anthropic API. In the future I could imagine replacing that with a local model too, but the quality of Claude Sonnet for this task is hard to beat right now.

And the original motivation problem? Gone. When a project actually improves something you deal with every day, it becomes fun again.

And yet they are all saying the same thing: producing more is not the same as creating value.

The Common Thread

Newport’s Slow Productivity rests on three principles: do fewer things, work at a natural pace, obsess over quality.

Kim’s The Phoenix Project introduces the Three Ways: flow, feedback, and continuous learning. His follow-up The Unicorn Project describes the Five Ideals: locality and simplicity, focus and flow, improvement of daily work, psychological safety, customer focus.

Farley’s Modern Software Engineering argues that software development is an exercise in learning and discovery. He emphasizes iterative work, fast feedback, and empirical approaches over big upfront design.

Martin’s Clean series makes the case for professionalism and quality as non-negotiable foundations. Code should be readable, systems should be maintainable, and developers should take pride in their craft.

Strip away the terminology and the pattern is clear:

• Less is more. Fewer concurrent projects, smaller batches, shorter feedback loops.
• Flow over busyness. Uninterrupted focus beats fragmented multitasking.
• Quality is not a phase. It is built in continuously, not inspected in at the end.
• Trust enables speed. Autonomy and psychological safety are not luxuries, they are prerequisites for good work.
• Learning is the work. Software development is discovery, not manufacturing.

This is not news to anyone who has been paying attention. What is harder to find are concrete examples of what this looks like in practice, especially in regulated industries where process overhead is often seen as inevitable.

When Everything Clicks: A Greenfield Project

I once worked on a greenfield project where I rebuilt a component from scratch in C. It was embedded software in a regulated industry, which usually means heavy process overhead, but this project was different.

We had a strong product manager who spoke directly with customers and kept us developers in close, ongoing exchange. That meant we understood what customers actually needed without waiting for a formal technical specification. This was unusual. In most projects I had been on, requirements arrived as finished documents, weeks after the customer conversation had happened.

The workflow stayed tight throughout. I would refine a piece of the design, write the tests, implement the change, and open a pull request. Communication happened asynchronously over chat. Code reviews came back fast. We could deploy directly onto test hardware and get immediate feedback, not just on our own component but on how it behaved in the full system. Small changes, integrated frequently, validated against reality rather than assumptions.

What made it work was the presence of trust. No one told me which design pattern to use or how to structure my modules. I could start building before having every detail figured out, try things, fail fast, learn, and iterate.

The principles from the books were not something I applied consciously. They just described what was already happening when the conditions were right.

When It Is Hard: Legacy Systems

But here is the question I kept asking myself: does this only work for greenfield projects? What about the legacy systems, the inherited codebases, the regulated environments where everything feels immovable?

I have worked on several of those too. And the answer is: the same principles apply. It just requires a different approach.

The improvements came through deliberate, incremental steps:

• Start with requirements. Reduce them to the essentials. Focus on what and why, not how. Bloated specifications create bloated software and tests. When you force clarity at the requirements level, you often discover that half the complexity was never necessary.
• Improve the documentation. This sounds mundane, but it is powerful. The act of documenting what the system actually does, not what it was supposed to do, gives you a first sense of where the architecture has drifted and where refactoring is possible.
• Write tests against the legacy code. Unit tests, component tests, whatever you can get in. They are your safety net for everything that follows. Without tests, every change is a gamble. With tests, you can move with confidence.
• Modularize through refactoring. And critically: remove unnecessary code entirely. Dead code is not harmless. It is confusion waiting to happen. Every line you delete is a line no one has to understand, maintain, or debug.
• Reduce tool fragmentation. Instead of a separate tool for every step, limit yourself to two or three. Every tool switch is a context switch, and context switches kill flow.
• Keep the team small and the communication direct. Every handoff, from requirements engineer to architect, to developer, to tester, loses information. In my experience, a small group of developers working closely with a strong product manager creates far more value than a large team with many handoff points.

This is Kim’s Third Ideal: improvement of daily work. The Scout Rule, popularized by Martin: leave the code cleaner than you found it. It is Farley’s emphasis on managing complexity through continuous, small improvements. It is not a separate initiative. It is how the work gets done.

The Myth of Regulatory Constraints

There is a common excuse in regulated industries: “We would love to work this way, but compliance makes it impossible.”

I do not buy it.

Standards like ASPICE describe best practices. You need requirements, architecture, design, and tests. You need traceability and verification. But ASPICE does not prescribe how heavyweight those need to be. You can do all of this lean. The standard asks that you demonstrate rigor, not that you drown in documents.

On one project, we replaced huge specification documents with a structured set of few requirements, each linked directly to its test. The auditor accepted it. The team could actually maintain it.

The real barrier is rarely the regulation. It is the assumption that regulation demands bureaucracy. Once teams realize they can improve incrementally, within the rules, the path opens up.

I have seen this happen. Teams that thought they were stuck discovered they had more freedom than they assumed. The constraint was not the standard, it was their interpretation of it.

Kim describes exactly this in The Phoenix Project: the bottleneck is rarely where you think it is. Often the biggest constraint is not technical or regulatory, it is organizational belief.

The Work That Does Not Show Up on the Board

There is a less obvious effect of this kind of work, one that affects the people writing the code as much as the code itself.

There is a concept in psychology called self-efficacy: the belief that your actions matter and that you have agency over how you do your work. Research consistently shows it is one of the strongest predictors of motivation, performance, and resilience in knowledge work.

This is what depth over busyness enables. Not just better software, but better working conditions. The sense of ownership over technical decisions. The tight feedback loop. The freedom to explore and recover on your own terms.

The best developers I have worked with all share one quality: they see work that others overlook.

They notice that a module has grown too complex, that an abstraction no longer fits, that a dependency could be removed. They write the test that will save three hours of debugging next month. They improve the documentation so the next person does not have to reverse-engineer the code. They delete the dead code that everyone else steps around.

This work rarely shows up on a board. There is no ticket for “make tomorrow’s feature possible.” No story points for removing confusion. But it is often the reason the next feature can be built cleanly instead of hacked on top of existing debt.

I think of it as gardening. A garden is never “done.” It grows, it changes, and it needs continuous care. Good developers treat codebases the same way. They do not wait for permission to pull weeds. They do not need a feature request to improve the soil. They understand that small, consistent effort prevents the kind of decay that eventually requires a rewrite.

The job of a software engineer is not to produce code. It is to reduce complexity and create value. Doing fewer things is not laziness, it is judgment. Working at a natural pace is not slowness, it is sustainability. Obsessing over quality is not perfectionism, it is professional responsibility.

Starting Today

The conditions for this kind of work can be cultivated. Not everywhere, not always, but more often than we assume. It starts with one project, one team willing to experiment with fewer meetings, smaller batches, more trust.

Newport, Kim, Farley, and Martin come from different worlds. But they all arrived at the same place: less, but better. The only question left is whether we are willing to do less, so we can build something that matters.

It was a relaxed evening on vacation. No todo list, no deadline, no code review. Just me, my laptop, and the idea to code something for fun.

Spoiler: It didn’t stay simple. But that’s exactly why this story is worth telling.

A few days earlier, I had found my old bachelor thesis while cleaning up. The topic back then was a particle simulation. Geometric shapes get randomly distributed in a 2D space. When two particles overlap, they push each other away. Over time, order emerges from chaos. Physicists call this Random Organization.

I really enjoyed that project back then. There were conference talks, even a published paper. I was proud of the physics results.

Less proud of the code.

How did I develop back then? Chaotic. No version control, no tests, no documentation. The result: The code is gone. No backups, nothing.

But the idea was still there. And now I also had the knowledge of how to do it better. So that evening, I started rewriting the simulation. This time properly.

Rapid Prototyping

Was my old code bad? Yes. But that’s okay. Back then I was a physicist, not a developer. Code was a means to an end.

Today, after several years as a professional developer, I see it differently. Code is communication with the computer, with other developers, with my future self.

So I started fresh. Quick, without thinking too much. Get something running first. The prototype worked. The simulation did what it was supposed to. But I wanted to do better. So I started refactoring.

Design Patterns: Bringing Structure to Chaos

The prototype worked. But reading through the code, I realized this can be better. Over the past years, I had learned a lot about design patterns and used many. Patterns aren’t magic. They’re documented solutions to problems other developers had before me. Why reinvent the wheel?

So I looked at the code again. Where does something repeat? Where is the code inflexible? Where are potential bugs lurking?

1. Factory Pattern: Three Methods Are Two Too Many

In the prototype, I had three almost identical methods:

void Simulation::initCircleSetup(uint16_t num) { /* ... */ }
void Simulation::initRectangleSetup(uint16_t num) { /* ... */ }
void Simulation::initSquareSetup(uint16_t num) { /* ... */ }

Classic Copy Paste Programming.

Plus there was a string comparison for selecting the type:

if (cfg.type == "circle") {
    initCircleSetup(cfg.num_particles);
}
else if (cfg.type == "rectangle") {
    initRectangleSetup(cfg.num_particles);
}
else if (cfg.type == "square") {
    initSquareSetup(cfg.num_particles);
}
else {
    initCircleSetup(cfg.num_particles);  // Silent fallback
}

What happens with a typo? “cicle” instead of “circle”? The program silently falls back to the default. No warning, no exception. The bug only gets discovered hours later during debugging.

After refactoring, the code looks like this:

// ShapeType.hpp
enum class ShapeTypes {
    Circle,
    Rectangle,
    Square
};

class ShapeType {
public:
    static Shape create(ShapeTypes type, const Vec& pos,
                       double size1 = 1.0, double size2 = 1.0) {
        switch (type) {
            case ShapeTypes::Circle:
                return Circle(pos, size1);
            case ShapeTypes::Rectangle:
                return Rectangle(pos, size1, size2);
            case ShapeTypes::Square:
                return Square(pos, size1);
        }
        throw std::invalid_argument("Unknown shape type");
    }
};

// Simulation.cpp
void Simulation::initParticles(uint16_t num)
{
    std::uniform_real_distribution<double> xDist(0.0, config.area_width);
    std::uniform_real_distribution<double> yDist(0.0, config.area_height);

    particles.reserve(num);
    for (uint16_t i = 0; i < num; i++) {
        Vec pos(xDist(rng), yDist(rng));
        particles.push_back(ShapeType::create(config.shape_type, pos));
    }
}

The result: Three methods become one. The DRY principle (Don’t Repeat Yourself) in action. Plus type safety through the enum class. A typo in ShapeTypes::Cicle leads to a compiler error, not a silent bug at runtime.

Shape creation is now in a central place. If the creation changes, for example because Circle now needs two parameters instead of one, I change it in one place. Not three. And a new shape? Extend the enum, add a case in the factory, done.

2. Strategy Pattern: More Than Just True or False

The next problem was the boundary condition. In the prototype, I only had a boolean:

class Simulation {
    bool periodicBoundary;  // true or false, nothing more
};

That works for the simple case. But what if I want reflective boundaries or hard walls? With a boolean, that’s not possible. I’d need an enum and then a switch statement that grows with every new boundary. The Simulation class would keep getting bloated.

The solution: The Strategy Pattern. Instead of having the logic in the Simulation class, it gets extracted into separate classes. The simulation only knows the interface and doesn’t know which concrete implementation is behind it.

// Boundary.hpp
enum class BoundaryTypes {
    Hardwall,
    Periodic,
    Reflective
};

class Boundary {
public:
    virtual ~Boundary() = default;
    virtual void apply(Shape& particle, double width, double height) const = 0;
};

class PeriodicBoundary : public Boundary {
public:
    void apply(Shape& particle, double width, double height) const override { /* ... */ }
};

class ReflectiveBoundary : public Boundary {
public:
    void apply(Shape& particle, double width, double height) const override { /* ... */ }
};

class HardwallBoundary : public Boundary {
public:
    void apply(Shape& particle, double width, double height) const override { /* ... */ }
};


// Simulation.cpp
void Simulation::randomPush(Shape& particle1, Shape& particle2) {
    // displacement logic
    move(particle1, displacement);
    move(particle2, displacement * (-1.0));

    // Apply boundary
    boundary_->apply(particle1, config.area_width, config.area_height);
    boundary_->apply(particle2, config.area_width, config.area_height);
}

The beauty of this: The Simulation class never needs to be touched again when a new boundary is added. That’s the Open/Closed Principle in practice. The code is open for extension but closed for modification.

Plus: Each strategy can be tested in isolation. A unit test for PeriodicBoundary, one for ReflectiveBoundary, without having to spin up the entire simulation.

3. Builder Pattern: Configuration Without Silent Failures

For the simulation configuration, I had a simple struct:

SimulationConfig config;
config.num_particles = 50;
config.area_width = 50;
config.area_height = 50;
config.max_displacement = 0.5;
config.max_iterations = 5000;
config.boundary_type = BoundaryTypes::Periodic;

Simulation sim(config);

That works. But what if someone forgets to set num_particles? Division by zero, somewhere deep in the code.

The solution is the Builder Pattern. Instead of manipulating the struct directly, there’s a builder that gets configured step by step and validates the input:

auto config = SimulationConfiguration()
    .withShapeType(ShapeTypes::Circle)
    .withParticles(50)
    .withArea(50.0, 50.0)
    .withMaxDisplacement(0.5)
    .withMaxIterations(5000)
    .withBoundaryType(BoundaryTypes::Hardwall)
    .build();

Simulation sim(config);

The trick: The SimulationConfig struct has sensible default values. In build(), it checks if all values are valid. Is a value invalid? Exception. No silent failure. Plus, the code reads almost like a sentence. withParticles(50) is clearer than config.num_particles = 50. The methods document themselves.

The Moment Everything Collapsed

I was proud. Factory, Strategy, Builder. Everything there. The code looked professional. I even wrote tests. Time for a demo.

Circles? Perfect. Rectangles? Works. Circles AND Rectangles mixed? Segmentation Fault.

No warning. No exception. Just crash.

I stared at the screen. “This can’t be. I did everything right!”

After a debugging session, I found the culprit. The classes for geometric shapes are built according to classic inheritance hierarchy:

class Shape {
public:
    virtual ~Shape() = default;
    virtual bool overlaps(const Shape& other) const = 0;
protected:
    Vec position;
};

class Circle : public Shape {
public:
    bool overlaps(const Shape& other) const override;
private:
    double radius;
};

The problem lurked in the overlaps implementation:

bool Circle::overlaps(const Shape& other) const
{
    const Circle* otherCircle = dynamic_cast<const Circle*>(&other);

    Vec diff = position - otherCircle->position;  // CRASH!
    double distance = diff.magnitude();

    return distance < (radius + otherCircle->radius);
}

The problem: dynamic_cast returns nullptr if other is not a Circle. And what do I do? I dereference the pointer directly without checking. Classic mistake.

The quick solution would be a nullptr check:

if (!otherCircle) {
    throw std::invalid_argument("Circle::overlaps called with non-Circle");
}

But that felt wrong. I had put a lot of effort into design patterns and now I’m supposed to add nullptr checks everywhere? Throw runtime exceptions? That’s exactly the kind of defensive code I wanted to avoid.

The problem wasn’t the implementation. The problem was the design.

The Search for a Better Solution

I researched. How do other developers solve this problem?

Double Dispatch? Possible, but lots of boilerplate. Visitor Pattern? Works, but even more code.

And then I came across std::variant.

A feature from C++17 that I had heard of but never really understood. It comes from functional programming and solves exactly my problem. Not with runtime checks, but at compile time.

What Is a Sum Type?

std::variant is a sum type: A value can be exactly ONE of several types. Not “maybe one”, not “multiple simultaneously”. Exactly one. Always. No nullptr.

std::variant<int, double, std::string> value;
value = 42;         // now int
value = 3.14;       // now double
value = "hello";    // now string

The crucial difference to classic inheritance: With inheritance, the type hierarchy is open. Anyone can derive new classes from Shape. That makes it easy to add new types. But adding a new operation? Then every class needs to be touched. Every class needs a new method.

With sum types, it’s the opposite. The type set is closed. All types are known at compile time. That makes it harder to add new types because all handlers need to be adjusted. But a new operation? Just write a new handler. The existing code stays untouched.

For my simulation, that means: My shapes are known. Circle, Rectangle, Square. New shapes? Rare. But new operations? Overlap check, rendering, collision response? Those come all the time.

Sum types fit better here than inheritance.

The killer feature: The compiler enforces completeness. Forget a case, you get a compiler error. Not a crash at runtime. Not a nullptr. An error at compile time.

From Classes to Data Structs

Before, I had a class hierarchy:

class Shape { virtual bool overlaps(...) = 0; };
class Circle : public Shape { ... };
class Rectangle : public Shape { ... };

std::vector<std::unique_ptr<Shape>> particles;

After, I have pure data structs:

struct CircleData {
    Vec position;
    double radius;

    CircleData(const Vec& pos, double r) : position(pos), radius(r) {}
};

struct RectangleData {
    Vec position;
    double width;
    double height;

    RectangleData(const Vec& pos, double w, double h)
        : position(pos), width(w), height(h) {}
};

struct SquareData {
    Vec position;
    double size;

    SquareData(const Vec& pos, double s) : position(pos), size(s) {}
};

using Shape = std::variant<CircleData, RectangleData, SquareData>;

Since shapes are now pure data structs, not classes with methods, operations are implemented as free functions:

Vec getPosition(const Shape& s)
{
    return std::visit([](const auto& shape) { return shape.position; }, s);
}

void setPosition(Shape& s, const Vec& pos)
{
    std::visit([&pos](auto& shape) { shape.position = pos; }, s);
}

void move(Shape& s, const Vec& delta)
{
    std::visit([&delta](auto& shape) { shape.position = shape.position + delta; }, s);
}

For collision detection, we create a handler with overloaded operator():

struct OverlapHandler {
    bool operator()(const CircleData& a, const CircleData& b) const
    {
	// Implementation of circle-circle collision
    }

    bool operator()(const RectangleData& a, const RectangleData& b) const
    {
	// Implementation of rectangle-rectangle collision
    }

    bool operator()(const SquareData& a, const SquareData& b) const
    {
	// Implementation of square-square collision
    }

    bool operator()(const CircleData& c, const RectangleData& r) const
    {
        // Implementation of circle-rectangle collision
    }

    bool operator()(const RectangleData& r, const CircleData& c) const
    {
        return (*this)(c, r);  // Symmetric
    }

    // more combinations
};

bool overlaps(const Shape& a, const Shape& b)
{
    return std::visit(OverlapHandler{}, a, b);
}

What does all this bring? The most obvious advantage is type safety. With inheritance, type checking happens at runtime with dynamic_cast. With std::variant, it happens at compile time. A forgotten case leads to a compiler error, not a crash at 3 AM in production.

Plus: There’s no nullptr anymore. The value is always one of the defined types. Never null, never undefined. That alone eliminates an entire category of bugs.

Performance is also a factor. Inheritance needs virtual dispatch. The compiler doesn’t know which method gets called, that’s decided at runtime. With std::variant, the compiler knows exactly which function gets called. Direct call instead of indirection. Plus, the data sits on the stack instead of scattered on the heap. More cache-friendly, faster.

The only downside: Adding new types is more expensive. Every handler needs to be adjusted. But for my simulation with three known shapes, that wasn’t a problem.

What I Really Learned

This project taught me more than just C++ features. It changed how I think about software design.

1. “It Works” Is Not the Same as “It’s Good”

My prototype worked. But the difference between working code and good code is the time I invest in refactoring. Not because someone demands it, but because I want to do better.

2. Design Patterns Are Not an Academic Exercise

Patterns are documented solutions to problems others had before me. Why reinvent the wheel?

3. Paradigms Are Tools, Not Religions

OOP or FP? The answer is: Both. My final code uses OOP patterns for structure and FP concepts for type safety. The best solution is often a hybrid.

4. The Compiler Is My Friend

The more I can check at compile time, the less can go wrong at runtime. Type-safe enums instead of strings. std::variant instead of inheritance plus dynamic_cast. Every compiler error is a bug I don’t have to debug.

Conclusion: Why It’s Worth It

What started as a relaxed vacation evening became several weeks of work. Was that too much for a hobby project? Maybe. But it was about getting better. Not perfect. Better.

I learned how std::variant works. I understood when OOP and when FP fits. I experienced that refactoring isn’t a waste of time.

And most importantly: It was fun again.

No deadline. No code review. No stakeholders. Just me, the code, and the question: “How can I make this better?”

This freedom is often missing in daily work. That makes it all the more important to find it in hobby projects.

The complete code is available on GitHub and Codeberg.

Let me be clear upfront: this isn’t about “Rust is better than C.” It’s about recognizing patterns in how we ensure safety. Some languages require discipline and process. Others provide guarantees through their type system. Understanding the difference helps us make better choices about where to invest our effort.

I was working on a linear algebra library in Rust (yes, the 1000th one) when I wrote a test for matrix addition with mismatched dimensions. cargo build compiled fine, but cargo test surprised me: the test never ran. Instead:

error[E0308]: mismatched types
   --> src/matrix.rs:122:29
    |
122 |         let result = mat1 + mat2;
    |                             ^^^^ expected `2`, found `3`

My first thought was: “Okay, I need to add a check in the library…” Then I stopped. Rust had already caught the error. At compile time. I didn’t need to check anything.

This is Rust’s famous safety, I thought. In C or C++, I would need error handling to catch this case. But then I realized: is that really true? In C, definitely. But in C++, maybe not? This made me think about how different languages handle errors, and more importantly, when those errors are caught. The answer reveals fundamental design philosophies.

The Three Philosophies of Error Handling

Before diving into specific languages, let’s establish what we’re actually talking about. Error handling isn’t just about preventing crashes. It’s about where and how we detect problems.

Compile-Time Detection means errors are caught by the compiler before the program runs. Type mismatches, invalid operations, memory safety violations: these never make it into production.

Runtime Detection means errors are detected during execution. Invalid inputs, resource failures, unexpected states: these require explicit handling in code.

No Detection is the most dangerous category. Undefined behavior, silent corruption, race conditions: errors that slip through and cause unpredictable failures.

Each language chooses a different balance. That choice defines not just how we write code, but how we think about safety, performance, and maintainability. Let me show you what this means in practice.

The C Reality Check

Professionally, I write a lot of C for automotive ECUs. The approach in C to error handling is straightforward: it gives you nothing. No exceptions, no type system enforcement, no safety nets. Just return codes and discipline. So how would I solve the same matrix problem there?

typedef struct {
    int* data;
    size_t rows;
    size_t cols;
} Matrix;

int matrix_add(Matrix* result, const Matrix* a, const Matrix* b) {
    // Check 1: Null pointer?
    if (!result || !a || !b) {
        return ERR_NULL_POINTER;
    }
    
    // Check 2: Null data?
    if (!a->data || !b->data) {
        return ERR_INVALID_DATA;
    }
    
    // Check 3: Dimension mismatch?
    if (a->rows != b->rows || a->cols != b->cols) {
        return ERR_DIMENSION_MISMATCH;
    }
    
    // Check 4: Result buffer allocated?
    if (!result->data) {
        return ERR_NO_MEMORY;
    }
    
    // Finally: The actual addition
    for (size_t i = 0; i < a->rows * a->cols; i++) {
        result->data[i] = a->data[i] + b->data[i];
    }
    
    return SUCCESS;
}

Four defensive checks for a simple operation. And this is just the function itself. Every caller must also check:

Matrix mat1, mat2, result;
// ... initialization ...

int err = matrix_add(&result, &mat1, &mat2);
if (err != SUCCESS) {
    // Error handling
    switch(err) {
        case ERR_NULL_POINTER: /* ... */
        case ERR_DIMENSION_MISMATCH: /* ... */
        // ...
    }
}

This is defensive programming in its purest form. I must anticipate every conceivable error and catch it. Every function is half error handling, half logic.

What this means in practice is straightforward but demanding. Every function that can fail returns an error code. Every caller must check that code. Forgetting a check? Undefined behavior. Passing wrong types? Undefined behavior. Race condition? Undefined behavior.

In the field of automotive software, this becomes critical. MISRA C exists precisely because C requires perfect discipline. We compensate with strict coding guidelines, static analysis tools, and extensive code reviews. But these are processes trying to replace what the language doesn’t provide.

The Automotive Reality: AUTOSAR, MISRA, and ISO 26262

Let me be clear about something: I’m simplifying. The automotive software world is more structured than “just use C everywhere.”

We have AUTOSAR Classic for traditional ECUs, built on C with strict architectural patterns. We have AUTOSAR Adaptive for high-performance computing platforms, which allows modern C++. These aren’t just coding standards. They’re complete software architectures that define how components communicate, how resources are managed, and how safety is achieved.

The defensive checks I described aren’t random. They’re systematically required by these architectures. AUTOSAR’s Runtime Environment expects certain error codes. Safety standards like ISO 26262 require certain failure detection mechanisms. MISRA C and MISRA C++ define exactly which language features we can use.

So when I talk about defensive programming versus type-driven design, I’m not suggesting we abandon these standards tomorrow. I’m asking: within these frameworks, where could better language features reduce the burden?

Could AUTOSAR runnables benefit from safer type systems? Could the communication between software components use types instead of runtime validation? Could we use modern C++ features where AUTOSAR Adaptive allows them? Features like std::optional, std::variant, or constexpr validation could reduce defensive checks while staying within the standard, instead of staying in the C-style comfort zone.

The Daily Cost of Defensive Code

This pattern appears everywhere in typical C codebases. Whether I look at open source projects, tutorial code, or professional embedded software, the ratio is similar.

Let me show you a more automotive-specific example. Consider CAN message handling, something every ECU developer knows:

typedef struct {
    uint32_t id;
    uint8_t data[8];
    uint8_t length;
} CANMessage;

int validate_and_send_can_message(const CANMessage* msg) {
    // Check 1: Message exists?
    if (!msg) return ERR_NULL_POINTER;
    
    // Check 2: Valid CAN ID?
    if (msg->id > 0x1FFFFFFF) return ERR_INVALID_ID;
    
    // Check 3: Valid length?
    if (msg->length > 8) return ERR_INVALID_LENGTH;
    
    // Check 4: Standard vs Extended ID consistency?
    if (msg->id > 0x7FF && !(msg->id & 0x80000000)) {
        return ERR_ID_FORMAT_MISMATCH;
    }
    
    // Finally: Send the message
    return can_driver_send(msg);
}

Every CAN stack has similar functions. Most of the code is validation, not communication. And every project implements these checks slightly differently, leading to subtle bugs across teams.

In my experience with typical embedded C codebases, defensive checks and cleanup logic often make up more than half of the code. The actual business logic gets buried under layers of safety validation. And here’s the dangerous part: forget a single check or a cleanup path, and you have a bug. Every code review must verify these paths. Every test must cover them. This is the cost of safety in C.

Now imagine if the type system could enforce this:

struct StandardCANId(u16);  // 11-bit, 0x000-0x7FF
struct ExtendedCANId(u32);  // 29-bit

enum CANId {
    Standard(StandardCANId),
    Extended(ExtendedCANId),
}

struct CANMessage<const N: usize> {
    id: CANId,
    data: [u8; N],  // Length is part of the type
}

impl<const N: usize> CANMessage<N> {
    fn send(self) -> Result<(), CANError> {
        // No validation needed - invalid messages can't be constructed
        can_driver_send(self)
    }
}

// This compiles:
let msg = CANMessage { 
    id: CANId::Standard(StandardCANId(0x123)),
    data: [0u8; 8] 
};

// This doesn't - length is wrong:
let msg = CANMessage { 
    id: CANId::Standard(StandardCANId(0x123)),
    data: [0u8; 9]  // Compiler error!
};

The validation isn’t gone. It’s moved to the constructor of StandardCANId and ExtendedCANId:

impl StandardCANId {
    fn new(id: u16) -> Result<Self, CANError> {
        if id > 0x7FF {
            return Err(CANError::InvalidStandardId);
        }
        Ok(StandardCANId(id))
    }
}

Once you have a valid StandardCANId, the compiler guarantees it stays valid. The type system enforces CAN protocol constraints.

This is conceptual. In practice, you’d still need unsafe code to interface with the CAN hardware driver. But the type safety lives in the application layer, where most bugs actually happen.

Now, I need to be realistic here. Rust in automotive production is still early days. There are no ISO 26262 qualified Rust compilers yet. Tool qualification is expensive and time-consuming. The automotive industry moves slowly, and for good reasons.

Rust also has unsafe blocks where you explicitly opt out of safety guarantees. You still need discipline there. And Rust doesn’t prevent logic errors or deadlocks. It’s not magic.

But here’s what matters: Rust demonstrates that compile-time safety and zero-cost abstractions are possible. Even if we can’t use Rust in production tomorrow, it shows us what modern C++ could be if we used it more fully within AUTOSAR Adaptive contexts.

The Shift in Thinking

This is the moment that hit me. In C, I think: “What can go wrong? What checks do I need?” In Rust, I think: “What do I want to express? What type describes that?”

This is not just less code. It’s a fundamental difference in mental model. With defensive programming in C, I catch errors at runtime. Every developer must think about every edge case. I write a lot of code for safety, and testing proves correctness.

The difference is profound. In defensive programming, safety is a property I must actively maintain through discipline and process. In type-driven design, safety is a property the language guarantees through its type system. With type-driven design in Rust and modern C++, I make certain classes of errors impossible at compile time. The compiler prevents type mismatches, null pointer dereferences, and memory safety violations. Safety comes through design, and types prove these specific properties. They don’t catch logic errors or algorithmic mistakes, but they eliminate entire categories of bugs that plague C codebases.

The Automotive Perspective

In typical automotive ECU code, we have thousands of defensive checks scattered throughout the codebase. Everywhere you look, you see the same patterns. Null pointer checks, bounds checks, range checks, state validation checks. All necessary, all manually written, all manually verified in code reviews.

With MISRA C and MISRA C++, code reviews, and static analysis, we try to ensure that no check is forgotten. This is process as a replacement for language features. We build elaborate systems to catch what the language cannot.

In Rust or modern C++, many defensive checks would take different forms:

Null pointer checks could become Option<T> (explicitly marking that a value might be absent) or references (guaranteeing a value exists). Bounds checks could become compile-time const sizes or explicit .get() calls that return Option. State machines could use the type-state pattern, where each state is a different type and transitions are enforced by the compiler. Value ranges could use newtype patterns, where a type like Speed0to100(u8) can only be constructed with valid values.

These aren’t just different syntax. They shift validation from runtime checks scattered throughout the code to compile-time constraints or explicit construction points.

The Practical Takeaway

My realization is not “Rust is better than C.” It is this: I am spending time writing defensive code that would be unnecessary in modern languages.

This doesn’t mean I should rewrite all our C code tomorrow. But it changes how I think about new components. For the hardware layer, C remains essential for direct register access. There’s no choice there. For the safety layer, I must invest in defensive checks because the language requires it. But for business logic? There, Rust or modern C++ could replace defensive checks with types.

This distinction matters. It means I can choose where to invest effort. Where I must use C, I accept the cost of defensive programming. Where I have flexibility, I can leverage type systems to reduce that cost.

The Elephant in the Room: Why Not Switch Tomorrow?

Let me address what I’m not saying. I’m not suggesting we rewrite our ECU software in Rust tomorrow. That would be naive for several reasons:

First, there’s certification. Our tools are qualified according to ISO 26262. Our processes are qualified according to ASPICE. Introducing a new language means re-qualifying everything. That’s years of work and significant investment.

Second, there’s legacy. We have millions of lines of proven C code. It works. It’s tested. It’s certified. Rewriting it would be expensive and risky.

Third, there’s the team. We have engineers trained in C and AUTOSAR. Learning Rust or modern C++ takes time. Not everyone will embrace it.

Fourth, there’s the ecosystem. Our suppliers deliver AUTOSAR components in C. Our customers expect C interfaces. The entire automotive software supply chain is built around C.

So why am I writing this article? Because I believe we should ask different questions. Not “should we switch to Rust” but “where are we paying unnecessary costs with current tools?”

Are there new components where we could use AUTOSAR Adaptive with modern C++ features? Are there application layers where type safety would prevent entire bug categories? Are there areas where the process overhead of defensive C programming outweighs the migration cost?

These are the questions this compiler error made me ask.

The Question I’m Left With

How much of my daily work is actually just manual type checking? How many bugs have I fixed that wouldn’t have compiled in Rust? How much code review time do we spend verifying that all defensive checks are present?

These are the real questions this compiler error raised. Not “which language is better,” but “how much of my work could be automated?”

For new projects, I will think more about this: Can I model the problem so that erroneous states are excluded at compile time? For legacy code: Can I at least move the defensive checks into clearly bounded layers instead of scattering them everywhere?

Looking back, this small moment with a matrix library taught me something fundamental. The goal is not to eliminate error handling. The goal is to move it to where it’s most effective. Sometimes that’s runtime checks with discipline and process. Sometimes that’s compile-time guarantees with types and compilers.

Knowing the difference and choosing deliberately makes all the difference.

What I’m Taking Away

This compiler error taught me to recognize a pattern in my daily work. When I write defensive checks, I should ask: is this a runtime constraint that could fail (like “file not found”), or is it a type constraint that should never occur (like “wrong matrix dimension”)?

Runtime constraints need runtime checks. That’s fine. But type constraints masquerading as runtime checks? That’s where we’re wasting effort.

In new code, I’m trying to push more constraints into types. Not because it’s fashionable, but because every type constraint I encode is one less thing I need to check, test, review, and debug.

In existing code, I’m thinking more about boundaries. Can I at least isolate defensive checks at system boundaries, rather than scattering them throughout? Can I create a safe inner core where certain classes of errors are impossible by construction?