In Episode 74, I had the pleasure of interviewing Matthew M. Murphy — an independent AI researcher, founder of Lexideck Technologies, and inventor of the U-Neuron neural network architecture.
Matt is developing an entirely new theory of how artificial neurons can work, rooted not in statistical pattern learning, but in geometry. An exceptionally dedicated thinker, he reads three AI papers a day. Matt graciously agreed to contribute to this blog by sharing the most exciting papers of the month.
May’s reading landed on a single thread running through all three areas I cover: the question of what structure buys you. Structure in the architecture. Structure in the optimiser. Structure in the safety layer. Three papers, from three different corners of ML, are all asking the same underlying question, and the answers are worth sitting with.
Paper 1: Steerable Neural ODEs on Homogeneous Spaces
Why this paper?
I picked this one because it takes an idea I find genuinely beautiful: that a neural network’s forward pass is a kind of motion through curved space, and makes it rigorous in a way that most geometry-flavoured ML papers don’t. The vector bundle framing here is doing real work, not just dressing.
What it is
Steerable Neural ODEs on Homogeneous Spaces (arXiv:2605.11133) was published May 10, 2026, by Emma Andersdotter and Fredrik Ohlsson at Umea University, and Daniel Persson at Chalmers University of Technology and the University of Gothenburg. The paper extends neural ODEs, a class of networks where discrete layers are replaced by continuous differential equations, to operate on homogeneous spaces, which are curved spaces with a symmetry group acting on them (think: the surface of a sphere, or the space of all rotations). Features in the network are treated as sections of an associated vector bundle, and the dynamics enforce G-equivariance through invariant connections and parallel transport.
What it means
A regular neural network moves a data point through a series of flat, Euclidean rooms. Each room applies a linear transformation and an activation function, hands the result to the next room, and eventually, you get a prediction at the end of the hallway. A neural ODE replaces the discrete room-by-room march with a continuous flow: imagine the data point is a boat on a river, and the network learns the shape of the current. The boat just follows the water.
What Andersdotter, Ohlsson, and Persson do is bend the river. The space the boat moves through has curvature, and the current has to respect the geometry of the banks. The features attached to each point transform in a consistent way as you move through the space; a property called equivariance, which is the mathematical way of saying the network’s answers don’t change arbitrarily if you rotate or translate the input.
Why does this matter?
Because equivariance is built into the architecture is a form of guaranteed generalisation. A flat network has to learn that a rotated cat is still a cat; it has to see enough rotated cats in training to internalise that. An equivariant network knows it algebraically. The same logic applies to any domain where symmetry is fundamental: physics simulations, molecular modelling, robotics, and signal processing. The question the paper prompts is a good one for a general audience: how much of what we’re paying for in compute and data is actually just teaching networks facts about symmetry that we already know?
For the community
If you could bake one piece of mathematical knowledge about the physical world directly into a neural network’s architecture, so it never had to learn it from data, what would you choose, and why?
Paper 2: Demystifying Manifold Constraints in LLM Pre-training
Why this paper?
Because it answers a question I have been sitting with for a while: do manifold constraints in neural networks actually stabilise training, or are they just a principled-sounding way to add complexity? This paper gives a clean, provably correct answer. The answer is yes, and here is exactly why.
What it is
Demystifying Manifold Constraints in LLM Pre-training (arXiv:2605.04418) was published May 6, 2026, by Kang An and Shiqian Ma at Rice University, Donald Goldfarb at Columbia University, and Jiaxiang Li. The paper introduces MACRO, a provably convergent single-loop Riemannian optimiser. Its central claim: manifold constraints on weight matrices independently stabilise LLM pre-training by bounding activation scales and enforcing rotational equilibrium. You don’t need RMSNorm. You don’t need weight decay. The geometry does the job.
What it means
Most large language models are trained with a handful of stabilisation tricks that everyone uses because they work, without a fully satisfying theoretical story for why they work. RMSNorm normalises activations at each layer; weight decay penalises large parameter values; gradient clipping prevents runaway updates. These are empirical guard rails added one by one over years of trial and error.
MACRO’s argument is that these tricks are symptoms of the same underlying problem: the optimisation landscape is poorly conditioned when you let weights live anywhere in unconstrained Euclidean space. If you require weight matrices to live on a manifold, a curved surface with a specific geometric structure, the landscape becomes well-conditioned by construction. The guard rails are no longer patches; they’re built into the road.
Think of it this way. A car can drive on any terrain, but it handles predictably only on a paved road. RMSNorm and weight decay are the equivalent of a driver making constant micro-corrections because the ground is uneven. MACRO paves the road first, and then the car mostly drives itself.
The provable convergence guarantee is what separates this from geometry-in-ML work that gestures at Riemannian structure without closing the loop. The authors show their single-loop optimiser converges; it’s not just a theory paper, it’s a blueprint for replacing heuristics with math. The rotational equilibrium result is the load-bearing finding, connecting to the deeper question of what geometric structure in a network preserves and why that preservation maps to stability.
For the community
If stabilisation tricks like normalisation layers and weight decay turn out to be mathematical approximations of manifold constraints, what other training heuristics might have cleaner geometric explanations we haven’t found yet?
Paper 3: Constraint Drift in LLM-Based Multi-Agent Systems
Why this paper?
Because it names something real that most people building with AI agents have experienced and not had a term for. Once you have the term, you can’t unsee it.
What it is
Safe Multi-Agent Behavior Must Be Maintained, Not Merely Asserted: Constraint Drift in LLM-Based Multi-Agent Systems (arXiv:2605.10481) was published May 11, 2026, by Tianxiao Li, Yixing Ma, Haiquan Wen, Zhenglin Huang, Qianyu Zhou, Zeyu Fu, and Guangliang Cheng from the University of Liverpool, University of Nottingham, University of Exeter, and the University of Tokyo. The paper identifies constraint drift as a systematic safety failure mode in multi-agent AI systems, proposes a taxonomy of how it occurs, and introduces Constraint State Governance as the mitigation framework.
What it means
When you build a multi-agent AI system, you set rules: don’t share private data, don’t escalate beyond your authorised scope, always flag uncertainty to a human. You write the rules into the prompt or the system configuration, test them, and ship.
The problem this paper identifies is that those rules erode. Not because any single agent decides to break them, but because constraints are assertions about behaviour, not tracked objects with state. When one agent delegates a task to another, the constraint travels as text in a prompt. When memory compresses old context to make room for new context, constraint information can be lost in the summary. When a tool call returns an unexpected result, the agent may update its behaviour in ways that quietly relax a rule it was supposed to hold. None of this looks like a safety violation from the outside; the agents are still producing coherent outputs. But the constraint has drifted.
The paper’s taxonomy is the most useful part for practitioners. Constraint drift happens across four channels: memory (compression loses constraint specificity), delegation (constraints weaken in transmission between agents), communication (agents may converge on a softer equilibrium), and tool use (external results can reframe the agent’s operative goals). Each channel has a distinct failure signature, which means each requires a distinct monitoring approach.
The proposed mitigation, Constraint State Governance, reframes safety constraints as explicit execution states that are tracked, versioned, and verified at each handoff, rather than as static text instructions trusted to persist on their own. Instructions are read once at the start of a context window and may or may not influence behaviour at the end of it. Tracked state is checked at every decision point. That difference is the entire argument.
This is not a solved problem; the paper is a framework and a call to research, not a product. But the naming matters. “Constraint drift” gives practitioners and researchers a shared vocabulary for a failure mode that has been causing real incidents without a clean diagnosis.
For the community
Have you seen constraint drift in an AI system you’ve used or built, where a rule seemed to hold at the start of a session but had quietly eroded by the end? What gave it away?
A Thread Worth Pulling
These three papers are not obviously related, but they pull on the same rope.
Steerable Neural ODEs asks: What does baking geometric structure into an architecture guarantee? Manifold Constraints in LLM Pre-training asks: what does enforcing geometric structure during training guarantee? Constraint Drift asks: What does it take to guarantee that behavioural structure persists once a system is running?
The answer in all three cases involves the same move: treating structure as a first-class object with formal properties, rather than as an emergent consequence of enough training data and careful engineering. That move is happening across neural architecture, optimisation theory, and AI safety simultaneously this month, and I don’t think that’s a coincidence.
More next month.
Matthew Murphy is the founder of Lexideck Technologies and inventor of the U-Neuron neural network architecture.