How Geometry Influences Artificial Intelligence

The Shape of AI Decisions A new lens for understanding AI decision-making Everyone complains AI gets stuck or black box but what if that moment had a measurable signature? Recent work from the University at Albany tracked exactly what happens inside a transformer-based reinforcement learning agent as it plays a navigation game. When the environment is straightforward and the agent has committed to a direction, its internal representation lives in a low-dimensional space - compact and stable. Introduce complexity, crowded screens, overlapping objectives, competing moves and something striking happens: the internal geometry expands. This expansion isn’t random noise. It aligns with specific moments where the agent must evaluate options or resolve ambiguity. Rather than encoding knowledge on smooth, thin manifolds, the system forms stratified geometric layers - clusters of different dimensionality tied to task difficulty. The researchers used a mathematical tool called the Volume Growth Transform to reveal this stratified structure, something that outright challenges old assumptions about smooth, continuous latent spaces in neural networks. In practical terms for enterprise AI: - You don’t just want a score or loss metric - You want a signal that tells you when the model is genuinely challenged. - Measuring shifts in geometric complexity could become a real-time diagnostic That means adaptive training isn’t a guess, it’s data-driven. Intervene where the system’s structure shows stress not just error. If we can measure when an AI’s internal geometry expands, we can pinpoint where it’s truly struggling and adapt training or intervention right when it matters instead of after the fact. This gives us a real diagnostic signal not a guess. #AI

Everyone's arguing about benchmarks and how to do LLM evals. We decided to take it one step further. Benchmarks tell you what a model did. They don't tell you what it can do, what it's about to be able to do, or why it fails when it fails. So we built something different. At the Chocolate Milk Cult, we've been developing tools to probe the actual geometric structure of how models organize knowledge—and what we're finding changes how you should think about model selection, fine-tuning, and capability evaluation entirely. This has insane implications for a lot AI in sensitive fields (we're already using this to pick better judges for our reasoning strategy at Irys, Legal AI (formerly Iqidis). Digging into model geometries, we're able to learn some cool things like-- Deepseek-7b is 4x better at code than sentiment. Not because of training data. Because of geometry. When we measure concept direction strength—how cleanly a model separates "yes" from "no" for a given capability—code scores 361.85. Sentiment scores 85.93. That's not a benchmark result. That's a structural reality baked into the representational space. The model doesn't "try harder" at code. Code lives in a cleaner neighborhood. GPT-2 and RWKV use completely different spatial strategies. But their deep structure is weirdly identical. GPT-2 spreads representations across 20x more dimensions than RWKV. Totally different architectures, totally different geometric fingerprints. But concept entanglement? Nearly identical. Something about how concepts relate to each other might be architecture-invariant. That has massive implications for cross-architecture transfer. Every model has a geometric waist—and we can now tell you exactly where to intervene. Deepseek-7b compresses from 42 effective dimensions down to 3 in its middle layers, then expands back out. Layer 3 is the cleanest steering point. Layer 12 is maximum compression. The final layer spikes in entanglement—everything mixes right before output. This isn't interpretability for interpretability's sake. This is the foundation for understanding why your fine-tune worked or didn't, which model will actually handle your use case, and where to intervene when you need control. We're not the only ones who should have access to this. More coming soon. Research drops and open tooling on the roadmap.

Neural networks might speak English, but they think in shapes. Understanding their rich neural geometry is key to understanding how they work, and to debugging and controlling them with precision. Starting today, our research team is publishing a new series on the hidden shapes inside AI models. Much of interpretability treats a model's concepts as arrows: linear directions in activation space. But that view flattens the richer structure that models actually learn. Neural networks build complex inner worlds with geometry that reflects the structure of reality. Days of the week form a circular loop in language models. The tree of life appears as a complex structure in a genomics model. We found a novel class of Alzheimer's biomarkers as a clean curve in an epigenomic model. This pattern shows up across models, modalities, and domains. Neural geometry lets us both understand models more deeply and also control their behavior more effectively. Steering often fails when we treat concepts as linear, but succeeds when we follow the geometric structures that models actually use. We think understanding neural networks is the most important problem and opportunity in AI, and research like this is a big part of how we get there. Huge credit for the incredible work behind this series to: Atticus Geiger, Ekdeep Singh Lubana, Daniel Wurgaft, Noah Goodman, Can Rager, Thomas Fel, Matthew Kowal, Vasudev Shyam, Sheridan Feucht, Usha Bhalla, Tal Haklay, Eric Bigelow, Raphaël Sarfati, Tom McGrath, Owen Lewis, Jack Merullo, and Michael Byun.

Learning solution operators for systems with complex, varying geometries and parametric physical settings is a central challenge in scientific machine learning. In many-query regimes such as design optimization, control and inverse problems, surrogate modeling must generalize across geometries while allowing flexible evaluation at arbitrary spatial locations. In new collaborative work between Lawrence Livermore National Laboratory, Pacific Northwest National Laboratory and Sandia National Laboratories, with Wenqian Chen , Yucheng Fu, Michael Penwarden, and Pratanu Roy, and funded by the U.S. Department of Energy (DOE), we propose Arbitrary Geometry-encoded Transformer (ArGEnT), a geometry-aware attention-based architecture for operator learning on arbitrary domains. ArGEnT employs transformer attention mechanisms to encode geometric information directly from point-cloud representations. We present three variants—self-attention, cross-attention, and hybrid-attention—that exploit different strategies for incorporating geometric features. By integrating ArGEnT into a Deep Operator Network (DeepONet) as the trunk network, we develop a surrogate modeling framework capable of learning operator mappings that depend on both geometric and non-geometric inputs without the need to explicitly parametrize geometry as a branch network input. Through evaluation on benchmark problems spanning fluid dynamics, solid mechanics and electrochemical systems, we demonstrate significantly improved prediction accuracy and generalization performance compared with the standard DeepONet and other existing geometry-aware surrogates. In particular, the cross-attention transformer variant enables accurate geometry-conditioned predictions with reduced reliance on signed distance functions. By combining flexible geometry encoding with operator-learning capabilities, ArGEnT provides a scalable surrogate modeling framework for optimization, uncertainty quantification, and data-driven modeling of complex physical systems. More details can be found in the arXiv preprint: https://lnkd.in/gcnmuXva #doe #research #appliedmathematics #machinelearning #llnl #pnnl #snl

We thought true intelligence would emerge from scale. It didn’t. So we lowered the definition. Correlation became reasoning. Fluency became thought. Intelligence isn’t statistical. It’s structural. Structure has shape. That’s where Geometric AI begins. For 2,400 years we’ve understood something basic. Geometry isn’t about drawings. It’s about invariants. Plato argued that geometry reveals truths independent of perception. A triangle is not the chalk on the board; it’s the structure that remains true under transformation. Euclid formalized this, not by scaling examples, but by defining constraints. From a small set of axioms, entire worlds followed, not through brute force, but through necessity and proof. Today we talk about AI as if it’s fundamentally about data and scale. But real reasoning has shape. Beliefs form structures. Assumptions create constraints. Causal relations define transformations. Decisions carve trajectories through possibility space. That’s geometry. Not metaphorically but structurally. Today’s connectionist AI optimized correlation. It learned to interpolate surfaces in high-dimensional space. That improved perception. It did not create structural stability. Reasoning requires invariants under change, constraints that must hold, directional causality, localized revision instead of global recomputation, and commitments that survive uncertainty. That is geometric structure. Geometric AI is not about prettier embeddings. It’s about architectures that preserve invariants across transformation. I’m drafting a longer piece on what a geometric architecture for AI would actually look like, one that goes beyond pattern recognition and into structured reasoning under uncertainty. #AI #ArtificialIntelligence

Geometry and AI. What do they have to do with each other? I have built and audited models, both AI and non-AI. But the term ‘geometry’ very rarely appears next to these models. Now, I’ve designed transformer models that are able to learn graph structures that make the most sense for a specific prediction. So I have always known that models can learn some form of structure but this diptych of two papers shows something quite fascinating. One was shared with me by the geometry guru Agus Sudjianto, and the other I serendipitously chanced upon, straight after reading Agus’ paper. 📖 Left Panel: "Deep sequence models tend to memorize geometrically" We usually view model predictions as something that comes from associations. A→B, B→C and so on and so forth. This paper found that even after models learned associations, they still naturally go on to find what the paper calls geometric memory. Instead of A→B, B→C, they want to learn A→C. Or even A→Z. Even when it takes 100x the number of steps to learn this geometric memory. Somehow, geometric patterns emerge from the learning process. It’s like learning a new city. Home → coffee shop one day. Coffee shop → office another. Now you know the way home from the office. 📖 Right Panel: "BLADE: Bivector-Driven Logical Adaptive Decoding" Geometry can help with confused AI too. The paper uses three basic geometric concepts to think about a model's internal state: Scalar: "Is this path compliant?" Vector: "Where is this reasoning heading?" Bivector: "How much tension between competing paths?" When the bivector is high, branch and verify. When it's flat, let the model proceed. Works as a triage method to filter out what’s more important to focus on. I also liked the way the paper applied this to ‘stressed’ states: conjunction; disjunction; exception; nested negation etc. A taxonomy of how logic trips up AI. Same city. Picking between two 7-Elevens a block apart? Just pick one. Don't think too much. Choosing between two alleys that look similar? One is a shortcut, the other leads to a dead end after a long walk. Think twice. And harder. One paper explains the natural emergence of geometry. The other uses geometry for control. I need to go and brush up my geometric math. #AI #AIRiskManagement #Geometry

👏  𝗖𝗼𝗻𝘀𝘁𝗿𝗮𝗶𝗻𝘁𝘀 𝗜𝘀 𝗔𝗹𝗹 𝗬𝗼𝘂 𝗡𝗲𝗲𝗱 - European Researchers Introduce 𝗚𝗲𝗼𝗺𝗲𝘁𝗿𝘆-𝗜𝗻𝗳𝗼𝗿𝗺𝗲𝗱 𝗡𝗲𝘂𝗿𝗮𝗹 𝗡𝗲𝘁𝘄𝗼𝗿𝗸𝘀 (𝗚𝗜𝗡𝗡𝘀) for Constraints Driven Generative Modeling in Industrial Design 📂 Data in good quality and on a large scale is one of the biggest challenges of industrial AI. Traditional design processes often depend on 𝗹𝗮𝗿𝗴𝗲 𝗱𝗮𝘁𝗮𝘀𝗲𝘁𝘀 𝘁𝗼 𝘁𝗿𝗮𝗶𝗻 𝗔𝗜 𝗺𝗼𝗱𝗲𝗹𝘀, which can be a significant limitation in fields like 3D computer graphics and engineering where such data is scarce. This scarcity hampers the application of advanced supervised learning techniques, necessitating alternative strategies to meet the industry's evolving demands. 📑👏 The exciting question of this research is: 𝐼𝑠 𝑖𝑡 𝑝𝑜𝑠𝑠𝑖𝑏𝑙𝑒 𝑡𝑜 𝑡𝑟𝑎𝑖𝑛 𝑎 𝑠ℎ𝑎𝑝𝑒-𝑔𝑒𝑛𝑒𝑟𝑎𝑡𝑖𝑣𝑒 𝑚𝑜𝑑𝑒𝑙 𝑜𝑛 𝑜𝑏𝑗𝑒𝑐𝑡𝑖𝑣𝑒𝑠 𝑎𝑛𝑑 𝑐𝑜𝑛𝑠𝑡𝑟𝑎𝑖𝑛𝑡𝑠 𝑎𝑙𝑜𝑛𝑒, 𝑤𝑖𝑡ℎ𝑜𝑢𝑡 𝑟𝑒𝑙𝑦𝑖𝑛𝑔 𝑜𝑛 𝑎𝑛𝑦 𝑑𝑎𝑡𝑎? European researchers at the 𝗟𝗜𝗧 𝗔𝗜 𝗟𝗮𝗯, 𝗝𝗞𝗨 𝗟𝗶𝗻𝘇 (Austria); SINTEF (Norway); and NXAI GmbH (Austria) have presented a major advancement in industrial design and engineering. Their new approach, Geometry-Informed Neural Networks (GINNs), is a framework for training shape-generative neural fields 𝘄𝗶𝘁𝗵𝗼𝘂𝘁 𝗱𝗮𝘁𝗮 by leveraging user-specified design requirements in the form of objectives and constraints Key Highlights of the Research: 🗂️ 𝗗𝗮𝘁𝗮 𝗜𝗻𝗱𝗲𝗽𝗲𝗻𝗱𝗲𝗻𝗰𝗲: GINNs eliminate the need for extensive training data, making them ideal for fields like 3D design and engineering, where relevant datasets are often scarce. 🔀 𝗖𝗼𝗻𝘁𝗿𝗼𝗹𝗹𝗲𝗱 𝗗𝗶𝘃𝗲𝗿𝘀𝗶𝘁𝘆 𝗶𝗻 𝗢𝘂𝘁𝗽𝘂𝘁𝘀: Built-in constraints enable GINNs to generate multiple viable design options, addressing the issue of limited design diversity often seen in traditional AI models. 📏𝗘𝗻𝗵𝗮𝗻𝗰𝗲𝗱 𝗗𝗲𝘀𝗶𝗴𝗻 𝗣𝗿𝗲𝗰𝗶𝘀𝗶𝗼𝗻: Engineers can adjust geometric properties—such as surface smoothness or the number of holes—to meet specific requirements, expanding flexibility in manufacturing, aerospace, and architecture. Geometry-Informed Neural Networks could offer a transformative approach to industrial design, enabling the generation of complex shapes without the need for extensive datasets. Congrats to the research teams and let's wait for the soon integration of GINNs into the existing design workflows to explore their applicability in real-world scenarios and expand their capabilities to address a broader range of design constraints and goals. Sepp Hochreiter Robert Weber Daniel Spiess Dr. Moritz Grawunder Erik Scepanski Liam Friel #IndustrialDesign #Engineering #AI #EuropeanResearch #GenerativeDesign 👉 Follow for more insights on Industrial #AI applications and the future of engineering innovation.

AI is hungry for geometry. Reaching its potential in engineering will require enormous amounts of it (along with physics and engineering intent) -- I just wrote up my thoughts on this here: https://lnkd.in/eQudpwsS & After spending the past year with customers applying nTop advanced design applications -- watching them generate hundreds of thousands (over a million in some cases) of design variants at a scale and speed that wasn't possible before -- I'm convinced that nTop is the geometric infrastructure for AI. There's a pattern I'm seeing across aircraft design and propulsion systems: they are both trying to essentially find a needle in a haystack -- the best design that meets a given set of difficult requirements. What's been so interesting for me to see is how they've applied it. We've seen these datasets flow into simulation pipelines, into PhysicsAI training, into optimization loops across entire engineering stacks. These customers all have different programs and different toolchains, but they have the same underlying geometry infrastructure powering everything. And that matters because modern simulation, optimization, and PhysicsAI have all gotten faster, more composable, and more capable of running at scale. But they're starved, and it's not for food, it's for geometry. Without geometric datasets, there are no physics datasets. You can't train on performance data you were never able to generate. The same reason human engineers struggle to explore the design space with B-rep CAD is the same reason AI does. Models break. Variants require manual intervention. Meshing fails. Engineering organizations are under enormous pressure to deliver faster, and AI is the new computer. They're investing heavily to figure out how. But without the geometry infrastructure to generate, operate, leverage AI, they'll keep failing. That's the bottleneck nTop breaks. Link: https://lnkd.in/eQudpwsS Marc Jacobs, our CTO, is publishing more around what we're building, keep an eye out for that.