The unreasonable effectiveness of mathematics: from Wigner to Karpathy

The phrase "the unreasonable effectiveness of mathematics" originates with Eugene Wigner, who, in his seminal 1960 paper, reflected on the astonishing, almost mysterious capacity for mathematics—a creation of the human mind—to so aptly describe the laws of nature ([1]). Wigner contended that the correspondence between mathematics and physical reality goes far beyond mere coincidence, and poses a philosophical puzzle about why abstract mathematical concepts, often developed long before their application is found, become foundational to physics and other sciences. This enigmatic resonance continues to fascinate philosophers and scientists alike ([3], [4]).

Decades afterward, Andrej Karpathy invoked Wigner’s phrase with a twist for the computational age. In his essay "The Unreasonable Effectiveness of Recurrent Neural Networks", Karpathy marveled at how relatively simple mathematical structures—specifically RNNs—can uncover and generate coherent, creative, and contextually rich sequences out of seemingly chaotic texts ([2]). Karpathy’s observation is practical: within artificial intelligence, mathematical models like neural networks develop the capacity to recognize patterns and structure previously only attainable by human intelligence.

While Karpathy’s tribute signals continuity with Wigner’s insight, a key distinction must be drawn. Wigner's investigation is metaphysical and universal, interrogating the very foundation of how mathematics relates to reality ([4], [5]). Karpathy’s wonder, though profound in its own context, is embedded in engineering and the practical demonstration of how mathematical constructs can achieve, via learning, a simulation of natural and formal language.

Nevertheless, both perspectives fuel the debate on why mathematics "works" so well. Wigner fuels the philosophical contemplation about the existence and indispensability of mathematical entities ([5]). Hamming, for example, revisits Wigner’s question, suggesting that both the formulation of mathematical problems and the development of their solutions are heavily guided by the structure of physical reality and our cognitive predispositions ([3]). Tegmark goes further, even arguing for a "mathematical universe" where physical existence is ultimately mathematical structure ([4]).

Still, there are counterpoints—some philosophers, such as Mortensen, raise questions around the consistency and limitations of mathematics in describing the empirical world, highlighting cases in which mathematics itself can be inconsistent ([6]). Others like Colyvan probe the indispensability arguments in the philosophy of mathematics, suggesting that the effectiveness of mathematics is intimately tied to the pragmatic and sometimes messy nature of scientific inquiry ([5]).

In summary, Wigner’s and Karpathy’s observations, though different in scale and scope, converge on the awe-inspiring and often mysterious alignment between mathematical abstraction and the structure of reality—whether atomic particles or artificial neural patterns. Their shared sense of wonder continues to push the boundaries of both theoretical curiosity and practical achievement.