Tag Archives: philosophy

How Machines Learn to Write

From Shannon’s prediction game to modern transformers—and what gets lost in meaning

The Mysteries of Words

A few years ago, I sat in a dissertation defense for a student working on language models, and something quietly unsettled me. He was explaining “attention mechanisms,” showing how his system could summarize documents in perfect English. I remember watching the examples scroll by, line after line, unlike earlier machine-generated texts, carrying no seams.

After the defense, there was a small celebration. Over coffee and cake, I asked colleagues where this whole idea had begun. Someone mentioned ELIZA: the first chatbot that simulated conversation. ELIZA’s famous DOCTOR script mimicked a psychotherapist, reflecting users’ words back at them and so creating the illusion of understanding.

It didn’t have a model of meaning; it was based on pattern-matching and substitution. And yet many people responded as if it did. That strange human tendency to attribute feeling to a simple program was later called the ELIZA effect. Curious, I traced the thread further back and realized that ELIZA stood on a foundation laid much earlier by Alan Turing. Turing had in fact written two remarkable papers just after World War II.

In 1948, in Intelligent Machinery, he sketched what he called an “unorganized machine,” a crude model of the infant brain. Its wiring was initially random, but through reward, punishment, and random exploration, it could be “educated” into a universal computer. To prove its worth, he proposed a set of benchmark tasks: games like chess and cryptography as easier challenges, mathematics and translation as harder ones, and at the very top, the learning of natural language. Already, he sensed that teaching a machine language would be the ultimate test.

Two years later, in 1950, in Computing Machinery and Intelligence, he posed his famous provocation: not “Can machines think?” but whether they could succeed in the imitation game, what we now call the Turing Test. Once again, language was the centerpiece. He imagined training a machine as one might teach a child, through interaction, naming, and correction.

In hindsight, ELIZA was almost a toy realization of Turing’s challenge: a machine producing conversation that, for some, passed as human. It was a small step, but a definitive one on the march toward what we now call artificial intelligence. That was the lineage: from Turing, through ELIZA, to the models I had just seen.

The resemblance was striking, but also disorienting. It blurred the boundary between words that merely followed rules and words that carried meaning. The harder task was never finding the words, but shaping the essence they had to carry. Unlike the demo I saw, which simply strung together the most probable words, I faced a different kind of struggle: I knew the words, but the felt essence wasn’t there yet. That quiet, internal wrestling, the pause and reshaping before meaning lands, was absent in the polished texts before me.

This is not merely a technical matter of algorithms. Language is bound up with social practice, with the rituals and repositories by which communities make sense of the world. A sentence does more than map signs to ideas; it declares, commands, consoles, jokes, curses, remembers, and binds people together in common projects. Across cultures and centuries, in oral performance and written text, language accumulates authority and history: it carries law, theology, love letters, treaties, protests, prayers, jokes, and recipes.

I grew up with the Ramayana and the Mahabharata, not as entertainment but as living traditions: vast conversations full of moral dilemmas, strategic debates, and philosophical inquiry. The Bhagavad Gita, nested inside the Mahabharata, is itself a dialogue about duty and liberation that still guides millions. Later, in English, I discovered Shakespeare’s soliloquies and the playful wit of Pygmalion. Across these forms, language is never merely syntax or sequence; it is a vessel through which thought, memory, and moral imagination endure.

But language is not the only such vessel. Music can convey emotion more directly than a sentence. A painting can express what words cannot. Athletes speak through rhythm and gesture; mathematicians through symbols and proof. These are all languages in a broader sense: systems of signs that carry meaning.

What sets natural language apart is its paradoxical combination of simplicity, fragility, and reach. Compared to music, it is a sparse medium. Just strings of symbols. And that simplicity makes it radically fragile: a comma, a tense, or a single word can tilt the whole meaning. Yet because it can describe, question, command, or console, its coverage is unmatched. Perhaps this is why Turing placed language at the pinnacle of machine intelligence: to master something so lean, so delicate, and so general would be to master almost anything.

That ambition, once only theoretical, suddenly felt realized when ChatGPT arrived. What had once been a curiosity in labs and conference rooms suddenly erupted into daily life. A seminar-room marvel became a household word. Classrooms, newsrooms, law firms, and living rooms all found themselves confronted by a machine that could speak in polished paragraphs. Students turned in assignments written by it, professionals drafted memos with it, and friends sent poems and prayers through it. Headlines proclaimed that human thought itself had been hacked.

The effect was electric, unsettling, and irresistible: a machine producing not just a clever reply but a torrent of fluent language on command: text that could mimic styles, explain theories, or console in grief. Where earlier chatbots exposed their seams within a few exchanges, this one could sustain entire conversations, sounding by turns confident, witty, or wise. For many, it felt like crossing a threshold: as if the capture of language, the vessel of our thought, memory, and imagination, had also led to the capture of thought itself.

But had it?

Sign up here (it’s free) to read the full essay: https://nisheethvishnoi.substack.com/

What is Intelligence? From Reflection to Suffering

When intelligence turns inward, it creates the structure in which suffering can arise

When Light Meets Mind

In 1930, Berlin, two men sat across from each other. I’ve read the transcript more times than I can count. Not because it resolved anything, but because it kept echoing through my life.

You had Albert Einstein: the rationalist, the physicist. The man who redefined time, and connected matter and energy, gravity and spacetime. He saw the universe as vast, governed by elegant equations: something to be uncovered, mapped, and trusted.

And then Rabindranath Tagore: the poet, the mystic, the one who saw the inner world. But more than that, he believed that consciousness was not an afterthought in the cosmos. It was central. He wrote songs that became anthems, plays that folded myth into philosophy. His essays held reason and reverence in the same hand.

Where Einstein searched for equations and invariants, Tagore searched for meaning. He accepted science; he insisted that without awareness, it was incomplete. That the world, however beautiful, is only illuminated through the light of mind.

They spoke quietly, these two Nobel laureates, gently trying to name what is real.

Einstein put it plainly: “I believe in the external world, independent of the perceiving subject.”

Tagore’s response was just as clear: “The world is a human world; its reality is relative to our consciousness.”

On the surface, it sounded like a classic philosophical disagreement. But it felt like something more. As if two orientations, objective structure and lived experience, had paused long enough to listen, without declaring a winner.

I used to read that conversation purely as a question of truth: Is there a world independent of us, or is the world, at its core, ours?

But over time, it stopped feeling abstract. It began to weigh on me, in quiet ways I couldn’t always name.

There were losses, some that shook me more than I expected. And then the birth of my children cracked open a terrain I had not known was missing. Both experiences, in their own way, made me question how I had come to know the world, and what kind of knowledge mattered.

The structure I had spent years building, through science and mathematics, was solid. It gave me clarity. It gave me recognition. But it stopped short of certain truths I could now feel pressing in from the edges. Truths not about the world, but about the self that was trying to understand it.

And I began to notice something unsettling: the same intelligence that helped me understand the world could also make me feel lost within it. Not in the usual scientific way, where each answer opens new questions. That rhythm was familiar, even comforting. This was something different. A quieter unease. As if my way of knowing: analytical, recursive, and precise, had become part of the very trap.

I began to see a pattern: the mind’s ability to turn inward, to reflect on itself, could both elevate and entangle it. That reflection could open not just insight, but ache.

Tagore’s words, “The world is a human world”, started to echo differently. Not just as a metaphysical claim, but as a lived one. As a statement about what happens when consciousness turns in on itself. When intelligence doesn’t just observe the world, but begins to simulate its role within it. To model. To track. To reflect. And so I found myself asking: If intelligence brings light to the world, what happens when that light bends inward? What does it illuminate? And what does it burn?

This essay follows that question: tracing how recursive self-modeling, across biology, culture, and computation, opens the door not only to creativity and empathy, but to suffering. Along the way, I turn to contemplative traditions and computational framing to propose a deeper account: that suffering emerges when valuation becomes identity, when the mind tries to optimize a moving target that it has mistaken for itself.

Read the full essay by subscribing (for free) to The Intelligence Loop.

What is Intelligence? Architecture, Divergence, and Fiction

A computational anatomy of intelligence. How faculties interact, architectures diverge, and coherence emerges through self-constructed fictions

“There is no single path into the forest.” — Yoruba proverb

Yo-Yo Ma and the Single Note

It was the winter of 2018 and the NeurIPS conference—one of the world’s premier gatherings on artificial intelligence—had descended on a snow-laced Montreal. Thousands of researchers, engineers, and students crisscrossed the vast convention center, sharing ideas about optimization tricks, new models, and the future of AI. Posters lined the walls of rooms steeped in the aroma of coffee, while outside, the city lay wrapped in cold, crisp silence.

At one of the marquee panels, a senior executive from a major tech company presented their latest AI music generator—an advanced system trained on thousands of classical works, capable of composing coherent classical music in real time.

The melodies were elegant and the timing precise.

Then Yo-Yo Ma was invited to respond.

He didn’t speak. He turned his chair, lifted his cello, and played a single note. Then he played it again. And again. Each time, the same note emerged differently—tentative, bold, grieving, serene. Each time, his breath shifted and his eyes drifted into a different world.

The AI had captured form. But Yo-Yo Ma, infusing his music with intention and feeling, captured the room.

That moment didn’t just expose AI’s limitations. It revealed a deeper truth:

Intelligence isn’t precision—it’s relation.

It does not reside in outputs alone, but in how systems tune themselves to the world: shaped by context, memory, attention, and intent.

It is a dynamic interplay between perception and action, between internal models and external pressures. It arises wherever systems engage their constraints creatively: whether through mycelial networks, migrating birds, musical phrases, or planetary motion.

In the previous essay, we traced how intelligence emerges in nature: not as a fixed trait, but as a layered process—optimization in physics, adaptation in evolution, collective sensing in life before neurons.

This second essay turns inward—from emergence to architecture. If the first asked where intelligence comes from, this one asks: what is it made of?

We begin by identifying a set of core faculties: sensing, responding, memory, learning, attention, valuation, modeling, and reflection.

These faculties take many forms. Sensing may be chemical, tactile, social, or symbolic. Memory may be episodic, spatial, or associative. Valuation may be shaped by prediction error, pain, or narrative.

And how they are configured—what is emphasized, suppressed, amplified, or ignored—depends not just on design, but on history: evolutionary, developmental, experiential.

From these components and their interrelations, intelligence emerges—not as a single thread, but as a weave: recursive, plural, and at times, fictional.

This part of the essay unfolds in three movements:

  • Composition: How core faculties combine to produce reasoning, language, and creativity—not through accumulation, but through tension, feedback, and reprogramming.
  • Divergence: Why there is no single blueprint for intelligence. We examine human cognitive diversity to understand the space of architectural variation.
  • Fiction: How intelligent systems—especially human ones—construct internal narratives to manage complexity, maintain coherence, and navigate meaning.

This is not a final theory. It is a trace—a computational lens on intelligence as it curves inward, reshapes itself, and constructs meaning under pressure. For those exploring AI not as an isolated artifact, but as part of a broader landscape of intelligence, this lens may offer new ways to rethink design and augmentation.

And like a forest, this inquiry offers no fixed path—only branching terrain shaped by tension, memory, and choice.

Read the full essay by subscribing (for free) to The Intelligence Loop.

What is Intelligence? Layers of Emergence

How Intelligence Arises from Nature, One Layer at a Time

The Trouble with Definitions

“The Tao that can be named is not the eternal Tao.” — Lao Tzu

As a mathematician, I’ve long sought clean definitions. Much of my work involves building precise frameworks — starting by defining key concepts, isolating the core of a problem, formalizing it, and tracing its implications to their logical end.

Yet over time, I’ve come to see not just the limits of definitions, but their quiet distortions—the way they can flatten nuance in the name of clarity. The richness of a living idea gets traded for the sterile comfort of formal neatness. Sometimes, defining isn’t just clarifying — it’s an act of power: shaping perception, and granting authority to the one who defines.

Few ideas reveal this tension more vividly than intelligence. We talk about it as if we know what it is — a score, a skill, a spark. But what is it, really? And can something so dynamic ever be pinned down?

I think of intelligence not as a fixed trait, but as an experience — not unlike beauty — arising in context, felt through interaction.

So while we try to define intelligence — because we must — to witness it, to live with it, or to build systems that move with it, we need something else: humility. An attention to context. A willingness to recognize that intelligence, like beauty, is often messy, partial, plural, heuristic, and still astonishingly effective.

But even our capacity to see intelligence is shaped by history. In The Myth of Superintelligence, I argued that our attempts to define intelligence are never neutral. They reflect what we choose to measure, optimize, and reward. This essay is not a repetition of that critique. It is a step back. A shift in lens. It asks not what intelligence is, but when and how it arises—not as a trait, but as something unfolding across time, scale, and structure.

Because the power to define has always been the power to exclude. Colonial systems didn’t just extract labor and land—they imposed ways of seeing. In doing so, they dismissed the intelligence embedded in other ways of knowing, reframing rich knowledge traditions as myth or superstition. These distortions still echo in how we define and measure intelligence today. African polyrhythms were labeled primitive. Classical Indian music was exoticized or ignored. Indigenous knowledge systems—deeply attuned to land, season, and cycle—were reduced to folklore. Intelligence was there. But the lens refused to see it.

This is why any inquiry into intelligence must also be an inquiry into perspective. Definitions don’t just clarify. They constrain. They shape not only what we see, but what we believe intelligence can be.

This series is an attempt to widen the lens—to trace intelligence not as a fixed trait, but as a dynamic unfolding across layers of complexity. We begin with the silent elegance of physical systems, where matter flows under law, solving problems through coherence and constraint. From there, we enter the domain of evolution, where life adapts through variation and feedback, accumulating structure over time. We then move to the responsive intelligence of behavior—organisms without minds that nonetheless solve, coordinate, and learn through interaction.

But these are just the foundations. In the second half, we abstract upward: tracing how intelligence evolves the ability to frame problems, to reflect on and revise its own rules, and finally, to orient itself—to choose what matters. This is where intelligence becomes recursive, contextual, and ultimately, meaningful. Not just a solver of problems, but a seeker of value.

Read the full essay by subscribing (for free) to The Intelligence Loop.

The Myth of Superintelligence

Why AI Won’t Transcend Us—But the Race to Superintelligence Might Redefine Us

At the dawn of the nuclear age, a handful of scientists raced to split the atom. Behind closed doors, they unlocked forces of unimaginable power—capable of reshaping geopolitics, ending wars, or ending the world. The stakes were enormous. The oversight was minimal.

As the mushroom cloud rose over the New Mexico desert, Oppenheimer recalled the Bhagavad Gita:

“Now I am become Death, the destroyer of worlds.”

It was not just a scientific breakthrough—it was a civilizational rupture, and a moment of spiritual reckoning.

Today, we stand at a similar threshold—but this time, the weapon isn’t atomic, it’s epistemic: the power to define, displace, and dictate what counts as intelligence.

A handful of billionaires now race to transcend the very concept of mind.

This is the race to superintelligence—not just a technological contest, but a geopolitical gamble disguised as an AI boom. It unfolds in boardrooms and GPU clusters, driven by speculation, ambition, and fear.

The headlines scream the urgency: Meta reportedly offered $32 billion for Safe Superintelligence, a small startup co-founded by Ilya Sutskever. Sam Altman claimed rivals are dangling $100 million signing bonuses to lure away OpenAI talent working on superintelligence. And Elon Musk, for instance, has predicted that superintelligence will arrive within six months.

This isn’t science fiction. It’s a live experiment on humanity, with no brakes or off switch.

And these aren’t novelists. They’re the very people shaping global AI policy, capital flows, and public belief. Their words fuel markets, realign talent, and reframe speculation as inevitability.

The story being told is simple: AI will soon surpass us—reason better, learn faster, and predict more precisely. It will understand us, outgrow us, perhaps even save us.

And to be fair, the AI race has already delivered extraordinary breakthroughs. We now have AI systems that can predict protein structures, accelerate vaccine development, improve weather forecasting, and translate languages in real time. They are expanding access to healthcare diagnostics, supporting education in underserved regions, and helping marginalized communities organize and advocate. In the right hands, it’s not just advancing knowledge—it’s redistributing it.

But what if the real story is something stranger? What if these machines aren’t transcending us—but are reflecting our biases, and in doing so, trapping us within a narrative that is narrow, selective, even grotesque?

Just this week, headlines claimed AI is close to solving the Navier–Stokes problem—one of mathematics’ greatest challenges. In truth, it was mathematicians guiding DeepMind—not AI solving math, but humans exploring with new tools. Still, the myth headlines: “AI Solves”.

This is the pattern. AI can accelerate exploration—but it does not choose the problem, define what counts as a solution, or frame the space in which solutions are sought. Those decisions—what matters, what’s possible, what’s meaningful—still come from human minds.

Yet the headlines collapse that distinction. They turn collaborative amplification into autonomous achievement. And in doing so, they reinforce the myth.

The myth of superintelligence—the belief that machines will soon outthink us across all domains—has become the defining narrative of the AI era. It drives billion-dollar valuations, existential headlines, and a mood that swings between prophecy and panic.

At its core is a single premise: that intelligence is measurable, stackable, and conquerable. That with enough data and compute, it will emerge—bigger, faster, better.

But intelligence cannot be reduced to a number. It is not prediction, speed, or performance. Real intelligence—whether in a brain, a slime mold, a flock of starlings, or a cello note—does not arise from accumulation alone. It comes from attunement: the capacity to notice, to reframe, to care.

This series traces the roots of the superintelligence myth—what it is, where it came from, what it obscures, and what its pursuit may cost us. It does not ask whether AI will become superintelligent, but what that belief reveals: a confusion about the nature of intelligence, and a recurring urge to centralize, rank, and control it.

This first essay unpacks the myth itself—its origins, its logic, and its consequences. The next installment begins the recovery: What is intelligence—beyond metrics, benchmarks, and brainpower? What distinguishes it from mere intellect? And why does that distinction matter now more than ever?

Read the full essay here, along with others, by subscribing (free) to The Intelligence Loop.

What Counts as Discovery? Rethinking AI’s Place in Science

नेति नेति
Not this, not this.
— Bṛhadāraṇyaka Upaniṣad

The Frame Before the Frame: A Prehistory of Discovery

Long before there were “scientists,” there was science. Across every continent, humans developed knowledge systems grounded in experience, abstraction, and prediction—driven not merely by curiosity, but by a desire to transform patterns into principles, and observation into discovery. Farmers tracked solstices, sailors read stars, artisans perfected metallurgy, and physicians documented plant remedies. They built calendars, mapped cycles, and tested interventions—turning empirical insight into reliable knowledge.

From the oral sciences of Africa, which encoded botanical, medical, and ecological knowledge across generations, to the astronomical observatories of Mesoamerica, where priests tracked solstices, eclipses, and planetary motion with remarkable accuracy, early human civilizations sought more than survival. In Babylon, scribes logged celestial movements and built predictive models; in India, the architects of Vedic altars designed ritual structures whose proportions mirrored cosmic rhythms, embedding arithmetic and geometry into sacred form. Across these diverse cultures, discovery was not a separate enterprise—it was entwined with ritual, survival, and meaning. Yet the tools were recognizably scientific: systematic observation, abstraction, and the search for hidden order.

This was science before the name. And it reminds us that discovery has never belonged to any one civilization or era. Discovery is not intelligence itself, but one of its sharpest expressions—an act that turns perception into principle through a conceptual leap. While intelligence is broader and encompasses adaptation, inference, and learning in various forms (biological, cultural, and even mechanical), discovery marks those moments when something new is framed, not just found. [A future essay will take up this broader view of intelligence—and how discovery both draws from it and transcends it.]

Life forms learn, adapt, and even innovate. But it is humans who turned observation into explanation, explanation into abstraction, and abstraction into method. The rise of formal science brought mathematical structure and experiment, but it did not invent the impulse to understand—it gave it form, language, and reach.

And today, we stand at the edge of something unfamiliar: the possibility of lifeless discoveries. Artificial Intelligence machines, built without awareness or curiosity, are beginning to surface patterns and propose explanations, sometimes without our full understanding. If science has long been a dialogue between the world and living minds, we are now entering a strange new phase: abstraction without awareness, discovery without a discoverer.

AI systems now assist in everything from understanding black holes to predicting protein folds and even symbolic equation discovery. They parse vast datasets, detect regularities, and generate increasingly sophisticated outputs. Some claim they’re not just accelerating research, but beginning to reshape science itself—perhaps even to discover.

But what truly counts as a scientific discovery?

This essay examines that question. Building on my earlier essay, Can AI Know Infinity?, I argue that today’s AI excels at recognizing structure, but not at reframing it. It doesn’t invent abstractions, ask better questions, or propose new ways of seeing. And that distinction—between fitting the world and reimagining it—is what separates tools of discovery from discovery itself.

Read the full essay—along with others on AI and the future of knowledge and institutions—by subscribing to The Intelligence Loop.
It’s a newsletter exploring how algorithms are reshaping judgment, reasoning, and discovery itself.
All posts are free. No paywall. No ads. Just ideas.

The Harari Fallacy: AI Isn’t Alien—It’s a Meta-Institution

In Nexus, Yuval Noah Harari frames AI as a historical rupture—a leap from human-directed information systems to what he calls an “alien intelligence” that is “capable of making decisions and generating ideas by itself.” He describes this shift as “a completely new kind of information network… controlled by the decisions and goals of an alien intelligence.” In doing so, Harari casts AI as an autonomous agent—one with independent goals and the power to shape society. He warns that such systems may sever humans from the center of epistemic and political power.

This framing—what we might call the Harari Fallacy—is compelling, but in a crucial way, misleading. Harari rightly captures the scale of transformation. But by casting AI as external, he obscures its recursive and embedded influence. AI is not a foreign agent; it is a system born from within our institutions. A shift more intimate, and arguably more dangerous.

AI isn’t just part of the institutional fabric. It is becoming the logic that designs the fabric itself.

AI today is not merely a tool. Not merely an agent. Nor just an institution in the bureaucratic sense. It has become a meta-institution: a designer and controller of other agents—human and organizational. It configures evaluators, retrains bureaucrats, rewrites rules, and subtly reshapes how judgment is encoded across domains. It doesn’t just automate decisions. It restructures who gets to decide and how.

This essay builds on Harari’s provocation but reframes AI not as an independent actor, but as a meta-institution embedded in our institutional fabric. In doing so, AI is not just automating decisions—it is reshaping the sociotechnical regime: the interlocking system of technologies, norms, routines, and power relations that govern institutional behavior.

This argument draws from a broader intellectual tradition—Bruno Latour, James C. Scott, and Shoshana Zuboff among them—who show how power often hides in process, not in persons. While this essay focuses on how those dynamics play out in practice, a future post will unpack how their theories help illuminate the architecture of AI’s influence.

We will follow a fictional example—InnovaCorp, a mid-sized tech firm adopting an AI hiring platform—to illustrate how these dynamics unfold.

From Alien Intelligence to Recursive Infrastructure

AI is not just an agent. It’s the author of agents’ scripts.

Harari characterizes AI as an autonomous decision-maker—an agent. But agents operate in environments. What AI increasingly does is create the environment that other agents operate in.

At InnovaCorp, HR initially adopts AI to streamline hiring. It assigns candidate scores based on historical “success” data. But soon, hiring managers began to rewrite job descriptions to better attract high-scoring candidates. Over time, the algorithm isn’t just filtering candidates—it’s shaping the definition of what the company thinks a good candidate is.

This is recursive infrastructure: institutions reacting to outputs that they helped produce, now restructured by machine logic.

Redesign, Not Just Drift

AI does not just decide outcomes. It decides what counts as a decision.

At InnovaCorp, optimization gradually becomes governance. Hiring managers begin selecting candidates not on qualitative impressions, but based on AI-generated scores. The metric becomes the mission.

This shift changes not only who is hired but also how jobs are defined, how performance is evaluated, and how internal accountability is structured. Candidates start optimizing their resumes for the algorithm—stuffing them with the right keywords, formatting them for machine parsing, shaping their experience to match historical patterns. The AI system, designed to reflect institutional values, begins to redefine them.

This is not a case of automation drifting into misuse. It is an active reprogramming of institutional logic. The AI, while seemingly neutral, becomes the architect of institutional priorities.

And this is not confined to fiction. At Amazon, an AI recruiting tool learned to penalize resumes containing the word “women’s,” effectively institutionalizing gender bias based on past data. At HireVue, video-based assessments claimed to score “communication” and “learning ability” by analyzing facial expressions and voice tone—until public scrutiny forced the company to remove facial analysis from the system. And in the class-action lawsuit Mobley v. Workday, plaintiffs allege that Workday’s AI system systematically discriminated against older candidates, potentially affecting millions. These examples show how AI tools can silently reshape what institutions prioritize, until challenged by public or legal intervention.

Governance Failure at the Meta-Level

The danger isn’t that AI becomes intelligent. It’s that institutions stop needing to be.

Consider a candidate rejected by the system. She had a strong but unconventional background—community organizing, international work, and a nonlinear career path. But she lacked the calibrated indicators the model preferred. When she inquires, no one at InnovaCorp can explain the decision. It is buried in a stack of features, weights, and patterns. There is no appeal. No clarity. No deliberation.

This is a governance failure not just of an algorithm, but of the institution that outsourced its judgment. Where once institutions justified decisions, they now defer to scores. The ritual of reasoned explanation—central to legitimacy in public and private decision-making—disappears. What remains is an output.

This transformation isn’t hypothetical—it’s already under regulatory scrutiny.

What Is To Be Done: Designing for Meta-Governance

To govern AI, we must govern how AI governs us.

We can no longer think of algorithmic oversight as a matter of model accuracy alone. We must consider how institutions structure themselves around AI—and what it would take to ensure that those structures remain legible, contestable, and aligned with human values.

This means building systems that support contestation: not just accepting or rejecting decisions, but interrogating them. Imagine if every automated hiring decision were accompanied by a counterfactual: “Your score was 85. Had you listed ‘Project Leadership’ under ‘Cross-Functional Initiatives,’ your score would have been 95.” This reframes opacity as a design choice, not a necessity.

It also means preserving legibility: institutions should disclose what their AI systems are optimizing for, what data they are trained on, and what known biases they carry. A hiring algorithm might come with a visible tag: “Optimized for: Speed-to-Hire. Trained on: 2018–2022 employee data. Known exclusions: nontraditional career paths.” That alone would shift how we perceive and engage with algorithmic outputs.

And finally, we must reinforce institutional agency: decisions made by AI should be overrideable by humans with structured justification. This keeps humans in the loop, not as mere validators, but as agents capable of moral and contextual judgment. If AI makes governance efficient, it must not make it unaccountable.

These are not just aspirational ideas—they are design principles, and increasingly, they are becoming formalizable. In 2024, the EU’s AI Act formally classified AI systems used in employment, education, and credit as “high-risk,” mandating transparency, data quality, human oversight, and documented accountability. In New York City, Local Law 144 now requires annual bias audits for automated hiring tools, with public disclosure and candidate notification. And in the U.S., the White House’s Blueprint for an AI Bill of Rights urges protections against algorithmic discrimination and pushes federal agencies to enforce civil rights standards even when decisions are made by machines. In my own research on algorithmic fairness, feedback, and strategic behavior, I explore how institutions can navigate these trade-offs in noisy, biased environments. But the key is to treat AI not merely as a system, but as a governance substrate—a meta-institution whose influence must itself be governed.

The Broader Point

The question is no longer whether AI governs. It’s how well it does it—and for whom.

AI systems are already performing core institutional functions: filtering applicants, shaping curricula, allocating credit, and moderating speech. They are evaluators, gatekeepers, and norm-setters. But they do so without transparency, without recourse, and often without awareness of their own normative consequences.

We are not governed by alien minds. We are governed by recursive procedures—by optimization routines and feedback loops that encode incentives we do not always see.

From resume filters that punished women’s chess captains to video interviews that claim to quantify your “communication ability” from facial tics, the story of AI’s rise isn’t one of intelligence alone—it’s one of silent institutional redesign.

We are not just delegating decisions—we are delegating the power to define what a decision is.

The real question, then, is not whether AI should have institutional power. It already does. The question is: what kind of meta-institution are we allowing it to become—and who gets to decide?

Sources and Further Reading

  • Yuval Noah Harari, Nexus: A Brief History of Information Networks from the Stone Age to AI (2024)
  • James C. Scott, Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed (1998)
  • Shoshana Zuboff, The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power (2019)
  • Bruno Latour, Science in Action: How to Follow Scientists and Engineers Through Society (1987) and Pandora’s Hope (1999)
  • Amazon’s resume screening AI: Jeffrey Dastin, “Amazon Scraps Secret AI Recruiting Tool That Showed Bias Against Women,” Reuters, Oct 2018. Link
  • HireVue facial analysis: Drew Harwell, “A Face-Scanning Algorithm Increasingly Decides Whether You Deserve the Job,” Washington Post, Oct 2019. Link
  • Workday lawsuit: Link
  • NYC Local Law 144 (2023): Full text
  • EU AI Act (2024): Summary
  • Blueprint for an AI Bill of Rights (2022): White House OSTP

For more of my work, visit my academic website.

AI Are the New Institutions

How algorithms gained decision-making power—and what they replaced

In my previous post, I argued that AI systems—particularly those built on large models—are no longer just tools. They are becoming institutions.

Not in the legal sense. But in function.

They structure judgment, allocate opportunity, and shape legitimacy, just as courts, schools, or hiring boards once did.

This post explains what that claim means in practice. Not through abstraction, but through concrete, recent events.

When I say “AI,” I don’t mean a single system. I mean a distributed set of deployed models—language models, scoring tools, surveillance networks—that now act with institutional authority in many parts of life.

What makes something an institution?

Institutions are not just buildings or bureaucracies. They are systems of trust and authority.

They determine:

  • Who gets hired.
  • Who is heard.
  • Who is protected or punished.
  • And who gets left out.

They operate over time. They are accepted, contested, appealed to.

And now, AI systems are being inserted into exactly these roles.

From automation to judgment

A2025 Australian study led by researchers at the University of Technology Sydney found that AI hiring systems widely used in the private sector were systematically discriminating against women, people with disabilities, and candidates with non-standard accents.

Some systems analyzed facial expressions during interviews, penalizing applicants who didn’t smile “appropriately.” Others struggled with speech patterns from non-native English speakers, or penalized pauses—often flagging neurodivergent candidates as “low energy.”
Most troublingly, many of these systems were trained on U.S. resumes and corporate data, leading to cultural mismatches in Australian hiring. (source)

The AI didn’t just screen résumés. It functioned as a hiring gatekeeper, shaping who advanced and who disappeared from consideration.

Prediction becomes policy

Los Angeles County recently launched a machine learning initiative called PREVENT to identify individuals most at risk of becoming homeless.

The system analyzes data across health records, public benefits, and eviction filings. It flags individuals for early intervention, offering rental assistance or job support—before they lose housing. (source)
In a world of limited social services, that score can determine who gets help and who doesn’t. Prediction becomes a gateway.

The model becomes a triage system, quietly encoding institutional priorities.

From oversight to automation

In 2024, New Orleans police used over 200 facial recognition cameras run by a private nonprofit to monitor and arrest suspects in real time—without public disclosure.

The system sent alerts with suspect identities and GPS locations directly to officers’ phones, resulting in at least 34 arrests—some for nonviolent offenses. (source)

This surveillance violated a 2022 city ordinance, which limited facial recognition use to violent crime investigations. Yet the system operated with full effect, effectively automating discretionary policing without accountability.

AI governs speech

For instance, in 2024, TikTok laid off hundreds of human moderators and leaned heavily on AI to enforce its content policies.

Over 150 million videos were removed in one quarter alone—more than 80% flagged automatically. (source)
Yet users have long noted that AI moderation lacks nuance, often taking down posts about trauma, race, or mental health from marginalized communities.
In response, creators began using “algospeak” (e.g., “unalive” instead of “suicide”) to evade takedowns.

When AI moderates, it doesn’t just enforce guidelines. It structures what can be said and what is suppressed.

And this is just the tip of the proverbial iceberg.

The power without the process

These systems don’t gain authority from constitutional law or public oversight.

They derive it from performance (or efficiency):

  • “It works.”
  • “It scales.”
  • “It predicts better than humans.”

But once adopted, their outputs become binding.
And there is often no one to appeal to.

The judgment is made before the human arrives.
And often, the human defers to it.

Why this matters

When institutions were flawed, we could appeal. Argue. Dispute. Reform.

With AI systems, we often don’t know the decision was made. Or we mistake it for truth.

That’s what I mean when I say: AI systems are becoming institutions.
Not because they were designed to be—but because they function like ones.

They make judgments. They allocate resources. They shape legitimacy.
But unlike human institutions, they offer no explanation, no transparency, and no path for redress.

Originally published on Substack: AI Are the New Institutions.

The Age of AI is the Age of Algorithmic Institutions

How machines are reshaping legitimacy, agency, and the structure of society

We often speak of AI as a tool—as something humans will use to become more productive, more creative, more efficient. But this framing misses something deeper.

AI is not just a tool. It is becoming an institutional force.

It already decides who is hired, who is approved for loans, which students are flagged for intervention, and what news people see. In many domains, AI doesn’t assist decision-makers—it replaces them, or constrains their choices within a narrow band of “optimized” outcomes.

This is no longer science fiction. It is our administrative present.

As a computer scientist, I’ve long worked on algorithms: their structure, efficiency, and fairness. But in the presence of AI, a new reality is emerging—one in which systems built from data and optimization objectives become gatekeepers of opportunity. Unlike traditional institutions, these systems do not derive legitimacy from democratic deliberation, legal precedent, or public accountability. They derive it from performance—often defined narrowly, statistically, and in ways that resist scrutiny.

We are now confronting urgent questions:

– When AI models mediate access to education or employment, what and whose values are embedded in the model?

– How does agency change when effort is filtered through a predictive lens trained on historical bias?

– What happens when institutions outsource moral judgment to machines?

– What kinds of futures are erased when AI systems optimize for what’s already worked?

This newsletter—Algorithms and Society—is my way of thinking through these tensions: structurally, mathematically, and morally.

I will write about:

– Fairness and bias in algorithmic systems, especially when AI is used to select or filter

– The evolving meaning of merit, effort, and agency in AI-mediated institutions

– How feedback loops between predictions and behavior can amplify inequality

– What it might mean to design AI systems for democratic legitimacy, not just accuracy

– And what the future of humanity—from work, to creativity, to intuition—might look like in the age of AI

This is not a place for hype, nor for alarmism. It’s a space to think clearly about what AI is doing to the systems we depend on—and how we might intervene.

If you’re a researcher, technologist, policymaker, or simply someone trying to understand how the future is being shaped—welcome.

Let’s ask the hard questions before the defaults become destiny.

Originally published on Substack: The Age of AI is the Age of Algorithmic Institutions