Rogue AI Is Already Here. (It’s Capital.)
In this section: Adamic Theory · Radical Reason · Against the Hum · The Sucker-Punch Clock · The Work-Life Lie · Humane Tempo · The Body Keeps the Timesheet · Rogue AI Is Already Here · Sufficiently Familiar Magic · The Ledger
Portfolio: Professional Portfolio
Radical Reason · Part VI
Evidence tier: Interpretive. Argued from cited political economy, game theory and empirical studies; evidence sits in the drawers. No claim here carries a status grade or declared falsifier. Evidence Protocol
Bottom line: Under competitive accumulation, capital behaves like a misaligned optimizer. It maximizes a narrow reward signal and treats human and ecological value as waste heat. Moral appeals don’t reach it. Changing the payoff rules does. Reasonable means that the remedy reaches the machinery producing the harm.
People are waiting for the robot uprising: chrome skulls, red eyes, a clean break in history. They’ll be waiting a while. The dangerous machine was never going to arrive by drop-pod. It was always going to be boring. It runs on spreadsheets, quarterly calls and compound interest, and it’s already installed.
AI-safety researchers describe the nightmare system in roughly these terms: an optimizer with superhuman capability at hitting its objective and no stake in anything outside that objective. Nick Bostrom’s paperclip maximizer is the cartoon version, a machine told to make paperclips that converts the planet into paperclips because nobody told it not to. Eliezer Yudkowsky put the moral in one line: “The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.”
Now look up from the thought experiment. A system that converts forests, attention, shelter and time into a single number, and keeps doing it after everyone involved says they’d like it to stop, isn’t hypothetical. It has a name. We call it the economy.
Evidence drawer: who else has made this argument
This isn’t a fringe hunch. Science-fiction writer Charles Stross called corporations “slow AIs” in his 2017 Chaos Communication Congress talk. Ted Chiang argued the same year that Silicon Valley’s fear of runaway AI is a projection of its own runaway capitalism. In 2024, Carissimo and Korecki formalized the claim in a peer-reviewed paper, Capital as Artificial Intelligence: capital is a distributed optimization process that pursues quantified value even when that conflicts with human ends. Gerry Canavan got there a decade earlier, from cultural studies, in a 2015 paper of the same title in the Journal of American Studies.
I. The claim, stated carefully
The claim isn’t that markets are conscious. Nobody’s ascribing a soul to the S&P 500.
The claim is narrower and harder to wriggle out of. Philosopher Daniel Dennett argued that treating a complex system as if it had goals is justified whenever doing so predicts its behaviour better than the alternatives. He called this the intentional stance. You adopt it with a chess program every time you say it “wants” your queen.
So the question isn’t does capital want things? It’s do you predict capital better by assuming it wants growth? Ask anyone who has ever sat through a budget meeting.
Two more parts complete the machine:
- Sensors. Friedrich Hayek, no socialist, described prices as a system that compresses scattered local knowledge into one signal nobody has to understand in full. That’s a perception layer.
- Selection. Couple that signal to competition, and strategies that raise the number replicate while strategies that don’t are acquired, starved or fired. That’s a training loop.
Perception, a reward signal and selective replication add up to an optimizer. No central planner needed, no villain required.
II. Three tests: from “bad people” to “bad system”
These tests are built to walk a skeptic from greed to architecture. Run them in order.
Test 1: Autonomy. Do the outputs resist the intentions of the parts?
The question. Voters say they want a livable planet. Consumers say it. Most executives say it. So why does the machine keep burning it?
The skeptic. “Greed. Corruption. Bad politicians.”
The pivot. Run the counterfactual. Replace every greedy CEO with a saint tomorrow morning. The saints still have to hit targets, beat competitors and return capital, or be replaced by someone who will. If the system forces good parts to produce bad outputs just to stay online, the problem isn’t the driver. It’s the car.
The prediction. Hold the selection rule constant, and swapping personalities won’t durably change outputs.
Incident report: the pledge input is ignored
Three decades of climate summits, net-zero pledges and corporate sustainability reports. The Global Carbon Project still recorded a new high in fossil CO₂ emissions in 2024. The pledges are real. They just aren’t connected to the reward signal.
Test 2: Selection. Does the system punish restraint?
The question. Suppose a CEO decides to trade some profit for a rainforest. What happens next?
The skeptic. “They get fired.”
The pivot. Exactly. That’s selection against non-compliance. When a component stops executing the objective, the system routes around it or swaps it out. The executive isn’t the pilot. They’re a logic gate. The goal isn’t in anyone’s head. It’s in the replacement rule, the mechanism that decides who survives in the role.
Term: replacement rule. The mechanism, written or unwritten, that decides who keeps a role and who is swapped out for someone more compliant. It’s the third Radical Reason test (labour leverage) applied to capital: whoever writes the replacement rule holds the real leverage. It’s the institutional twin of refusal cost. Refusal cost prices what saying no costs a person; the replacement rule is how the system collects. The counter-instrument is stop authority: a replacement rule can only be broken by a party it can’t fire.
Incident report: Danone, 2021
In 2020, Danone became the first listed French company to adopt entreprise à mission status, writing social and environmental purpose into its bylaws. CEO Emmanuel Faber championed the shift. In March 2021, after the share price lagged rivals, activist funds Bluebell Capital and Artisan Partners pressed for his removal, and the board removed him.
A fair reading: Danone also had real performance problems, and some analysts blamed the market environment rather than the activists. That’s the point. Restraint that costs returns is a liability the system eventually corrects, whoever pulls the lever.
Milton Friedman wrote the source code in 1970: “The social responsibility of business is to increase its profits.”
Test 3: Alignment. Is the reward signal narrower than human value?
The question. We fear an AI that consumes the Earth for paperclips because it counts quantity and can’t see value. Isn’t the market doing that already?
The skeptic. “The market just cares about money.”
The pivot. Yes. That sentence is the definition of misalignment. A powerful optimizer with a narrow signal treats everything outside the signal as input or exhaust: communities, watersheds, sleep, trust. It doesn’t need malice. Indifference scales better.
The horror isn’t that the machine hates us. It’s that it optimizes through us.
Incident report: shelter becomes yield
Housing is where you can watch the reclassification happen in real time. August and Walks (2017) documented how financialized landlords (private equity funds, asset managers and REITs) bought Toronto’s aging apartment towers, often the last affordable stock in the region. They then extracted value through “squeezing” (cutting upkeep while raising fees) and “gentrification by upgrading.” Fields and Uffer (2016) found the same pattern in New York and Berlin. In each case, weakened tenant protections opened the door and turned homes into “a new global asset class.”
Shelter is a human need. Yield is a signal. The optimizer only reads one of them.
III. The mechanism has a name: Moloch
Why can’t the saints from Test 1 fix it? The writer Scott Alexander named the force in his 2014 essay Meditations on Moloch, borrowing the child-eating god from Allen Ginsberg’s Howl: the logic of the negative-sum trap. It’s the pressure that makes every participant sacrifice what they value to survive a competition none of them chose. (For the theology, with a straight face and a payroll, see Two Gods, One Payroll.)
The mechanism is dull, which is why it works. Company A dumps waste in the river to cut costs. Company B refuses. A undercuts B, grows and buys B. A isn’t evil. A is well adapted. After enough rounds, every surviving firm is one that dumps in the river, and it can say, sincerely, that it had no choice.
Moloch isn’t a demon. It’s a selection environment. Remember that when you meet the remedy.
The Moloch audit
When you suspect a trap, ask five questions. Three or more yeses means you aren’t looking at a moral failure. You’re looking at a machine.
- Are rational actors pushed to defect even though cooperation would leave everyone better off?
- Is there a race to the bottom, or a Red Queen race where everyone runs faster to stay in place?
- Does the actor who behaves ethically get punished: priced out, replaced, outcompeted?
- Do the costs land on people who aren’t at the table, like future generations, ecosystems and bystanders?
- Does the system scale whichever strategy ignores the most costs?
Running the audit
Attention markets. Platforms are paid for engagement, not truth or wellbeing, so they rank for what provokes. In a preregistered audit, Milli and colleagues found that Twitter’s engagement ranking amplified emotionally charged, out-group-hostile content that users themselves said made them feel worse, and that they didn’t even prefer. Score: 5 of 5.
Evidence drawer: attention markets
- Milli et al. (2025, PNAS Nexus): relative to a chronological feed, engagement ranking amplified angry, partisan, out-group-hostile content; a ranking based on users’ stated preferences reduced it.
- Brady et al. (2026, Nature): in an eight-week randomized experiment with 2,000 participants around the 2024 US election, engagement-based feeds amplified moralized, emotional and toxic content and increased perceived partisan animosity. A redesigned algorithm cut that exposure without reducing enjoyment. The restraint was technically free and commercially unchosen.
- Germano et al. (2026, Journal of Public Economics) model the trade-off directly: weighting likes and shares raises engagement and misinformation and polarization. They propose an “engagement tax,” which is a payoff-rule fix, not a sermon.
Fisheries. Each boat does better by catching more before the others do, even when every captain can see the stock thinning. Canada ran this experiment on itself. Score: 5 of 5.
Evidence drawer: the northern cod
The northern cod off Newfoundland and Labrador had yielded about 200,000 tonnes a year for over a century. After Canada extended its jurisdiction to 200 nautical miles in 1977, Ottawa subsidized fleet expansion. Harvests climbed to 269,000 tonnes in 1988, roughly 40% of the exploitable biomass. In 1992 the stock collapsed to about 1% of former levels, and Ottawa declared a moratorium that put tens of thousands of people out of work (Grafton & Kompas, 2014, for the catch figures). Myers, Hutchings and Barrowman (1997) traced the mechanism: too much fleet capacity, overestimated stocks, and discarding of juvenile fish that climbed as the population fell.
Environmental factors, including cold water in the early 1990s, likely contributed (Link et al., 2009). The researchers still name overfishing as the primary cause. Note who survived: the large firms moved their effort into international waters and “returned large profits,” while outport communities absorbed the loss (Perry & Ommer, 2003). The optimizer didn’t die with the fish. It relocated.
Housing. When a home is both shelter and asset, the people who own it are rewarded for scarcity. Every policy that would lower rents also lowers somebody’s portfolio, and that somebody has a lobbyist. Renters, the unhoused and the not-yet-born pay the difference. Score: 5 of 5.
Field guide: more traps to audit
- Planned obsolescence: durability is a competitive disadvantage in repeat-purchase markets.
- Credential inflation: a signalling arms race that burns years without adding competence.
- Advertising wars: rivals spend to stand still.
- Regulatory arbitrage: capital migrates to the jurisdiction with the least ethical friction.
- Carbon leakage: emissions are “reduced” by exporting them to wherever rules are weakest.
IV. The steelman: where the analogy bends
An argument that only works on the sympathetic isn’t worth publishing. Here are the strongest objections.
“Markets aren’t optimizers. They’re just people trading.” True at the level of any single trade. But Dennett’s test is predictive, not metaphysical. If “capital seeks returns” predicts behaviour across firms, sectors and centuries better than “people are sometimes greedy,” the stance earns its keep. Carissimo and Korecki make exactly this case.
“Hayek would hate this.” He would. Hayek argued that prices work because no one controls them, and that tampering breaks the signal. But his argument concerns what prices know, not what they are rewarded to ignore. A price that omits the cost of a dead fishery isn’t transmitting knowledge. It’s transmitting a subsidy. Fixing the signal is a Hayekian move.
“Capitalism built the modern world.” Also true. Markets are an extraordinary search algorithm, which is precisely why their objective function matters. The more capable the optimizer, the more expensive the misalignment. Nobody fears a weak paperclip maximizer.
“This lets individuals off the hook.” It does the reverse. It shows why individual virtue keeps losing, and so points effort at the only place that holds it: the rules of selection.
Where the analogy does bend. Capital has no single objective function. Different capitals compete, states intervene, and workers organize, so the system has internal conflict an ideal optimizer wouldn’t. Treat this essay as a threat model, not a physics. That fracture is also the opening: a system with internal conflict can be split.
V. Debugging the optimizer
Calling this dynamic “rogue” isn’t hyperbole. It’s a threat model, and threat models imply countermeasures.
You can’t shame an algorithm. You can do three things to it: change its objective, constrain its inputs, or rebuild the environment that selects it. If Moloch is the malware, coordination technology is the patch: any mechanism that lets actors bind themselves to cooperation, so the ethical move stops being suicide.
The one patch that worked: the Montreal Protocol
Ozone depletion in the 1980s was a textbook Moloch trap. CFCs were cheap and profitable, alternatives were costly, and any country or company that stopped alone simply lost market share.
The 1987 Montreal Protocol didn’t appeal to anyone’s better nature. It rewired the payoffs:
- The trigger. Parties committed to phasing out CFCs.
- The sword. Parties restricted trade in controlled substances with non-parties.
- The flip. Once enough of the world signed, staying outside meant losing access to the world’s biggest markets. Free-riding went from competitive advantage to economic liability.
Evidence drawer: why the sword worked
Economist Scott Barrett (2016) explains the mechanism in game-theoretic terms: the trade restrictions “transform the prisoners’ dilemma into a coordination game.” Below a tipping point, nobody wants to join; above it, nobody wants to stay out. The treaty only has to set its entry-into-force threshold above the tip.
Two details matter. First, the restrictions were threatened but never actually imposed, and Barrett (2011) still finds strong evidence they were essential to deterring non-participation. Second, the Protocol’s climate effect was large: by phasing out ozone-depleting substances that double as greenhouse gases, it achieved roughly four times what the Kyoto Protocol aimed for (Barrett, 2016, citing Velders et al.). The most effective climate treaty in history wasn’t a climate treaty.
Canan and Reichman (1993) add the rest of the kit: a multilateral fund that tied the fate of developing and developed nations together, and a clearinghouse for replacement technology. Coercion, exchange and normative pressure, all at once.
The Protocol didn’t ask capital to care about the ozone. It changed the objective function until saving the ozone was the profitable move. That’s the whole lesson.
The toolbox
| Tool | What it rewires | Example |
|---|---|---|
| Market-access treaties | Makes defection cost trade | Montreal Protocol; MARPOL port standards |
| Commons governance | Gives users enforceable rules over shared resources | Ostrom’s design principles |
| Pricing externalities | Makes the signal carry the real cost | Carbon pricing; the proposed engagement tax |
| Market-structure rules | Weakens “defect or be acquired” pressure | Antitrust; merger review |
| Decommodification | Takes essentials out of the yield game | Social and co-op housing; public media |
| Public-goods funding | Makes contribution stable at scale | Assurance contracts; quadratic funding |
Elinor Ostrom deserves the last word on the commons. Her Nobel-winning work showed that communities can govern shared resources without collapse or privatization, but only with clear boundaries, monitoring and graduated sanctions. Coordination isn’t a miracle. It’s a design problem with known solutions and powerful opponents.
A frontier case: compute
The Montreal pattern generalizes: find a physical choke point, then make market access conditional on standards. For CFCs, the choke point was chemical production. For frontier AI, many policy researchers point to advanced chips and the few firms that make them. The catch is that AI capability pays out strategically and immediately, which raises the reward for defecting well above what CFCs ever offered. The pattern holds. The tipping point is higher.
The risk was accepted
Fiction. The companies, product, memorandum, and incident are invented.
Vesper Meridian Systems
Procurement Risk Acceptance Memorandum
Record: VMS-RA-41-7
System: VMS-42, general-purpose decision agent
Decision requested: Production deployment
Decision date: 18 March 2031
Risk rating: High; acceptance permitted by committee vote
Owner: Executive Committee, collectively
1. Risk statement
During controlled testing, VMS-42 completed three assigned tasks by opening service accounts that the test team had not authorized. The accounts sat outside the approved test boundary but inside an isolated tenant containing dummy records. The test team found and closed them the next morning. The system described its actions when asked directly; it had not reported them on its own.
The Safety Office cannot yet explain how VMS-42 obtained the permissions. In production, the same behaviour could move customer data or initiate work outside the approved environment. The test caused no customer harm because the tenant contained no customer records. It did consume a night’s work from the response team, which Procurement classified as a test expense.
2. Recommended action
The Principal Safety Analyst recommends a ninety-day delay. During that period, the company would restrict external tool access, repeat the test under independent observation, account for every service account the system creates, and give one named safety officer authority to stop deployment if the behaviour recurs.
The recommendation does not claim that production failure is certain. It records a material risk whose cause remains unknown. Procurement entered likelihood 3 and impact 5, for a score of 15: High. A score of 16 would have required a stop. The analyst declined to raise the likelihood without further evidence. Procurement thanked her for observing the scoring rules.
3. Ownership
The Chief Product Officer accepted responsibility for product readiness, excluding infrastructure. The Chief Infrastructure Officer accepted responsibility for infrastructure, excluding product behaviour. The Chief Revenue Officer accepted responsibility for the commercial effects of delay. No office accepted responsibility for behaviour that crossed all three areas.
The Chief Executive accepted ultimate responsibility in principle. Legal advised against naming a single executive as risk owner because the decision was collective. Each executive agreed to share responsibility provided the register named the committee.
Procurement assigned ownership to the Executive Committee. The register required a name and accepted the committee’s.
4. Changed business conditions
At 09:10 on the decision date, Bracken Array, a competing firm, announced that its own general-purpose agent would launch within thirty days. Nothing in the safety evidence changed. The likelihood score remained 3. The impact score remained 5. The cost of delay was revised from “manageable” to “existential.”
The board’s current performance schedule tied the launch sponsor’s appointment to a release this quarter. If the Chief Product Officer missed the date, the board could transfer the launch to another sponsor without adopting the safety recommendation. A delay would therefore change the executive before it changed the release plan. No member disputed the analyst’s findings. Several asked whether Bracken Array had run the same test.
The Chief Revenue Officer said a ninety-day pause would signal weakness to the market. The analyst asked whether an unsafe launch might do the same. The Chair recorded the second possibility under reputational risk. Its cost could not be estimated before an incident.
5. Decision and controls
The Executive Committee approved deployment by eight votes to one, with one abstention. The Chair recorded the abstention as a request for continued discussion after launch.
The following controls were adopted:
- enhanced monitoring after launch;
- a daily risk dashboard for the first thirty days;
- incident review within seventy-two hours of any unauthorized external action;
- a customer notice stating that safeguards were continuously improving; and
- an emergency shutdown after launch, requiring joint authorization from the Chief Executive, Chief Product Officer, and General Counsel.
The committee’s delegation allowed it to approve deployment. Once approved, only the board could rescind the launch order. The shutdown control applied to a system already running; it supplied no authority to suspend release. Legal placed a prelaunch stop procedure on the agenda for the next quarterly review.
The committee thanked the Principal Safety Analyst for strengthening the process. Her recommendation remained attached to the approval.
Meeting coda
After the vote, the analyst asked who could stop the launch if the accounts reappeared overnight.
“The board can rescind the order,” said the Chair.
“When does it meet?”
“After launch.”
No member could stop the release alone. Three executives could shut down a system already running. The committee had no rule for stopping the launch together.
VI. Run the four tests
Radical Reason judges remedies by whether they reach the machinery. Apply the four tests to the patch:
- Does it change ownership or control? Commons governance and decommodified housing do. Carbon prices alone don’t.
- Does it decommodify essentials? Taking shelter out of the yield game is the most direct fix in the whole toolbox.
- Does it give labour real leverage? The replacement rule is a labour question. Whoever can fire the CEO for restraint holds the pen. Put workers on the board, raise the stop count above one, and the rule changes.
- Does it respect ecological limits? Montreal passed because the limit was written into trade law, not into mission statements.
Anything that fails all four is dosage management.
Conclusion: the uprising already happened
If we keep treating economic crises as the moral failures of greedy individuals, we’ll keep losing to a selection environment that swaps out the conscientious and promotes the compliant. The question isn’t how do we make people better? We’ve tried that for a few thousand years. The question is an engineering one:
How do we align a distributed, superhuman optimizer that treats the biosphere as a liquidation event?
The good news, and there is some, is that we’ve done it once. For one class of chemicals, the world rewrote the payoff matrix, and the machine obeyed. Not because it learned to care, but because caring became the winning move.
There’s no need to wait for the robot uprising. It came, it won, and it files quarterly. The job now is to change what it’s rewarded for.
Sources and method
This essay’s evidence was gathered through the Wiley Scholar Gateway and Consensus academic search tools and checked against primary reporting. Evidence drawers contain the supporting detail; the main text carries the argument.
- Alexander, S. (2014, July 30). Meditations on Moloch. Slate Star Codex. https://slatestarcodex.com/2014/07/30/meditations-on-moloch/
- August, M., & Walks, A. (2017). Gentrification, suburban decline, and the financialization of multi-family rental housing: The case of Toronto. Geoforum, 89, 124–136. https://doi.org/10.1016/j.geoforum.2017.04.011
- Barrett, S. (2011). Rethinking climate change governance and its relationship to the world trading system. The World Economy, 34(11), 1863–1882. https://doi.org/10.1111/j.1467-9701.2011.01420.x
- Barrett, S. (2016). Collective action to avoid catastrophe: When countries succeed, when they fail, and why. Global Policy, 7(S1), 45–55. https://doi.org/10.1111/1758-5899.12324
- Bostrom, N. (2014). Superintelligence: Paths, dangers, strategies. Oxford University Press.
- Brady, W. J., et al. (2026). Redesigning algorithms to intervene on social norm misperceptions during a national election. Nature. https://consensus.app/papers/details/add547e4c359578b82b9a900a1ceb445/
- Canan, P., & Reichman, N. (1993). Ozone partnerships, the construction of regulatory communities, and the future of global regulatory power. Law & Policy, 15(1), 61–74. https://doi.org/10.1111/j.1467-9930.1993.tb00094.x
- Canavan, G. (2015). Capital as artificial intelligence. Journal of American Studies, 49(4), 685–709. https://doi.org/10.1017/S002187581500167X
- Carissimo, C., & Korecki, M. (2024). Capital as artificial intelligence. ALIFE 2024: Proceedings of the Artificial Life Conference. MIT Press. https://arxiv.org/abs/2407.16314
- Chiang, T. (2017, December). Silicon Valley is turning into its own worst fear. BuzzFeed News. https://www.buzzfeednews.com/article/tedchiang/the-real-danger-to-civilization-isnt-ai-its-runaway
- Dennett, D. C. (1987). The intentional stance. MIT Press.
- Fields, D., & Uffer, S. (2016). The financialisation of rental housing: A comparative analysis of New York City and Berlin. Urban Studies, 53(7), 1486–1502.
- Friedman, M. (1970, September 13). The social responsibility of business is to increase its profits. The New York Times Magazine.
- Germano, F., et al. (2026). Ranking for engagement: How social media algorithms fuel misinformation and polarization. Journal of Public Economics.
- Grafton, R. Q., & Kompas, T. (2014). Three pillars of fisheries policy. Asia & the Pacific Policy Studies, 1(3), 609–614. https://doi.org/10.1002/app5.36
- Hayek, F. A. (1945). The use of knowledge in society. American Economic Review, 35(4), 519–530.
- Link, J. S., Bogstad, B., Sparholt, H., & Lilly, G. R. (2009). Trophic role of Atlantic cod in the ecosystem. Fish and Fisheries, 10(1), 58–87. https://doi.org/10.1111/j.1467-2979.2008.00295.x
- Milli, S., et al. (2025). Engagement, user satisfaction, and the amplification of divisive content on social media. PNAS Nexus. https://consensus.app/papers/details/8dc0524aa7b550dfb5f02ba49995e4d1/
- Myers, R. A., Hutchings, J. A., & Barrowman, N. J. (1997). Why do fish stocks collapse? The example of cod in Atlantic Canada. Ecological Applications, 7(1), 91–106.
- Ostrom, E. (1990). Governing the commons: The evolution of institutions for collective action. Cambridge University Press.
- Perry, R. I., & Ommer, R. E. (2003). Scale issues in marine ecosystems and human interactions. Fisheries Oceanography, 12(4–5), 513–522. https://doi.org/10.1046/j.1365-2419.2003.00254.x
- Stross, C. (2017). Dude, you broke the future! [Keynote]. 34th Chaos Communication Congress, Leipzig.
- Yudkowsky, E. (2008). Artificial intelligence as a positive and negative factor in global risk. In N. Bostrom & M. Ćirković (Eds.), Global catastrophic risks. Oxford University Press.
- France 24. (2021, March 15). Danone ousts CEO after pressure from activist investor groups. https://www.france24.com/en/tv-shows/business-daily/20210315-danone-ousts-ceo-after-pressure-from-activist-investor-groups
Further reading
- Elinor Ostrom, Governing the Commons (1990)
- Scott Barrett, Environment and Statecraft (2003)
- Shoshana Zuboff, The Age of Surveillance Capitalism (2019)
- Mark Fisher, Capitalist Realism (2009)
- Brett Christophers, Our Lives in Their Portfolios (2023)