Satya Nadella, the CEO of Microsoft, recently proclaimed on X (formerly Twitter) that we should assume all AI models are “compromised,” highlighting the perils of highly advanced AI and how to tackle these risks. Nadella argues that AI can no longer be treated as a “set of nested black boxes” whose advice and actions we simply accept or reject. He advocates for a more transparent system where models can be contained, observed, and leave behind “tamper-proof human-readable evidence.” Many of his recommendations echo those from other industry leaders: timely incident disclosure, independent audits, verifiable data, and containment. It’s on the last point—containment—that Nadella appears to stumble.

While Nadella’s vision of AI containment is laudable, it borders on the overly optimistic. Consider the sheer scale of modern AI models, from GPT-4 to the next-gen transformer architectures. These models are trained on vast datasets and have billions of parameters, making them as intricate as a bustling metropolis. To contain such a city within a single glass box, as Nadella suggests, might be easier said than done. One could argue that the evidence these models produce—tamper-proof human-readable evidence—is often as elusive as spotting a unicorn in a dense forest.

Moreover, Nadella’s call for independent audits and timely incident disclosure is spot-on, but it raises the question: who pays for these audits? The cost of comprehensive AI audits can be astronomical, often exceeding the budget of smaller AI startups. If Nadella’s vision is to be realized, the industry must shoulder the financial burden, possibly through government subsidies or AI-specific tax credits. Without such support, the promise of independent audits may remain a pipedream.

Lastly, Nadella’s emphasis on “tamper-proof human-readable evidence” suggests that the AI’s inner workings should be as clear as a crystal ball. However, the very nature of AI—its reliance on complex algorithms and vast datasets—means that its decision-making processes can be as opaque as a foggy morning. While Nadella’s “tamper-proof” evidence is a noble goal, it may require new paradigms in AI explanation, such as visualizing neural network activations or employing explainable AI (XAI) techniques. Until these paradigms mature, Nadella’s vision of fully contained and observed AI models remains a tantalizing but not yet fully realized future.