AI Snake Oil

Introduction: Real Progress Versus Fake Artificial Intelligence

Imagine a world where people only use the word vehicle to describe everything from a bicycle to a spacecraft. Discussions about whether vehicles are good for the environment would be impossible because one person is thinking of a truck while the other is thinking of a bike. This is the current state of our conversation regarding artificial intelligence. AI is not one single thing; it is an umbrella term for vastly different technologies. To understand the modern landscape, we must distinguish between generative AI, which creates new content, and predictive AI, which attempts to forecast human behavior.

Arvind Narayanan suggests that the most significant problems arise when we fail to see these distinctions. Generative AI, like ChatGPT or image creators such as Midjourney, has shown remarkable, genuine progress. It works by identifying statistical patterns in massive amounts of data to remix and generate text or visuals. While it is prone to making up facts because it does not actually know things, it is a functional tool that can assist with research or coding. The danger here is primarily misuse, such as people using it to churn out low-quality, error-filled books or students using it to cheat on essays.

Predictive AI is a different story entirely. This technology is sold as a way to predict social outcomes, such as who will be a good employee, who will commit a crime, or how long a patient needs to stay in a hospital. Narayanan argues that much of this is AI snake oil, meaning products that do not and cannot work as advertised. Unlike identifying a face in a photo, which is a clear technical task, predicting a person’s life path is inherently difficult because human behavior is complex and influenced by countless invisible factors. When companies claim their software can judge a job candidate’s character based on a thirty-second video, they are often selling a random number generator dressed up in scientific jargon.

The proliferation of this fake technology is fueled by a cycle of hype involving researchers, companies, and the media. In the academic world, many studies claiming high accuracy for AI are flawed due to the phenomenon of data leakage, where the AI is accidentally given the answers during its training phase. In the corporate world, companies often keep their methods secret and avoid independent audits that would prove their tools do not work. Meanwhile, the media often treats every corporate press release as a breakthrough, rarely asking the skeptical questions necessary to protect the public.

This lack of clarity has real-world consequences. In healthcare, faulty predictions have led to insurance companies cutting off care for elderly patients who were still in pain. In the legal system, risk scores used to determine bail are often barely more accurate than a coin flip, yet they influence whether a person stays in jail. Even technologies that work, like facial recognition, present a double-edged sword. While it can help find missing children, it can also be used by governments to track protesters or by private venues to ban lawyers who are suing them.

To navigate this era, we must move past sweeping generalizations. We should embrace the AI tools that clearly work and make life easier, like spam filters or navigation apps, while maintaining deep skepticism toward any system that claims to predict the future of a human being. The goal is to develop the vocabulary to challenge fake AI and ensure that as technology advances, it serves to augment human dignity rather than automate discrimination and error.