AI, SI or BI? The case for calling it Behavioural Intelligence

On Tuesday, in a speech to the UN General Assembly, Donald Trump announced that artificial intelligence should now be called “super intelligence”. He said the word “artificial” made the technology sound fake (BBC News). Some of his allies adopted the term within hours. The name did not come out of a technical process. Trump had recently run an online survey asking followers to choose between superior intelligence (SI), extreme intelligence (EI) and supreme intelligence (SI). “Super intelligence” was not one of the options (Washington Times). 

Academics were less keen. Simon Coghlan, Senior Lecturer in Digital Ethics at the University of Melbourne, told the BBC the rebrand was misleading, because super intelligence usually refers to a system that can improve itself. Ben Leong, Associate Professor of Computer Science at the National University of Singapore, expects professionals to reserve “SI” for systems more advanced than anything available today.

Both are right, but the problem started long before Tuesday. “Artificial intelligence” has misled people since the 1950s, and “super intelligence” would make it worse. Ahead of #RISK Expo Europe this November, I want to propose a name that describes what these systems do: Behavioural Intelligence.

A name chosen to focus a field and exclude a rival

The term first appeared in a 1955 funding proposal for a summer workshop at Dartmouth College, written by John McCarthy with Marvin Minsky, Nathaniel Rochester and Claude Shannon. McCarthy later said he chose it because few papers in Automata Studies, a collection he had co-edited, dealt with making machines behave intelligently, and he wanted to focus the workshop on that goal (McCarthy, Stanford).

He had a second motive. Norbert Wiener had already named a neighbouring field in his 1948 book Cybernetics: Or Control and Communication in the Animal and the Machine. McCarthy wanted to escape any association with cybernetics. He wrote that he did not want to accept Wiener as a guru or have to argue with him. As Foreign Affairs has pointed out, a blander name would probably have attracted less attention from Hollywood and journalists.

The new name raised expectations that the technology could not meet. Early researchers predicted human-level machines within a generation. When those predictions failed, funding collapsed twice, first in the 1970s and again in the late 1980s. ChatGPT brought both the name and the expectations back.

What the everyday person hears

Most people outside the industry associate “intelligence” with understanding, judgement and feeling. Add “artificial” and they picture a synthetic mind that thinks like us but runs on silicon. Add “super” and they picture a mind that thinks better than we do.

Neither describes the tools people use today. A large language model predicts which words are likely to follow the ones you gave it, based on patterns in a vast amount of human writing. When a chatbot appears to understand your question, it is matching patterns, and when it sounds sympathetic, it is producing the words that sympathy usually looks like.

We have made this mistake before. In January 1966, Joseph Weizenbaum of MIT published a paper describing ELIZA, a program that responded like a therapist by picking out keywords and substituting text. In his published transcript, a woman says she might learn to get along with her mother, and ELIZA replies, “Tell me more about your family.” Weizenbaum later wrote that he was startled by how quickly and deeply people became emotionally involved with the program and how readily they treated it as human (Computer Power and Human Reason, 1976).

The Turing Test has the same blind spot. In his 1950 paper, Alan Turing avoided the question “Can machines think?” because neither word could be clearly defined, and proposed an imitation game instead. So from the start, the test has judged behaviour. In a study published this year in PNAS, people judged GPT-4.5 to be human 73% of the time when it was told to adopt a humanlike persona. Without that instruction, the figure fell to 36%. The result shows that the model’s performance was convincing. It does not show that the model understood anything.

Why SI moves in the wrong direction

In research, superintelligence means a hypothetical system that outperforms the best human minds and can improve itself. Nick Bostrom gave the term its standard definition in his 2014 book Superintelligence: Paths, Dangers, Strategies: an intellect that greatly exceeds human cognitive performance in virtually all domains of interest. Coghlan notes that experts would regard such a system, if it came about, as the most consequential technology ever made.

It also weakens the term at the moment governments need it. Ahead of the same assembly, more than 20 mostly European countries called for binding safety measures on advanced AI. Neither the United States nor China endorsed the proposal. A word that has to carry a specific meaning in that negotiation is now being used for tools that autocomplete email.

The case for Behavioural Intelligence

By McCarthy’s own account, the name was meant to focus researchers on making machines behave intelligently. The founders were describing behaviour from the start, and the branding got ahead of that description. Behavioural Intelligence brings the name back in line with it.

The name is accurate in three ways. First, these systems produce behaviour in the form of text, images, predictions and, increasingly, actions. We can only judge them by that output, because there is no inner experience to assess.

Second, they learn from recorded human behaviour. A language model learns from what people wrote, a fraud model from what fraudsters did, and a hiring model from who was hired before. The system reflects our past behaviour back to us, including our prejudices. A 2018 study in Science Advances found that COMPAS, a risk-scoring tool used in US courts, was no more accurate or fair than people with little or no criminal justice expertise. A simple model using two features matched the accuracy of COMPAS’s 137. Two years earlier, a ProPublica investigation found that the tool wrongly labelled Black defendants as likely to reoffend at twice the rate of white defendants.

Third, “behavioural” sets the right expectation for the person using it. Behaviour is something you observe, test and supervise, and someone has to be accountable for it. People are less likely to defer to it the way they might defer to an intelligent colleague.

The objection I expect

BI already stands for business intelligence, and a room of risk and compliance professionals will say so. “Behavioural” also describes behavioural analytics and behavioural biometrics, which analyse human behaviour rather than generate it. I think the clash is manageable, since business intelligence and AI increasingly sit in the same technology stack. It is still the strongest argument against the name.

What leaders should ask, whatever the label

The practical test does not depend on the name. Ask what the system predicts, what data it learned from, how it performs on data it has never seen, and who is accountable when it gets something wrong. A good label encourages people to ask those questions. “Artificial intelligence” discourages them slightly, and “super intelligence” discourages them far more.