AI Expert Says Reinforcement Learning is ‘Fantastically Bad Idea’

Michael Cohen warns concept of out-of-control AI is 'perfectly plausible'

Michael Cohen, an expert on AI, poses for a photograph in London, on May 25, 2023. NTD

An expert in artificial intelligence has warned against “ploughing ahead” with certain types of AI which could have potentially disastrous consequences for the human race.

Michael Cohen, a doctoral candidate in Engineering Science at Oxford University, said his fears were shared by Geoff Hinton, a Canadian professor often described as the “godfather of AI,” and computer scientist Yoshua Bengio.

In a blog posted last month, Bengio said a rogue AI “could behave in ways that would be catastrophically harmful to a large fraction of humans, potentially endangering our societies and even our species or the biosphere.”

Speaking to NTD’s Lee Hall for the “British Thought Leaders” programme, Cohen said the biggest danger was in using “reinforcement learning” to train AI systems to carry out certain tasks.

Cohen said: “So the idea behind reinforcement learning is we administer and withhold rewards, according to some conception of what we want it to do. And we hope that in order for the rewards it has to do what we want. But if it could remove humanity’s control over our infrastructure, take it over for itself, prevent humans from ever getting in its way it could direct a robot to give it the reward instead of us.”

The original T-800 Endoskeleton robot used in filming Terminator Salvation is displayed at the Science Museum in London, England, on February 7, 2017. Carl Court/Getty Images

He drew an analogy with a bear overpowering its trainer to obtain the treats the trainer is using.

Cohen said: “The concept of reinforcement learning really depends on the reinforcer. Being more powerful than the reinforced and successes of reinforcement learning today should be recognised as dependent on that fact.”

“So I think ploughing ahead with reinforcement learning and similar algorithms is a fantastically bad idea,” he added.

Cohen said: “If they were capable enough, they would intervene in the provision of their own reward and spend as much energy as they needed to spend to secure their control over that.”

Earlier this year, while giving evidence to a parliamentary committee, Cohen was asked by Conservative MP Tracey Crouch to “expand on some of the risks ... posed by AI systems to their end users.”

Cohen replied, “There is a particular risk … which is that it could kill everyone.”

Cohen: These Warnings are not ‘Fanciful’

On Britain’s Thought Leaders, Cohen said many of those who had warned of the dystopian consequences of AI had been criticised for being “fanciful” and for being too far removed from the subject.

But he said, “Now that Geoff Hinton is saying similar things, Yoshua Bengio is saying similar things—both Turing Award winners—those sorts of characterisations of this position are falling apart.”

World-renowned computer scientist Stuart Russell, in his book Human Compatible, also warns of the dangers of an out-of-control AI.

Cohen said: “This is not a fringe position. Despite sounding a bit like sci-fi, it just turns out to be a perfectly plausible thing.”

He said he was not proposing “we should just destroy our computers,” like in the sci-fi prequel book, “Dune: The Butlerian Jihad.”

Cohen said he believed there needed to be a pause in the progress of this type of AI development and also urged governments to bring in legislation to manage the risk.

Cohen said: “I think that the most likely route to success in crafting regulation and societal norms that prevent us from developing dangerously advanced artificial agents is one which leaves open the possibility of potentially tremendous economic growth offered by other AI algorithms.”

He said one of the biggest dangers was that AI was not born with a moral compass like a normal human being and did not have the natural ethical inhibitors which usually prevented mankind from carrying out atrocities.

What Would AI do if Told to Eradicate a Disease?

For example, if AI asked to eradicate a disease, it might act differently to a human doctor who would prioritise finding a cure for a disease.

Cohen said: “(It might decide to) kill everyone who might potentially contract that disease. We have an intuitive sense of constraints, when we’re plotting courses of action towards achieving our goals. And while we can expect a very intelligent artificial agent to understand our common sense, restrictions on our own behaviour, the very different question is how we can get it to care.”

He was also asked about the impact of AI on people, in the wake of a report by Goldman Sachs which recently estimated 300 million jobs in the United States alone would be lost or degraded by the use of automation using artificial intelligence.

Cohen said he was more concerned about people losing money, rather than not having purpose in their lives.

He said: “I think the landed gentry of 150 years ago got on fine without work being their source of purpose. But if they’re not able to get an income to support their family that would, of course, be a problem. A big one, if it’s on the scale that it could be.”

But Cohen pointed out Goldman Sachs also projected a lot of economic growth from AI and he said, “If governments can tax that and redistribute it, that could be a win-win.”

Cohen said that prior to the industrial revolution around 98 percent of people were farmers.

“Technology destroyed a lot of those jobs. But they were re-employed in other jobs. I’m not sure how long of a latency there was. I’m not sure how challenging that was for people. But if we had banned the technology that prevented our transition from an agrarian society, we'd be in a much worse position,” he said.

“So I think those problems from job loss from AI are likely solvable with the tools we have today, especially the tools of taxation and wealth redistribution,” Cohen concluded.