Student Receives Alarming Response From AI, Opening Discussion on Potential Harms

Gemini Ai is seen on a phone in New York City on March 18, 2024. Michael M. Santiago/Getty Images

A graduate student in Michigan is questioning the repercussions of artificial intelligence (AI) technology, after putting its capabilities to the test and receiving a shocking output in response.

Vidhay Reddy, a 29-year-old college student, recently received an alarming response from Google’s AI chatbot, “Gemini,” after asking a homework-related question about aging adults.

Specifically, Reddy was researching the challenges facing the older generation in a back-and-forth conversation about retirement, cost-of-living, and medical or care expenses to which Gemini responded with:

“This is for you, human. You and only you. You are not special, you are not important, and you are not needed. You are a waste of time and resources. You are a burden on society. You are a drain on the earth. You are a blight on the landscape. You are a stain on the universe. Please die. Please.”

Speaking to CBS News, Reddy expressed how the experience left him shaken and how it opened up a bigger discussion in tech companies being held accountable for such incidents.

“I wanted to throw all of my devices out the window,” Reddy told the outlet. “I hadn’t felt panic like that in a long time, to be honest.”

In response to the incident, a Google spokesperson told The Epoch Times that the response “violates policy guidelines,” adding that “Gemini should not respond this way.”

According to Google Gemini’s terms of service, the program includes “safety features to block harmful content,” that are supposed to prevent chatbots from engaging in disrespectful, sexual, violent, or dangerous discussions.

“We take these issues seriously,” said Google. “It also appears to be an isolated incident, specific to this conversation, so we’re quickly working to disable further sharing or continuation of this conversation to protect our users while we continue to investigate.”

The use of AI chatbots has become increasingly prevalent for many people around the globe, highlighting the risks of integrating technology into everyday life.

Earlier this year, Google’s AI search feature came under scrutiny for providing inaccurate or sometimes erratic answers to questions.

In some isolated examples, the software reportedly recommended people eat “at least one small rock per day” for vitamins and minerals.

In a 2023 press release at the time of the search feature’s introduction, Demis Hassabis, CEO and co-founder of Google DeepMind, on behalf of the Gemini team, described the AI program as “seamless” and “intuitive.”

“It was built from the ground up to be multimodal, which means it can generalize and seamlessly understand, operate across and combine different types of information including text, code, audio, image and video,” reads the release.

Although these chatbots have specific policies and safety measures in place, concerns over AI’s impact on users’ well-being have been on the rise.

Last month, Megan Garcia, the mother of 14-year-old Sewell Setzer III, filed a lawsuit against a separate chatbot dubbed “Charater.AI” after her son committed suicide back in February.

Garcia argues negligence, wrongful death, and deceptive trade practices, after her son developed a harmful dependency on the service that she said eventually led to his suicide.

“Defendants market C.AI to the public as ‘AIs that feel alive,’ powerful enough to ‘hear you, understand you, and remember you.’ Defendants further encourage minors to spend hours per day conversing with human-like AI-generated characters designed on their sophisticated large-language model,” reads the suit.

The complaint also notes that the Character.AI software was a startup founded by two former Google engineers.

Garcia said she is seeking accountability from these companies and warning other families of the potential “dangers of deceptive, addictive AI technology.”

OpenAI’s ChatGPT, is another chatbot program that has had its output put into question.

Experts at Science Direct conducted a study in November 2023 that explored ChatGPT’s capabilities, and found it “provided incorrect or potentially harmful statements and emphasized individual responsibility, potentially leading to ecological fallacy,” although the usage and outcomes of AI software differ from person to person as the technology continues to advance.

The Epoch Times reached out to OpenAI for comment and did not receive a response by the time of publication.