OpenAI says it stopped an Asian scam campaign hijacking ChatGPT to lure in victims

OpenAI disrupted a major scam campaign in Cambodia’s Poipet, banning accounts and blocking new registrations

Criminals used ChatGPT to run romance fraud, fake investments, bogus police scams, and even operations linked to human trafficking

AI was leveraged for fake personas, fraudulent messages, job lures, and worker administration; OpenAI shared findings with authorities after evidence of hundreds of victims losing thousands of dollars

OpenAI said it disrupted a major scam campaign coming out of Southeast Asia by banning accounts used by the operation and making it difficult to open new ones.

In a blog post, the ChatGPT maker said it was tipped off about the existence of the campaign by WhatsApp .

After investigating further, the company found that a group of criminals in Cambodia used its AI tool in different scam campaigns - romance fraud, fake investment scams, or bogus police investigations. It also used AI to help with day-to-day operations and, most worryingly, for things that could be related to human trafficking and forced criminality.

Detention and escape attempts

OpenAI did not say if the group had a name, just that it operated out of Poipet, “a city in Banteay Meanchey province that public reporting⁠ has repeatedly⁠ linked⁠ to online scam compounds and trafficking operations.”

The group used AI to create fake personas, to help draft fraudulent emails and chat messages, and to create posters for fake jobs which were probably used to lure people into human trafficking or forced labor.

The tool was also used for worker administration, since the operators maintained records of employee debts, salary deductions, disciplinary fines, and loan repayments. They also used the tool to translate discussions about immigration status, work permits, visa overstays, and recruitment incentives.

“Some conversations also referenced apparent detention, escape attempts, and potential criminal liability for people who had been trafficked and forced to work in scam operations,” OpenAI said. “While these conversations do not allow us to determine the circumstances of any particular individual, they are consistent with extensive public reporting⁠ describing⁠ the activities of organized crime groups in Southeast Asia.”

The company could not say how many people fell victim, or how much money this group has stolen, but said it found evidence of “hundreds” of victims losing “thousands of dollars.” These findings were shared with relevant authorities.

Via The Hacker News

Indexed and credited by AIPROPX. Originating outlet: TechRadar. Open at source →