Online harassment, predatory behavior, and toxicity is pretty common and rampant in video games but in the last three years or so it has been highlighted more than ever before.
Earlier this year Roblox was exposed as a breeding ground for predators and back in 2021 Sony’s own executive, George Cacioppo, was caught trying to meet a minor. It was then clear that there is a need for better measures to protect vulnerable groups like minors online.
Sony is now taking a big step in that direction. The makers of PlayStation plan to leverage AI technology to run pseudo accounts that are designed to mimic the behavior of specific demographics, including minors.
By imitating how minors interact online, the system can draw out predators who target this demographic.
When a predator account interacts with a pseudo account, the entire interaction in monitored and flagged without putting real users at risk of exposure.
But the applications go beyond just protecting minors.
While the primary focus is on creating decoy accounts that attract predatory users, the system’s broader design could also identify and deter toxic behavior, such as harassment or bullying.
Toxic users reveal patterns of harmful behavior that can be flagged before they escalate toward legitimate users.
The entire idea is to focus on both prevention and protection. The system detects harmful behavior through AI decoy accounts and makes sure real accounts aren’t exposed – are shielded from harmful interactions by intercepting offenders before they can act.
