We're used to video game characters repeating the same lines of scripted dialogue. But artificial intelligence is now allowing players to have unique, unscripted conversations with the characters they meet. In Tech 24, we look at this experimental tech and its path to the mainstream.
Hobbyists have turned classic games like The Elder Scrolls: Skyrim and Fallout 4 into AI-powered playgrounds. Thanks to the downloadable mod Mantella, you can talk to non-playable characters (NPCs) with a microphone, and they'll respond in character. The technology is powered by the groundbreaking capabilities of AI models released by companies like Google and Meta, and opens up whole new ways to interact in games.
Some players find it so immersive that they develop deep connections. One forum user, for instance, shared their experience with a Skyrim character called Uthgerd, with whom they claimed to have conceived a child.
“We role played her water breaking and her giving birth which was wild … SkyrimVR is definitely therapy ... when I play it I am instantly put in a good mood.”
We didn't go that far when we tested the experience on Tech 24. But we did have a few conversations with NPCs that proved the technology worked, and worked well enough that it was difficult to distinguish between AI dialogue and dialogue from the original game. We found that it required a lot of investment and patience to get it running smoothly; characters would often misunderstand or respond with a significant delay. But for those willing to invest time and money, it's easy to see how it can become an engrossing experience. Paying for better AI models can make the characters quicker and more interactive, even recognising things they see around them with computer vision. Downloading better voice models can make them sound more natural. And there's the option of putting on a virtual reality (VR) headset to get fully immersed.
Under the alias 'Brainfrog' on YouTube, Eddie publishes videos of his adventures talking to AI NPCs in VR.
"You're creating your own world with your own relationships and I found myself building genuine relationships with these characters," said Eddie, "There are a substantial amount of issues and times where you run into this stuff and you're just like, oh, I'm speaking to a dumb computer, but there are some moments that everything aligns and it's really, really breathtaking."
The tech might be expanding beyond the realm of the modding community and hitting mainstream games sooner than you think.
Just this week, developers for Meta’s Horizon Worlds VR experience were let loose on tools letting them design characters who can respond with AI.
The hugely popular battle royale game Fortnite introduced an AI-voice Darth Vader earlier this year which proved wildly popular, and developer Epic Games is promising custom AI characters by the end of this year.
The next boundary that AI labs are pushing is virtual characters that don’t just react verbally, but interact physically with the space around them. The Financial Times newspaper reported this Sunday that Elon Musk’s xAI is joining this race against the likes of Meta and Google, poaching specialists from AI behemoth Nvidia.
But while it seems the tech giants are going full steam ahead on supercharging games with AI, there are many dissenting voices within the industry.
Michael Douse, head of publishing at Larian Studios which developed Baldur's Gate 3, said this week that AI could not solve the biggest problem of the games industry, "leadership and vision."
"What this industry needs is not more mathematically produced, psychologically trained gameplay loops," he posted on X, "Rather more expressions of worlds that folks are engaged with, or want to engage with."
English (US) · 