Sometimes you read a quote and you can see the firestorm that’s about to set in, and that day happened with the head of Xbox game studios, Matt Botty, who “dreams” of using AI in quality assurance testing for future video games, so he asked Microsoft for his help. in knowing how to do it.
Booty, speaking in Pax Westhe had a lot of thoughts on the idea of AI-based QA testing, given the demands of modern games, but his comments are bound to spark controversy since QA has always been an underpaid part of game development, and abuse has caused syndicates to emerge. In some studios lately, including one that Microsoft is about to own.
Here’s what Booty says about the QA issue and potential AI fix, via VGC:
“Some of the processes that we have, have not really kept pace with the speed at which we can create content. One of those is testing.”
You think of a game, one of the biggest differences between a game and something like a movie, is if we’re working on a movie and you come and say ‘Hey, this is the end, let’s fix this, let’s straighten this out, let’s cut this scene’, it usually doesn’t break anything at the beginning the film.
“But in a game, you can be ready to charge, and the designer is like, ‘I have this little feature, I’m just going to change the color on that one thing’ and then somehow something explodes now the first 10 minutes of the game not playing. So, that aspect Demo, every time anything new goes into a big game, the whole game has to be tested, front to back, side by side.
“My dream – there is a lot going on with AI and machine learning at the moment, people using AI to create all these images. What I always say when I meet AI folks, is: ‘Help me figure out how to use an AI bot to go for a game test’ .
“Because I would love to be able to start 10,000 game instances in the cloud so there are 10,000 copies of the game running, deploy an AI bot to spend the whole night testing this game, and then we get a report in the morning. Because that would be transformative.”
Booty is right that testing is very difficult in games, and what we’re constantly seeing is a lot of big, bug-ridden titles released, not necessarily because anyone did their job wrong, but because millions of players will be making more time to “test” the latest version of the game in hours and days The first few things the entire QA team can do in months or even years. This has led to some uncomfortable situations, such as blaming testers for seemingly not detecting Cyberpunk 2077’s bugs, pre-launch, and discussions about how true this is. But the size problem is real.
But is artificial intelligence the answer? This is bound to be more controversial. The human side of the argument is to get studios to move towards more QA testers, pay them better, and reduce their hours so they are more efficient at work and better able to spot errors. But here, Booty seems to suggest automating the process, and if you want to an act Invested in an AI QA testing system, it looks like it will have the potential to get an entire part of the industry out of business. Not that anything like this actually exists in a meaningful way yet. But I know many will take Booty’s “my dream” feedback here as somewhat problematic given the plight of current QA workers. A reference to AI image creation, where programs like Midjourney currently bother a lot of human artists, might also not be the best point of comparison here.
We’ll see if Booty clarifies his observations, which he seems likely to do, given the conversation that resulted, but his initial statement is also very emphatic about where he’d like to see the industry move in, if that’s at all possible.