Will voice control to become a widely-used feature in video games?
Marian Viola / Voice User Interface Architect at VUI.agency / 05.10.2021
Sports gaming release season 2021 is here.
Imagine your all-time favorite games being taken to the next level of interaction with voice control.
From a sports game perspective, having intense last minutes on the pitch while playing FIFA, being down 1-2, and trying to outplay your opponent simultaneously can be frustrating.
Why not focus on not losing the ball and ask for a substitution or a new formation using voice control while keeping your fingers on the essential buttons?
In the next minutes you are going to read about different examples of games with added voice features, the challenges and opportunities that voice brings to the gaming field, in the quest to increase user engagement. Shouldn’t we experiment and use more openly new technologies, such as voice, that facilitate that?
FIFA 15 & 16 went exploring
Some of you might remember that substitutions via voice commands have already been possible in FIFA 15 and 16 but were later removed again for a reason: The process simply took too long, was too complicated, the voice control was more or less predetermined and thus the feature was not as helpful for the players as it could have been: FIFA 15 how to make a sub by using voice recognition.
What went wrong?
First of all, the ASR (Automatic Speech Recognition) and the NLU (Natural Language Understanding) have not been on a sufficient level for a smooth experience. Back then, it was not possible to speak in a natural way. Let’s take an example: if you are playing FIFA and want to make a substitution, you would say something like:
“Take Sané off and replace him with Gnabry.”
“Sané off, Gnabry on”
The users’ odyssey in FIFA 15 and 16 can be seen in the following pictures.
At first, you had to say “Make a sub”, then you had to choose the player’s position, e.g. “Defender”.
Then, FIFA showed you the four defenders that were currently on the pitch and gave them names from SUB 1 to SUB 4.
Source: FIFA 15 by EA SPORTS
Following that, you could choose from the three defenders who have been your substitution players (SUB 1).
The conversation between the player and FIFA then looked like this:
“Make a sub (1-2 sec pause).
Defender (1-2 sec pause).
SUB 1 (1-2 sec pause).
SUB 1 (1-2 sec pause).”
Not very intuitive, right?
However, just because the technology might not have been ready at the time of FIFA 15 and 16, or the implementation might not have been ideal, it doesn’t mean that we can’t add voice control features in the future that actually facilitate in-game interaction.
Current video game adaptations of voice technology and trends.
Up to now, voice has rather been an experimental feature for the big publishers and not always successful as the above example of FIFA shows. However, Ubisoft’s 2017 game Star Trek: Bridge Crew was a step forward for in-game voice interaction. You can play the game with or without a virtual reality (VR) headset, based on IBM Watson, to give single players the opportunity to command a virtual crew with their voice.
Voice technology is also used in a very innovative way in the co-op horror game Phasmophobia in which traditional screen or VR gameplay is combined with a voice chat that not only the other players react to, but also the ghosts in the game. This very immersive experience has led to a great deal of fascination among the gaming community: In October 2020, Phasmophobia was on rank 8 of the most-watched video games on Twitch with a total of 48.6 million viewing hours.
In a considerable number of mods (modifications of original games), games situated in fantasy worlds are enhanced with voice recognition and/or VR, often in combination.
Space-based action role-playing game (RPG) Star Wars: Squadrons is such a mod, together with fantasy RPG Skyrim, which is one of the best-selling video games of all time. Moreover, GTA V with mods nowadays is more popular than the original game that was released in 2013. If voice recognition becomes more widely available in games, players could interact with non-playable characters (NPCs) in a natural dialog. Some worlds would also lend themselves quite willingly to voice control, e.g. casting spells in Harry Potter would be much more natural if that could be done by just saying the spells out loud instead of having to draw a pattern or to choose from some menu.
In fact, one fan-made game allows the Potterheads among us to do just that: Ravenclaw Common Room VR Gameplay. Interacting via voice would also enable fans of a specific universe to immerse themselves much more deeply in the game.
Apart from including voice in-game, voice has also been used as an additional channel for players to interact with content that they know from an already-existing game:
Electronic Arts (EA) released an Alexa skill to go with The Sims 4 in 2019. Users can learn about the history of the game, play music from all The Sims games or play a Sims trivia quiz.
At the same time, the Sims in The Sims got their own voice assistant Hein-Z in-game. It remains to be seen if in the future voice can also be used by the players to control in-game interaction in The Sims.
Quite recently, EA has also patented voice-controlled RPGs (Role Playing Games). The patent describes a system that (machine-)learns to associate specific groups of custom user utterances with a specific in-game action of a NPC. With such a system, voice technology could be used to give commands to the NPCs (Non-Player Characters) while controlling the player’s character in a different way in parallel.
Even gaming hardware is becoming more and more apt for speech. For instance, the new PlayStation 5 controller comes with an in-built microphone which could make a headset superfluous as it also filters background noise.
/ Marian Viola
As voice technology is becoming more and more present in everyday life (today already 25% of Germans own a smart speaker and this number is predicted to grow), it will probably only be a matter of time until voice control is used more in video games.
Now, what do you think? Can you imagine voice to be part of your favorite game? As a game developer or product owner, would you add voice to your gaming project? Do you have an idea for an entirely new spoken gaming experience? Or do you think that voice will never partly or entirely replace traditional gameplay interaction? Let us know on our Twitter account.