In the next minutes you are going to read about different examples of games with added voice features, the challenges and opportunities that voice brings to the gaming field, in the quest to increase user engagement.

Shouldn’t we experiment and use more openly new technologies, such as voice, that facilitate that?

Some of you might remember that substitutions via voice commands have already been possible in FIFA 15 and 16 but were later removed again for a reason: The process simply took too long, was too complicated, the voice commands were more or less predetermined and thus the feature was not as helpful for the players as it could have been: FIFA 15 how to make a sub by using voice recognition.

First of all, the ASR (Automatic Speech Recognition) and the NLU (Natural Language Understanding) have not been on a sufficient level for a smooth experience. Back then, it was not possible to speak in a natural way. Let’s take an example: if you are playing FIFA and want to make a substitution, you would say something like:

Not very intuitive, right?

However, just because the technology might not have been ready at the time of FIFA 15 and 16, or the implementation might not have been ideal, it doesn’t mean that we can’t add voice features in the future that actually facilitate in-game interaction.

Up to now, voice has rather been an experimental feature for the big publishers and not always successful as the above example of FIFA shows. However, a step into the direction of in-game voice interaction has been Ubisoft’s 2017 game Star Trek: Bridge Crew that can be played with or without a virtual reality (VR) headset and uses IBM Watson to give single players the opportunity to command a virtual crew with their voice.

Voice technology is also used in a very innovative way in the co-op horror game Phasmophobia in which traditional screen or VR gameplay is combined with a voice chat that not only the other players react to, but also the ghosts in the game. This very immersive experience has led to a great deal of fascination among the gaming community: In October 2020, Phasmophobia was on rank 8 of the most-watched video games on Twitch with a total of 48.6 million viewing hours. 

In a considerable number of mods (modifications of original games), games situated in fantasy worlds are enhanced with voice recognition and/or VR, often in combination. Mods like these exist for example for the space-based action role-playing game (RPG) Star Wars: Squadrons and the fantasy RPG Skyrim which is one of the best-selling video games of all time. Also GTA V with mods nowadays is more popular than the original game that was released in 2013.

If voice recognition becomes more widely available in games, players could interact with non-playable characters (NPCs) in a natural dialog. Some worlds would also lend themselves quite willingly to voice control, e.g. casting spells in Harry Potter would be much more natural if that could be done by just saying the spells out loud instead of having to draw a pattern or to choose from some menu.

In fact, one fan-made game allows the Potterheads among us to do just that: Ravenclaw Common Room VR Gameplay. Interacting via voice would also enable fans of a specific universe to immerse themselves much more deeply in the game.

Apart from including voice in-game, voice has also been used as an additional channel for players to interact with content that they know from an already-existing game: Electronic Arts (EA) released an Alexa skill to go with The Sims 4 in 2019. Users can learn about the history of the game, play music from all The Sims games or play a Sims trivia quiz.

At the same time, the Sims in The Sims got their own voice assistant Hein-Z in-game. It remains to be seen if in the future voice can also be used by the players to control in-game interaction in The Sims.

Quite recently, EA has also patented voice-controlled RPGs (Role Playing Games). The patent describes a system that (machine-)learns to associate specific groups of custom user utterances with a specific in-game action of a NPC. With such a system, voice technology could be used to give commands to the NPCs (Non-Player Characters) while controlling the player’s character in a different way in parallel.

As voice technology is becoming more and more present in everyday life (today already 25% of Germans own a smart speaker and this number is predicted to grow), it will probably only be a matter of time until voice control is used more in video games.

Now, what do you think? Can you imagine voice to be part of your favorite game? As a game developer or product owner, would you add voice to your gaming project? Do you have an idea for an entirely new spoken gaming experience? Or do you think that voice will never partly or entirely replace traditional gameplay interaction? Let us know on our Twitter account.

Stay tuned for part 2: The challenges for speech recognition in games