The sound of success: how game audio shapes player experience

Lennart Nacke, Mark Grimshaw and Craig Lindley ran one of the first careful experiments on game sound. Participants played a fast first-person shooter in which sound effects and music were independently switched on or off, while the researchers recorded skin conductance and facial muscle activity and gave a standard game experience questionnaire. The physiological measures showed no significant effects. The questionnaire did: sound had a significant effect on every dimension it measured, including immersion, tension, competence, flow, challenge and both positive and negative feelings. Sound and music also interacted in their effect on tension and flow.

That combination is instructive. Players felt the difference, clearly and across the board, even though the body measures available at the time did not capture it. Sound is not decoration layered on top of play; for the player, it is part of what play is.

In practice, game sound does two kinds of job. Some sounds carry information: a click that confirms a move registered, a different tone when an action fails, a rising note as you approach a goal. Others carry feeling: music, ambience, the texture of a world. The informational kind works best when it is short, consistent and mapped to one meaning. The emotional kind works best when it matches what the game is trying to make you feel, which is where the next study comes in.

Sources1Source 1Nacke, L. E., Grimshaw, M. N., & Lindley, C. A. (2010). More than a feeling: Measurement of sonic user experience and psychophysiology in a first-person shooter game. Interacting with Computers, 22(5), 336–343.

Why does music make a game more enjoyable? Christoph Klimmt and colleagues tested two possible routes in two experiments with young male players, 68 playing Assassin’s Creed: Black Flag and 59 playing Alien: Isolation, each with the soundtrack present or absent. The first route was emotional: music intensifies feelings, and stronger positive feelings mean more enjoyment. The second was cognitive: music might deepen the sense of being in the game world or identifying with the character.

The evidence supported the emotional route. Soundtrack music raised enjoyment indirectly through positive emotions, and in the horror game, shocking music intensified the horror experience. The researchers did not find that music worked through a greater sense of presence or identification. The samples were narrow, so the details may not generalise, but the lesson for designers is plausible: music is best thought of as an emotional instrument, and its emotional tone should match what the game wants the player to feel.

Sources2Source 2Klimmt, C., Possler, D., May, N., Auge, H., Wanjek, L., & Wolf, A.-L. (2019). Effects of soundtrack music on the video game experience. Media Psychology, 22(5), 689–713.

Losing track of time is one of the most commonly reported signs of immersion. Timothy Sanders and Paul Cairns tested whether music could produce it. They added music to a simple maze game and measured players’ sense of elapsed time with two methods well established in psychology. Music changed time perception in one of the two measures but not the other, and depending on which music was chosen, it could either increase or decrease immersion.

The authors describe the overall picture as complex, and that is the honest summary. Music can make play more absorbing, but the wrong music can make it less so, and its effect on time is real but not simple. For a calm game, that is a reason to choose music carefully and to let players turn it down, rather than assuming more soundtrack means more enjoyment. We have written about the other ingredients of a restful game in what actually makes a puzzle game calm.

Sources3Source 3Sanders, T., & Cairns, P. (2010). Time perception, immersion and music in videogames. Proceedings of HCI 2010, the 24th BCS Interaction Specialist Group Conference, Electronic Workshops in Computing.

Sound does not always dominate. Katja Rogers and colleagues compared the same horror game on a monitor and in virtual reality, and then tested different kinds of spatial audio in VR. They found that audio had a more implicit influence on the experience in VR, because the overall sensory experience was so strong, and that audio dimensionality did not appear to be a significant factor in player experience there.

It is a useful corrective. The contribution of sound depends on everything else the player is receiving, and in a rich visual environment it may be felt rather than noticed. Read from the other direction, the same result suggests why a small, simple game can benefit from sound so much. When the screen is quiet, the ear does more work.

Sources4Source 4Rogers, K., Ribeiro, G., Wehbe, R. R., Weber, M., & Nacke, L. E. (2018). Vanishing importance: Studying immersive effects of game audio perception on player experiences in virtual reality. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 1–13.

Here is the practical tension. Sound matters to experience, yet phones spend much of their lives muted: on trains, in waiting rooms, next to a sleeping partner. A player who is deaf or hard of hearing may never hear it at all. So sound should reinforce, not carry. Every audio cue that tells the player something should have a visual partner, and ideally a haptic one; Tanay Singhal and Oliver Schneider found that well-designed vibration that reinforces on-screen feedback can itself improve how enjoyable and immersive a game feels.

OutBrick is designed to be fully playable in silence. The nine brick friends have no voices and speak only in text bubbles, and everything that matters on a board, from a brick’s stop to a clear, is shown on screen. Colour is backed by glyphs on every brick, on by default, for the same reason: no single sense should be the only way in. Our accessibility page lists the rest, and our piece on game feel and juice looks at how feedback works across the senses. For the case against colour as the only clue, see why colour should never be the only clue in a puzzle.

• Pair every meaningful sound with a visual cue.
• Keep music, effects and any voice on separate volume controls.
• Save the most distinctive sound for the rarest, best moment.
• Test the whole game muted before you ship it.

Sources5Source 5Singhal, T., & Schneider, O. (2021). Juicy haptic design: Vibrotactile embellishments can improve player experience in games. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1–11.