Playing games with a screen reader: how blind players game

A screen with nothing to feel looks, at first, like a poor fit for someone who cannot see it. Research helped show otherwise. Kane, Bigham and Wobbrock’s Slide Rule, presented in 2008, used multi-touch gestures to make a touchscreen phone usable without sight: sliding a finger reads out whatever is beneath it, a second finger selects, and flicks move through lists. In their study with ten blind participants, Slide Rule was significantly faster than a button-based alternative and was preferred by seven of the ten, though it produced more errors.

Apple’s VoiceOver, which arrived on the iPhone the following year, works on the same principles. Touch anywhere and VoiceOver speaks the item under your finger. Swipe right or left to move from item to item in order. Double-tap anywhere to activate what was last spoken. A virtual dial called the rotor changes what swipes do, from moving by headings to adjusting speaking rate. The screen can even be turned off entirely.

Learning it takes time. Rodrigues and colleagues followed blind people through their first weeks with a smartphone screen reader and found that mastering the device was a long, demanding process, confirming the worries participants had before they began. That is worth remembering when a game adds its own gestures on top.

Sources1Source 1Kane, S. K., Bigham, J. P., & Wobbrock, J. O. (2008). Slide rule: Making mobile touch screens accessible to blind people using multi-touch interaction techniques. In Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility (pp. 73–80). ACM.2Source 2Rodrigues, A., Montague, K., Nicolau, H., & Guerreiro, T. (2015). Getting smartphones to TalkBack: Understanding the smartphone adoption process of blind users. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility (pp. 23–32). ACM.

Andrade and colleagues surveyed and interviewed gamers with visual impairments in 2019, deliberately looking at the games they actually play rather than games built for a study. The picture that emerged was of a community with long histories of play, strong opinions about the industry, and a particular appreciation of games that balance complexity with accessibility. Many preferred games with real depth to simplified “accessible” games that felt patronising.

Gonçalves, Rodrigues and Guerreiro looked at playing together. From interviews with ten adults and ten children with visual impairments, 140 survey responses and a second survey of sighted people who play with them, they found blind players playing a wide range of games, but rarely the same games as their sighted friends and family. The problem was not only inaccessibility. It was that the two groups’ gaming worlds seldom overlapped, so there was little to share. We look at the family side of that in playing games with your grandchildren.

Sources3Source 3Andrade, R., Rogerson, M. J., Waycott, J., Baker, S., & Vetere, F. (2019). Playing blind: Revealing the world of gamers with visual impairment. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1–14). ACM.4Source 4Gonçalves, D., Rodrigues, A., & Guerreiro, T. (2020). Playing with others: Depicting multiplayer gaming experiences of people with visual impairments. In Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility (pp. 1–12). ACM.

Yuan, Folmer and Harris’s survey of game accessibility describes play as a loop: the game presents stimuli, the player decides on a response, and the player gives input. Most games deliver their primary stimuli visually, and a blind player needs those stimuli replaced, usually with speech, sound or vibration. When the stimulus can be put into words, a screen reader can do much of that work. When it cannot, the game has to carry the load with its own audio design, as the audio games made for blind players do.

That gives a useful test. Turn-based games, card games, word games and many puzzles can be described completely in words: what is where, what can move, what has changed. Fast action games usually cannot, because the information arrives faster than speech can deliver it. That is one more reason to value puzzles without a clock: listening takes time, and a timer punishes it. Research on text-based games by Spiel, Bertel and Heron found that even the way directions are phrased matters, with non-sighted players finding directions relative to themselves easier and more immersive than compass directions.

• Every control has a spoken label that says what it does, not what it looks like.
• The state of play, such as score, moves left or whose turn it is, can be read on demand.
• Changes are announced: a piece moved, a level cleared, a turn passed.
• Nothing depends on a timer or on hitting a small moving target.
• The game does not trap focus in a pop-up or leave VoiceOver reading stale content.

Sources5Source 5Yuan, B., Folmer, E., & Harris, F. C. (2011). Game accessibility: A survey. Universal Access in the Information Society, 10(1), 81–100.6Source 6Spiel, K., Bertel, S., & Heron, M. J. (2014). Navigation and immersion of blind players in text-based games. The Computer Games Journal, 3(2), 130–152.

Some games need gestures that VoiceOver would otherwise intercept, such as dragging a piece across a board. Apple’s platform lets a developer mark an area as allowing direct interaction, which passes touches straight to the game while VoiceOver keeps reading elsewhere. Apple’s VoiceOver guidance for designers covers the basics. Used carefully, direct touch lets a board feel physical; used carelessly, it creates a region where the screen reader goes silent and the player is lost.

The best games offer both: a direct way to play for players who can use it, and a step-by-step way, element by element, for players who cannot. In OutBrick, VoiceOver names every brick by its colour, shape and position, offers its slides and swaps as actions, says where a brick stopped and what went home, and labels controls such as Undo, Pause and Close; and because there is never a clock, nothing is lost by taking time to listen. A board has a move limit, not a timer, and the first undo on every board is free. We know a sliding puzzle is a demanding thing to play by ear and we would rather hear from VoiceOver players than guess; the accessibility page lists what ships today and how to reach us.

If you are new to VoiceOver, Apple includes a practice area in Settings, under Accessibility, VoiceOver, where you can try the gestures without anything happening. Start with games that are turn-based and text-rich; card games, word games and trivia are good first steps. Then look for games whose descriptions mention VoiceOver by name, and read their reviews from blind players.

If you are sighted and play with someone who is blind, the research above suggests the most useful thing you can do is find games you can both play, rather than games each of you plays alone. Turn-based puzzles on a shared device are an easy place to start. And if a game you love fails at the basics, say so to the developer. Many teams do not know there is a problem until someone tells them. Our own support page is where to start for OutBrick.

Sources4Source 4Gonçalves, D., Rodrigues, A., & Guerreiro, T. (2020). Playing with others: Depicting multiplayer gaming experiences of people with visual impairments. In Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility (pp. 1–12). ACM.