Does gamification work? What the meta-analyses actually say

In 2014 Juho Hamari, Jonna Koivisto and Harri Sarsa published one of the first systematic reviews of gamification research, looking at the peer-reviewed empirical studies available at the time. Their title asked the question directly. Their answer was a qualified yes: gamification did tend to produce positive effects, but those effects depended greatly on the context in which it was used and on the people using it.

The word itself is part of the problem. “Gamification” covers everything from a progress bar in a language app to a full story-driven training programme. Asking whether it works is a little like asking whether medicine works: the answer depends entirely on which one, for what, and for whom. The reviews that followed were attempts to break the question into parts that could actually be answered.

They also pointed out weaknesses that would dog the field for years. Many studies were small, many lacked a proper comparison group, and many bundled several game elements together, so it was impossible to tell which one did the work. A badge, a leaderboard and a progress bar introduced at once can lift engagement; the study cannot tell you which of the three to keep.

Sources1Source 1Hamari, J., Koivisto, J., & Sarsa, H. (2014). Does gamification work? A literature review of empirical studies on gamification. In Proceedings of the 47th Hawaii International Conference on System Sciences (pp. 3025–3034). IEEE.

By 2019 the field had grown enormously. Koivisto and Hamari reviewed 819 studies of gamification and found that results in general still leaned positive, but that the amount of mixed results was, in their word, remarkable. Education, health and crowdsourcing were the most common settings, and points, badges and leaderboards remained the most common tools, which is to say that much of the field was still testing the most obvious features in the most obvious places.

Johnson and colleagues reached a similar conclusion for health and well-being apps. Of the 19 studies they found, 59% reported positive effects and 41% mixed effects, with mostly moderate or lower quality of evidence. Results were clearest for health behaviours such as physical activity and much less clear for cognitive outcomes. Many studies did not compare a gamified app with the same app without the game elements, which is the one comparison that would isolate the effect of gamification itself.

Sources2Source 2Koivisto, J., & Hamari, J. (2019). The rise of motivational information systems: A review of gamification research. International Journal of Information Management, 45, 191–210.3Source 3Johnson, D., Deterding, S., Kuhn, K.-A., Staneva, A., Stoyanov, S., & Hides, L. (2016). Gamification for health and wellbeing: A systematic review of the literature. Internet Interventions, 6, 89–106.

Education is where the evidence is strongest, because classes can be compared. Michael Sailer and Lisa Homner combined the results of controlled studies and found small positive effects of gamification on cognitive learning outcomes (Hedges’ g = 0.49), motivational outcomes (g = 0.36) and behavioural outcomes (g = 0.25). Importantly, only the effect on cognitive outcomes held up when they looked at the most methodologically rigorous studies alone; the effects on motivation and behaviour were less stable.

Shurui Bai, Khe Foon Hew and Biyun Huang pooled 30 interventions with 3,202 participants and found a medium overall effect on learning (g = 0.504). They also looked at what students said. Students liked gamification because it fed their enthusiasm, gave them feedback, recognised their progress and helped them set goals. They disliked it when it added nothing useful or when it made them anxious or jealous, which is the leaderboard problem we looked at in do leaderboards motivate or discourage?

An effect size of about 0.5 means the average student in a gamified class did better than roughly two-thirds of students in a comparable class without it. It is a real effect, not a transformation.

Sources4Source 4Sailer, M., & Homner, L. (2020). The gamification of learning: A meta-analysis. Educational Psychology Review, 32(1), 77–112.5Source 5Bai, S., Hew, K. F., & Huang, B. (2020). Does gamification improve student learning outcome? Evidence from a meta-analysis and synthesis of qualitative data in educational contexts. Educational Research Review, 30, 100322.

When Sailer and Homner looked for moderators, two stood out. Game fiction, a story or setting that gives the activity meaning, made gamification more effective for behavioural outcomes. So did combining competition with collaboration, rather than setting everyone against everyone. Both point in the same direction: the gamification that works looks less like a scoreboard stuck onto a chore and more like an actual game.

The individual elements matter less than people expect. In the experiment by Mekler and colleagues, points, levels and leaderboards each increased how much work people did, but none changed how motivated they felt. Game elements are levers on effort. Whether that effort feels good depends on the task underneath.

Sources4Source 4Sailer, M., & Homner, L. (2020). The gamification of learning: A meta-analysis. Educational Psychology Review, 32(1), 77–112.6Source 6Mekler, E. D., Brühlmann, F., Tuch, A. N., & Opwis, K. (2017). Towards understanding the effects of individual gamification elements on intrinsic motivation and performance. Computers in Human Behavior, 71, 525–534.

Gamified apps are everywhere, from fitness trackers to language courses to workplace training, and most of them arrive with a confident claim about engagement. The research gives you a few simple questions to ask before believing it.

The first is what the claim is being compared with. A gamified app that beats doing nothing has shown very little; the useful comparison is the same app without the game layer, which is exactly the comparison the health review found missing in many studies. The second is what was measured. More logins and more taps are behaviour, and game elements reliably move behaviour in the short term. Whether people learned more, got fitter or enjoyed it more is a separate question with weaker answers.

The third is how long the study ran. Many gamification studies last days or weeks, and a feature that is exciting in week one can be wallpaper by week ten. The semester-long classroom study in our piece on whether leaderboards motivate or discourage is a reminder that effects can even reverse over time. None of this means gamification is useless. It means that the honest version of the claim is usually smaller, and more specific, than the marketing.

Sources3Source 3Johnson, D., Deterding, S., Kuhn, K.-A., Staneva, A., Stoyanov, S., & Hides, L. (2016). Gamification for health and wellbeing: A systematic review of the literature. Internet Interventions, 6, 89–106.2Source 2Koivisto, J., & Hamari, J. (2019). The rise of motivational information systems: A review of gamification research. International Journal of Information Management, 45, 191–210.

There is an irony in all this. Gamification borrowed its tools from games, and the research keeps rediscovering that the tools are not what made games compelling. Games work because the core activity is interesting, the feedback is immediate and honest, and progress feels earned. Points and badges are the receipts, not the meal.

That is the principle we try to follow in OutBrick. The reward for a board is clearing it; the Journey of 167 brick-built villages is a record of where you have been, not a chore list. If you want to see what a small, honest loop feels like, play a board in your browser. For the progress-and-collection side of the story, read why completing a set feels good.

• Make the core activity worth doing before adding any reward.
• Use rewards to give feedback and mark progress, not to disguise a dull task.
• Prefer collaboration or team competition to everyone-against-everyone rankings.
• Test one element at a time, against a version without it.