The Perils of Gamification

When my kids were younger, we got them a few "math learning games". Over the years I've wondered why we don't do more of that. Today I have a simple answer to that question: They don't work. Stanley Erlwanger, a math education research published a paper studying how a student named Benny had progressed through a math learning program, gaining levels faster than anyone else, but had failed to learn any math whatsoever.

What the designers of the IPI program had intended was that gaming the game required mastering the mathematics. Unfortunately, there is no way to prevent people, particularly smart ones, from coming up with alternative systems.

In Benny’s case, this involved developing a complete set of rules for adding, subtracting, multiplying and dividing fractions. Though his rules were symbolic manipulation procedures that made no sense mathematically, they enabled him to move through the sheets faster than everyone else in his cohort group, scoring 80% or better at each stage.

Whenever his rules yielded wrong answers, he simply adapted them to fit the new information he had acquired.

There's something familiar about this description. A high school teacher once told me about a student (in high school) who didn't know how to read, and his attempts to teach the student. This student had learned a bunch of tricks to navigate the world of letters and words, none of which actually constituted reading. But getting the student to walk away from those tricks and grind through phonetics and all the crazy rules that most of us learned in first grade was very difficult.

This is recounted by Math Guy Keith Devlin in a series called "How to Design Video Games That Support Good Math Learning".

The issue, it would seem, isn't limited to video games though. Here's a video showing a similar problem to Benny:




Devlin mentions that the same issues show up in gamification. It's hard to prove that getting good at a game demonstrates skill in anything at all other than playing that game.

Gamification, to my mind, has other problems as well, the chief of which is that it relies on extrinsic motivation rather than intrinsic.