Latice 2013 – achievement badges

I’m a little wary of achievement badges. I have mixed feelings about the idea, and so am not going to make a clear statement of support (or not) here, but I thought it might be worth jotting down some of my thoughts as to why am wary.

The TRAKKLA system is a gamification system that provides achievement badges for programming performance, and the study presented at Latice is actually looking at whether badges have an influence on student behaviour, with some statistical significance that some kinds of badges have an influence on student behaviour and engagement. They adopted two groups – both were assessed as to whether they achieved badges, but only one group knew about the badges. There was a roughly similar paper presented at SIGCSE this year that was exploring student perceptions of these kinds of systems, and how they used the achievement badges to form communities and peer assisted learning schemes.

These seem like interesting offshoots – but I wonder whether the results (from the second paper) are more from providing a sense of ownership of their learning to the students, which then prompts the students to take more ownership of other elements of their learning, and therefore creating their own support schemes and increased engagement. I have seen similar results in collaborative learning activities and problem based learning activities – where we give students more individual control of their learning, with some form of motivation whether it be gamification, peer support or intrinsic problem motivation, then they participate more as a community, engage more and take more care with their time management etc.

What concerns me about achievement badge systems, though, is whether it fits too much into specific stereotypes. And also whether students who commence their studies with low confidence and capacity would be turned off by such mechanisms. But I’ll have to do some more reading here. Unfortunately, the majority of studies in these spaces don’t tend to look at these aspects, and when they do try to do a demographic breakdown (which might give us some insight) the number of non-traditional CS students are so low that it is impossible to pull out statistically significant data. Perhaps if we looked at these studies as a whole we might be able to determine something?

In this presentation, they do identify that not all students seemed to be motivated by the badges, and that they didn’t particularly target struggling students. They too are concerned about whether there are potential downsides to this approach. Of course, almost any learning approach will not be attractive to a subset of students – my concern here is that we might be turning off an at-risk sub group. They are looking to do some interesting further work, exploring goal orientation to see whether they can map student goals to see whether different goals correlate with motivation to achieve badges.


One thought on “Latice 2013 – achievement badges

  1. My fundamental problem is that they didn’t isolate the badges for their study. They provided one set of students with a set of goals and associated badges, but didn’t provide the other group with a notion of the goals. The only way a control student would actually ‘develop’ carefulness was to stumble across it accidentally and this, to me, invalidated their claims about the badges. The goals that they set were interesting and I agree with their notion that it changed behaviours but I’m not convinced that there’s enough evidence to say that it was the badges that did it.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s