Setting problems is hard!

Creating problems that are both engaging and useful is difficult. There has been a significant call from several areas for authentic problems – but what makes an authentic problem? I have had an interest in authentic problems for a while (although more from the side of learning authentic problem solving processes rather than setting authentic problems) and from my reading, the emphasis is on selecting problems that: (1) are naturally interesting and understandable (at a high level) by the students, (2) would be similar to problems that they would be attempting as professionals, (3) have an appropriate degree of openness in order to prompt discussion and questioning, and (4) the solution demonstrates all of the characteristics and relevant aspects of the topic being addressed.

The skill in problem crafting is in meeting all of these. However, I think we have a bad habit as University educators of selecting examples and problems that meet criteria (4) regardless of the others – they are beautiful to us, the solution may have an elegance which an expert can appreciate, or a simplicity that we feel should be appreciated. However, if the students do not understand the problem or are not engaged by it, then it is doubtful that they will achieve deep learning, that curiousity that is needed to push the solution further, to think, to question, and understand rather than repeat.

It is not easy, of course, to meet all of these criteria. It takes a lot of experience, practice and rethinking of problems to ensure that they are meeting all of these, perhaps the most difficult is criteria (3) – ensuring that problems have enough openness, or opportunity for questioning, that the students ask the right questions, head in the right direction and do not get too overwhelmed by many possible directions. The “appropriate” degree of openness is also something that would vary depending on your student cohort – are they first year students, just being exposed to either the discipline content, or problem solving strategy, or are they postgraduate, which considerable experience in both? It takes more detailed knowledge of your cohort as well – as we all know, some years, your student cohort will struggle with one concept, while in another year, somehow it will manage to “click” and be something that they sail through. Where is your particular cohort sitting at _this_ point in time?

I guess one of the most difficult, and concerning, aspects of Problem Based Learning – where the degree of openness in a problem and the dependence on problem quality is greater  – is what to do when you have a problem that does not meet the above. As problem crafting is such a difficult skill, and one that requires time and expertise to master, it is likely that inexperienced facilitators will also have to deal with problems that are poorly crafted. This is what makes PBL so challenging for many educators, and – we should accept – so risky.

What happens if you have a poorly crafted problem? I think, for me, the greatest risk factor is the degree of openness. If you misjudge the others, things will be more difficult, but I think that they be handled more easily. If a problem is not that interesting to the students, or does not demonstrate professional behaviours, then the students will not be as easily engaged – but we can increase engagement with other techniques, so if necessary, we can rely more heavily on collaborative learning, or other activities to help keep the students interested. Or we can try to introduce additional examples that hep the students see the relevance of what they are learning – a bit of a bandaid, but not a total loss. If the problem does not encompass all of the concepts that we wish to address, that is ok too – we can always tackle those another day.

But if the degree of openness is wrong, we can have a potential disaster. I have seen cases before where the problem to be solved has been overspecified – not so much of a disaster you might think, but if we channel or scaffold our students too much then we can  end up with them essentially learning nothing. They have simply followed the steps that we have requested, with little motivation to think about why they are doing what they are doing. This, to me, is a waste of time.

What if we have an underspecified problem? I have seen cases where groups of students are effectively paralysed because they are unsure which of several directions to head. And so they go nowhere. They are so confused as to what is expected of them, and what the objectives actually are that they make no progress at all. Even if this is explicitly addressed and clarified, it is difficult for students to make progress, as they don’t seem to be able to lose that initial sense of confusion and lack of confidence and move through to working on the problem.

A key realisation here, and one that I had seen during my visit to Republic Polytechnic, is that good facilitation skills can deal with the issues in problem crafting identified above. A good facilitator, knowledgable in their field, is able to recognise these kinds of issues before they become insurmountable and to identify strategies for addressing issues as they arise. At RP the focus initially is on developing facilitation skills – using problems developed, tested and refined by experienced facilitators. This approach enables a separation of problem crafting and facilitation that supports their independent development and reduces risk.

Some disciplines have achieved the creation of libraries of authentic problems – because they take so much work to prepare, it makes sense for us to share. The PBL community has been particularly successful here in establishing resources for the community to share. There has been some great work on PBL in CS, namely Judy Kay et al’s work in Sydney, including their collection of PBL resources gathered to help others.


2 thoughts on “Setting problems is hard!

  1. Sorry, GUI error posted that too soon. This is something that regularly causes problems when people first start using Puzzle Based Learning, too. They find problems that look good to them, but forget that you need to design in pathways that reflect multiple approaches, support partial solutions and set the frustration bar sufficiently to allow for perseverance. The degree of openness is crucial.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s