Back to an early start for the second day of HERDSA, fuelled by awesome porridge from Pulp Friction (I can also recommend their black bean, cocoa and chilli cookies!).
The first talk addresses the issue of whether student feedback resulted in improvements to teaching, presented by Kathy Lefevere, Melbourne University. This presentation focussed on student evaluations of a pilot course within a postgraduate training program, aiming to see whether student feedback on teaching and course quality would result, over time, in am improvement in the course (again, assessed through student evaluations). Only one module demonstrated significant improvement, with an interesting aside that the only module that did change was taught by a single lecturer, who was able to change the course as they needed to; the other courses were taught by a large group of lecturers who were each responsible for a small section of the module, with the result that it was more difficult to make a holistic change to the module.
This brings back some comments from Session 1 yesterday – the best teaching, as measured through a focus on student outcomes, student centred learning and deep learning, occurs when the lecturer has a holistic view of their discipline. However, in modules where there are large collections of somewhat disjoint lecture material, then we don’t even have a holistic view of a module!
Our students have often indicated that they don’t like to have too many lecturers within a course, typically a maximum of two, as they take a while to adjust to the individual lecturer style. I wonder though, whether the impacts of a disjoint curriculum: repeated content, assumptions about content, gaps in presentation, have a bigger impact here, but are harder to recognise as a student.
Kathy reported some interesting results from a statistical analysis, analysing teacher improvement following student feedback over a two year period. Kathy found that when teachers received little or no individual feedback, they only make small improvements in their teaching, while those that received more individual feedback (either positive or negative), then their teaching improved considerably (as assessed through future evaluations). However, it is difficult to say whether this is a valid result, as this part of the study only looks at a two year period, and there were considerable changes to the teaching pattern within this period – the adoption of an intensive teaching style in the second year.
The second talk that I attended in this session was by Pauline Ross, UWS, regarding how students use feedback, and the resultant improvement in learning. Pauline is a very dynamic speaker, so this was a lovely talk to attend, reporting on a ten year multi-school study on student feedback, focussing on two schools in this presentation. The myth that she is debating is that if students receive the right feedback they will improve their learning – this reminds me very much of some of the work presented yesterday, reporting that low GPA students indicate that the teacher is the most important factor in their success. Her literature review identified that some studies did indeed indicate that feedback can result in improved learning, but others did not indicate that, while others indicated that this depends on how sensitive and open the students is. But what makes meaningful feedback? What if it does not correlate with the grade that you have?
Pauline’s methodology for her study analysed feedback over the duration of the study, classifying feedback as identifying an issue (tick or cross, for example), then adding a description of the problem, and then further expanding the feedback by indicating what could be done to improve. (reading here: feed forward learning).
This last category is classified as interventions, and have been established as improving learning (Cohen and Cavalcanti), but depending on several factors, including whether the students use the feedback, for example in developing a second draft of their work. Pauline continued her study with assessing the impact of feed forward feedback, initially not requiring resubmission, but in her second stage of the study, requiring resubmission in some areas, and no resubmission, but linkage between assignments in others. Within the data presented today, resubmission indicated a significant improvement in result (and an increase in student satisfaction).
Other results indicated that during the period of the study where no resubmission was required, the feed forward feedback did not indicate an improvement in the outcomes for future assignments, while when resubmission was indicated, there is an improvement but not a significant improvement. However, the students did appreciate the feedback more – which is one motivation for this work.
What is clear here is that the students appreciate the additional dialogues provided by feed forward feedback, and that there are some, small improvements in learning outcomes – and we need to explore further how we can do this better. Is it a matter of a holistic view of feedback across the program? How can we engage students further to be able to use their feedback more effectively.
An interesting comment in the discussion after the presentation, was that feedback is most useful for high GPA students, while low GPA students do not appear to be able to use feedback effectively. Is this a matter of self regulation? Is it a question of whether our students know how to use feedback, and by providing it we are only completing half of the step of education?