Debra Boyask
Faculty of Environment & Technology
University of the West of
+44 (0)117 3283211
Abstract
In evaluating progress on change in a faculty-based assessment strategy, students and staff were consulted about what characterised effectiveness in formative feedback processes. In addition to the more mechanical qualities that are often cited as desirable, such as promptness, legibility and clarity, a strong emphasis on two issues was clear. The first was the diversity of students‘ aspirations and confidence, and the ways in which these differences could affect the kind of feedback students desired. The second issue was the bidirectional effects of effective formative feedback processes and strong learning and teaching relationships upon one another. Possible approaches to adapting feedback practice with regard to these emergent issues are discussed.
Keywords: assessment, formative feedback, feedback dialogue, learning relationships, social capital, learner aspiration
Introduction
In 2000
the Faculty of the Built Environment at the University of the West of England
revised its approach to assessment to emphasise the importance of its formative
function. The new strategy was based on principles of designing assessment into
the curriculum in ways that ensured students’ efforts were most conducive to
learning (Gibbs,1999). Across all programmes there was a clearer division
between summative and formative assessment tasks. The summative work was
minimised, with emphasis placed on providing early feedback-yielding formative
work to better foster the development of knowledge and skills needed for
summative tasks.
The
importance of the provision of high quality formative feedback in enhancing
student learning has been well-documented (Brockbank & McGill, 1998; Gibbs,
Simpson & Macdonald, 2003; Nicol & Macfarlane-Dick, 2004; Swing, 2004; Yorke,
2003). Effective feedback processes can enable students to gain a variety of
perspectives on what they are learning. They can provide the scaffolding needed
to develop a deeper understanding of new or difficult concepts with which they
are engaging. Participation in a feedback discourse can yield not only
information to enable learners to better evaluate their own academic
performance, but it encourages them to develop self-critical habits and
scholarly independence.
We
believed our revisions had been effective in increasing the quantity of
formative feedback that students experience, but we needed to improve our
understanding of its quality. Quality of formative feedback had been a common
theme for staff development workshops in the Faculty, with an emphasis on doing
a better job of giving feedback that students could make most effective use of (Brockbank
& McGill, 1998). Despite this, we were aware that our students still considered
there was room for improvement in this area. In reviewing the Faculty’s results
for the 2005 National Student Survey, formative feedback was identified as the
lowest scoring attribute. According to HEFCE’s report on the Survey, this was
consistent with national data.
While the NSS data suggests that students continue to be dissatisfied with feedback, the survey items associated with feedback cannot provide a great deal of depth as to students' perceptions of quality. The three relevant items cover promptness, degree of detail and the ability for feedback to provide clarity, however the items might suggest not only a limited range of criteria, but also a rather limited view of what counts as feedback.
Nicol's Principles of Good Assessment and Feedback Practice (Nicol, 2007) provides a fuller framework for consideration of quality of feedback. They indicate that feedback practices should: clarify learning objectives; encourage a focus on challenging tasks; and facilitate motivation, independence, self evaluation, learner choice and learning dialogues and communities. These goals indicate a more holistic relationship between feedback and learning.
When
considering recent work on formative feedback, we became aware that the language
we had used to talk about feedback was still reflecting the view that it was
something teachers give to students. There is an increasing awareness in
the HE community that effective formative feedback as part of a sound and
holistic approach to learning is a two way process, and not simply something
provided by staff. Nicol and MacFarlane-Dick (2006) suggested that in some
respects formative assessment and feedback have lagged behind general shifts in
thinking about learning and teaching. Discourse around accepted good practice in
HE encompasses learner-centeredness, self-direction and active learning in which
learners construct their own understanding. When it comes to feedback, we often
fall back on a simple transmission model, talking in terms of feedback
provision.
In
carrying out our investigation, we hoped to increase our understanding of
students’ experiences and perspectives on the nature and value of formative
feedback processes and how they engaged with them, as well as simply finding out
about the kinds of formative activities happening in the Faculty. In this
report, the focus is on factors affecting the quality of formative feedback
processes mainly from the students’ point of view, although staff perspectives
have also been considered.
Project objectives
The
objective of this project was to gather information directly from students and
staff across the Faculty via group interviews to determine:
and
Methods
Student
interviews
Eleven
of the Faculty's undergraduate programmes were selected, with the intention of
ensuring a good coverage of the variety of programme types provided. From each
programme, focus groups of between six and eight students were selected to
discuss their thoughts about and experiences of formative feedback. Most groups
were second year students (where this was possible), with one first year and two
third year groups. Most participants were fulltime attending students; two
groups were either part time attending students, or enrolled on a course that
partially involved attending and was partially studied at a distance.
Sampling was not random; participants were asked to volunteer.
Focus
group discussions took between 30 and 50 minutes, and were semi-structured,
facilitated around a set of broad questions designed to determine:
and
The
interviewer for all student groups was the Faculty’s educational developer, and
no participants were in classes taught or assessed by the interviewer. Students
were assured that data from discrete groups would only be accessible to the
interviewer, and only the collated results from all groups would be reported.
Staff interviews
Teaching staff from across the Faculty were asked to participate in a group
discussion during a professional development day. All who attended were
included. There were four groups, each comprising between ten and fifteen
participants.
As for
the student groups interviews, the discussions took between 30 and 50 minutes,
and the question set was designed to determine:
and
The
four staff group discussions were conducted simultaneously, and each group was
facilitated by a peer of the participants who volunteered for the task. All
facilitators were briefed by the Faculty’s educational developer prior to the
discussion session.
Results from student interviews
Student discussions were spread across a term, and a grounded theoretical
approach was taking to analysing data sets to draw out themes. Data sets from
each session were coded according to the following
emergent themes found to relate to students’ perceptions of quality: timing;
consistency of feedback (across modules); clarity and ease of interpretation;
modes of feedback; general vs. individual feedback; diversity of learners’ self
perception and aspirations, and accessibility of and relationships between
students and staff. The data from all student sessions was then grouped
according to theme, and similar items identified in at least three of the eleven
data sets were recorded.
Timing:
·
Sometimes feedback comes too late to be useful.
·
Getting feedback can rely on students' time management skills.
·
Some students feel lots of formative tasks are all due at the same time, making
it difficult to complete formative work for every module.
·
Part-time and block course students have specific difficulties relating to the
timing of feedback.
·
Timing can also create high pressure points for accessing staff for support
(office hours are in more demand at specific points in the term).
Consistency across modules:
·
Most students felt that some modules did a really good job of providing
feedback, whereas others provided far fewer opportunities.
Clarity/ease of interpreting feedback:
·
The usefulness of the feedback depends on how well the student can interpret it,
or even read it (some tutors' handwriting is difficult to read).
·
There is a variety of discourses in marking and commentation. Markers use
different protocols and notations of their own design.
·
Very general comments, such as 'good', 'fine', 'okay', 'think about this' or a
line through something are puzzling. Students want to know why something is good
or fine. They want to know if 'good' means merely adequate, or whether it means
more.
·
Some students would prefer an indicative mark that indicates how well they are
doing. Some staff are not comfortable with supplying indicative marks.
Modes of feedback:
·
Having an opportunity for a dialogue is much better than just being handed
feedback; students need opportunities to get clarification.
·
Some students (especially part time) find it helpful to be able to get feedback
via email. Having to receive it in person can cause problematic time delays.
General versus individual feedback:
·
Some students feel that general feedback has its place, but sometimes the very
general points (not specific to the task) are raised over and over.
·
Some feel general feedback is aimed at weaker students; others feel there is
little that is relevant to them in general feedback sessions.
·
Specific, individual feedback is considered a lot more valuable by most
students.
Diversity of learners’ self-perception and aspirations:
·
Students have different needs in terms of feedback. Some are merely aiming to
pass a module, while others want to excel. Some students need their confidence
built up, while others feel feedback ought to be more critical.
·
Some feel there might be a mismatch between comments and marks. They think
perhaps teachers are afraid of being too critical, when really they would prefer
more bluntness.
Accessibility to and relationships with staff:
·
There is an awareness that different staff members respond differently to
different kinds of approaches. Students with initiative are good at identifying
how best to approach individual staff members, but others can be mystified by
the difference.
·
Some students are more confident than others about taking the initiative in
eliciting feedback. For those who are not as confident, having a sense that
their tutors are interested in them as individuals and in their learning is
important.
·
Some staff make an effort to get to know students, by learning their names,
talking to them during group work and even in the corridor. Students are more
comfortable asking these lecturers for help and feedback.
Results from staff
interviews
Staff discussions happened after the student discussions took place, and results were recorded in a more structured way by the four facilitators. Staff were asked to describe the ways in which they were attempting to incorporate formative feedback processes into their modules. Some approaches were more prescriptive in terms of building feedback opportunities into the course in a formal way. Others simply provided the invitation or suggested ways for students to seek feedback when required. Some approaches indicated a more active role for the teacher in terms of direct interaction with students, or at least with their work. Other approaches were more based around the provision of resources such as models and self-testing tools.
A summary of the
approaches identified
Pre-submission
hand-in:
Consultations:
Resources for
self-evaluation:
Activities in class
that yield feedback:
The variety of formative feedback approaches reflects a range of impacting factors, such as type of module content, class size, level and organisation, as well as the particular views of teaching staff.
In all four groups, issues of accessibility and approachability of staff arose. There were different views about how this should be managed, with some staff feeling the need to be more active in seeking ways in which they could best become accessible to students while others focused on ensuring, simply, that information was available about how to access them.
Beyond access, some staff had things to say about ways in which they attempted to persuade or encourage students to take advantage of opportunities for feedback. Again, there was some variation in as much how active staff felt they should be, with some talking about 'offering' or 'inviting' students to have consultations or comments on work, while others considered there was a need to pursue students more actively, using a variety of means to persuade them to take feedback opportunities and help them develop more confidence and independence as learners.
Discussion
Analysis of data on student and staff experiences of feedback processes yielded
some predictable themes. Not all issues raised by students, however, had been
prioritised as highly by staff in their discussions.
A number of the suggested quality
indicators were ones with which we were already very familiar, and had been
addressing through staff development, such as promptness, legibility and clarity
in communicating feedback. However, the
considerations of diversity of learners’ self-perceptions and aspirations and
the importance of learning and teaching relationships were areas in which we
felt we could have a particular and further impact.
Timeliness
We knew that in order for it to be useful, feedback has to happen at a point
when it can be best made use of (Brockbank & McGill, 1998). By ensuring there
was time for formative feedback to happen well before the summative assessment
processes, the Faculty had established a sound basis to enable this to happen.
However, students raised some additional factors that needed consideration to
further improve the timeliness of feedback processes in some cases.
Timely formative feedback processes rely, to some extent, on students' time
management skills, and perhaps there is more we can do to support the
development of these. Some student respondents suggested that they could be
encouraged to evaluate their own time management skills, and get some support
and planning tools within the context of their programmes if this was required.
This is something we have been able to raise with programme staff as an area to
consider for development.
Clarity and ease of interpretation
The
usefulness of feedback depends on how well it can be interpreted. Students
thought that written feedback was not always completely legible, and efforts
should be taken to ensure students have the opportunity to check if there is
something they can't understand.
Different tutors develop their own discourses and conventions in marking and
commentating work. It's important to be aware that any shorthand notation
teaching staff might develop may need to be demystified for students.
Comments that are very general, such as 'good', 'fine', 'okay' or 'no', might
not be easy for students to interpret. They need to know precisely what is good
or not good, and in what way. They also want to know if good merely means
adequate, or if it is indicative of a higher standard. Some students expressed a
strong preference for indicative marks that would give a quantifiable sense of
how well they were doing, but some staff were uneasy about this apparent focus
on quantitative measurement. However,
comments from staff did acknowledged the importance of giving students
indications of performance against assessment criteria and other benchmarks or
models.
Students need to have a clear understanding of module learning objectives and
the criteria against which they are to be assessed (Brown & Knight, 2004). It
was important to our student respondents that they knew what was expected of
them, and what kinds of standards they need to be working to in order to achieve
the results (marks) they aspired to. As well as encouraging staff to think
carefully about the issues around clarity, building a dialogic process into
programme strategies for formative feedback can facilitate opportunities for
clarification.
For
students to see clear links between general module learning objectives, specific
criteria for formative work and the final assessment criteria, we have
encouraged staff to make the process of how tutors work through evaluating work
and allocating marks more transparent, in an endeavour to further demystify the
process of assessment for them. Specifying and encouraging students to engage
with assessment criteria is a part of this, but dialogic feedback processes also
have an important role in helping students develop their understanding of what
comprises quality in academic work (Nicol & Macfarlane-Dick, 2004).
General versus individual feedback
While they acknowledged that generic feedback to the group can have a useful
place, the feedback that is most valued and likely to be used by students is
personalised
and specific to the individual. General feedback may often be reiterated across
different modules, and some students felt that the generic kinds of issues
covered are more relevant to weaker students; at times they might feel impatient
about the amount of time used on aspects of knowledge and skills they believe
they have already mastered.
The use of generic feedback is often justified by staff by the need to make best
use of limited time, but we have suggested that more consideration is given to
the process used for generic feedback to ensure it takes up a minimum of the
whole group's time. Using online communication tools along with a data bank of
common pointers for types of assessment activities for generic feedback could
save time in class.
Diversity of needs
Aside
from the obvious issues of differences in knowledge and skills amongst a group
of students, their needs for feedback can vary from one another with regard to
differences in confidence, self-perception and aspiration. Those with high
levels of confidence in their ability to learn can find it easier to make use of
critical comments, and in fact some of the more confident students felt
frustrated by what they saw as a lack of rigour, thinking that their tutors were
afraid of being too critical. On the other hand students who doubt their own
abilities may be easily discouraged and sometimes felt tutors had been
excessively tough on them
Students’ different aspirations mean some want to excel while others merely aim
to pass. An individual student’s aspirations might vary for different modules of
learning within the same programme. We
might wish that students were more intrinsically motivated, and of course we can
endeavour to attract their interest in the curriculum for its own sake, but many
students are very functionalist in their approach to their education.
Some students are more confident than others – they feel that they are able
students and they can feel frustrated when they believe that their teachers are
trying to ‘sugar the pill’ too much. They want their tutor to be more critical
and not pull any punches. However, others feel they are easily discouraged; they
are unsure of themselves and a firm critique might make them even less
confident.
Students were very aware of a diversity of aspirations and confidence levels
amongst their peers. They considered that these differences indicate differences
in needs for encouragement and critique. They also knew that some students were
better than others at soliciting feedback from staff.
While engagement at an individual level is not always possible in larger
classes, there are ways we can enable students to provide more information about
their aspirations and feedback needs so these can be addressed in ways that are
more helpful to them.
In a smaller group, this might involve asking them to tell us about what they'd
like from their feedback and what they're hoping to achieve. In larger groups
where it's not easy to have individual discussions, it could be useful to
provide a format for students to indicate the mark they are aiming for and the
level of critique they believe they would find useful.
By providing structure around feedback dialogues, less is left to students’
feedback elicitation skills.
As well
as providing more individualised information, such talking about feedback in
this way can raise students' awareness about the feedback process, its purpose
and how best to make use of it. Sometimes when students say they have had no or
little feedback, they are not always aware of all the things that might feasibly
count as feedback.
Encouraging reflective self-evaluation through feedback dialogue can be useful
in supporting students' development towards becoming increasingly independent as
learners. By involving students in the decision making about forms of feedback,
we can help students develop self efficacy in seeking feedback as well as in
their self-evaluative skills, an important function of effective feedback (Nicol
& Macfarlane-Dick, 2004).
Learning and teaching relationships
Students seemed to be most receptive to formative feedback when they felt that
the person feeding back to them was someone who knew them and who they believed
had an interest in their progress. Both students and teaching staff related ways
in which staff endeavour to engage students in effective learning and teaching
relationships. Students differed in their ability to engage in learning and
teaching relationships, which suggests that the quality of the feedback process
for an individual is likely to be at least partly dependent on factors such as
social confidence and skills and perhaps social capital.
Students valued efforts made by staff to provide opportunities for consultation
through office hours, or through individual email contact, however not all
students had the confidence to take advantage of these opportunities. Some were
also confused by the fact that the ways in which different staff were more
easily contactable varied, while others could easily appreciate these
differences and managed to gain access by working out individual staff members'
preferred approach.
Teaching staff and students talked about strategies teachers use to develop
relationships within which effective feedback processes can happen: they learn
names, use names during group work times or even outside of class, show interest
in students as individuals. We knew that these things were important generally,
but what we’ve learnt is how important they are to students in facilitating
feedback dialogue processes.
All of
these measures contributed to students having a sense that they know their
tutors and that their tutors know them and this made it more possible and
comfortable for less confident students to feel they belonged to their
programme, and to approach a staff member for assistance if it was needed.
While
it seems that effective relationships are beneficial to effectiveness of
feedback processes, there is evidence of a bidirectional relationship between
these two factors; there are ways in which feedback has been used to facilitate
more effective relationships. Some staff take a light-handed approach to
engaging students in feedback dialogues, inviting them to ask for support.
Others are more directing, structuring opportunities for dialogue between
students and staff very specifically into modules.
The issue of relationships has emerged as a significant strand for this project,
and sharing the findings with staff has resulted in valuable discussions around
strategising for using feedback processes to develop stronger learning/teaching
relationships in a more intentional way.
In the
context of increasing numbers and class sizes, we need to think creatively about
ways of getting to know students. Many staff mentioned the value of field trips,
visits and other activities outside of regular classes. Spending time in shared
areas available for students work informally (such as computer rooms, design
studios) was another approach that was found to be helpful.
The
individual nature of learning relationships means that the quality of formative
feedback processes students experience can depend to some extent on their own
ability to forge relationships with their teachers. Students are aware that
teachers respond differently to different kinds of approaches to soliciting
feedback. Some students are more confident than others about initiating
interactions with staff, and some are better than others about working out the
best way to communicate with staff members. It is likely that maturity, social
skills and social capital influence students’ ability to engage with feedback
processes, and this is something we want to investigate further.
Conclusion
The
project provided opportunities for staff to discuss the issues around formative
feedback and possibilities for improvement of its quality, and share effective
practice with peers. While some of the issues were predictable ones, the project
raised our awareness of the importance of learning and teaching relationships
for effective formative feedback, and also the ways in which feedback processes
might be used to strengthen such relationships. We intend to carry out further
work examining the nature of learning and teaching relationships and their
impact on feedback experiences. We were also induced to plan to elicit more
information about students’ specific feedback needs in terms of their
aspirations and what they found most helpful.
References
Brockbank, A. &
McGill,
Brown, S. & Knight, P.
(2004).
Assessing Learners in Higher Education. Oxon, RoutledgeFalmer
Gibbs, G.,
Simpson, C. & Macdonald, R. (2003).
Improving student learning through
changing assessment -- a conceptual
and practical framework, at EARLI Conference, Padova, 2003
Gibbs, G.
(1999). ‘Using assessment strategically to change the way students learn’, In S.
Brown & A. Glasner, (Eds.), Assessment Matters in Higher Education.
SRHE/Open University Press.
The National
Student Survey 2005. (2006).
Available at:
http://www.hefce.ac.uk/Pubs/rdreports/2006/rd22_06/
[Accessed
7th July 2008]
Nicol, D.
(2007). Principles of good assessment and feedback: Theory and practice.
From the REAP International Online Conference on Assessment Design for Learner
Responsibility, 29th-31st
May, 2007. Available at
http://ewds.strath.ac.uk/REAP07,
[Accessed 1st July 2008]
Nicol, D. & Macfarlane-Dick, D. (2004). Rethinking Formative Assessment in
HE: a theoretical model and
seven principles of good feedback practice. Conference presentation at
Enhancing the Student Experience in Scottish Higher Education: Workshop 7,
Nicol, D. & Macfarlane-Dick, D. (2006).
Formative assessment and
self-regulated learning: A model and seven principles of good feedback practice.
Studies in Higher Education, Vol 31
(2), 199-218
Swing, R. (2004). Understanding the
"Economies of Feedback": Balancing Supply and Demand, Keynote to 'Improving
feedback to students' workshop held on 4 June 2004 at the St
http://www.scottishenhancement.ac.uk/uploads%5Cdocuments%5CSwingpaper-revised.pdf
[Accessed 6th October
2008]
Yorke, M.
(2003). Formative assessment in higher education: Moves towards theory and the
enhancement of pedagogic practice.
Higher Education 45, 477--501.