Main Menu

Posted by on Oct 2, 2017

It Isn’t Feedback Without a Shared Goal

We’ve all received useless, incoherent, destructive, or extremely late feedback at one point or another. Sometimes all at once. In my first GE performance review, right after college, I received a “4” on a 1 to 5 scale. When I asked my manager what I could do to achieve a “5” he told me he didn’t “do fives.” Not really feedback I could use…

Mr. D, a Canadian sitcom about an aspiring Physical Education teacher forced to teach Social Studies, satirizes similarly half-hearted (immoral?) ‘feedback’ in its pilot episode:

I think we all recognize the output of a teacher like Mr. D., whether the unexplained “87” on a paper, or “-7” on a multi-step engineering problem worth 25 points, or a paragraph circled in heavy ink with “vague” scribbled next to it. Do these examples count as feedback? Are these examples of good feedback or poor feedback?

The answer depends on your definition of ‘feedback’.

Feedback (or the lack of it) is a major source of worry for most educators. Unlike Mr. D, many teachers routinely burn weekends “grading” (“marking” for those outside of the US) and then feel a sense of failure when their comments produce little effect, or worse, hit the circular file as students toss hardcopies on their way out of the classroom. Feedback creates both deep frustration and feelings of inferiority.

It does, at least, amongst teachers who try. Those happy few who assign authentic work, provide detailed comments, schedule one-on-one conferences, experiment with new technologies like audio notes, all in an effort to create a dialog with their students. But many courses eschew detailed personalized feedback for multiple choice automation. Educators can’t scale personalized feedback in high enrollment or low-cost correspondence-style courses that now make up a majority of the “online” learning happening in higher education. There is little room for personalized feedback in a course with the following weekly structure:

  1. Read chapter(s)
  2. Watch video(s)
  3. Post a discussion topic
  4. Reply to a fellow student’s discussion topic
  5. Take a multiple choice, online quiz

What would an alternative path look like? Three hundred students in a Gen Chem course writing a single 3-page research paper would result in one hundred hours of grading (assuming twenty minutes per), which is probably more instructional time than the professor and TAs would invest in the entire course. And the plagiarism risk? Off the charts. Small group tutoring sessions, oral exams, personalized written feedback, and similar strategies can’t work in these programs, so automated online homework and multiple choice exams are the standard.

Is this a problem though? Is Mr. D. right when he implies that his random, quickie feedback has no real negative impact on student learning?

John Hattie and colleagues have done the work on the factors that contribute to learning, rolling up thousands of studies to conclude that 50% of the variance in student achievement depends on students themselves. Socioeconomics, raw intelligence, personality, home life, day-to-day circumstances etc.

30% of variance, though, depends on teachers. And feedback, according to Hattie, is “the most powerful single moderator that enhances achievement.” This is all achievement, not just learning around higher order skills with which we often associate “feedback.” It isn’t a stretch to argue that, if feedback on all kinds of learning is important, then feedback on authentic, complex work is absolutely crucial.

It’s this kind of work – lengthy, complex projects that involve teams and multiple modalities of communication – that most closely resembles the real world. To paraphrase a cartoon showing two kids chatting as they leave school for summer break, “school is mostly multiple choice tests…but real life is all essay questions.” Employers seem to agree, calling for more graduates with liberal arts skills, at least for jobs with lots of potential for growth.

Effective feedback matters. Student work that requires personalized feedback and instructor-student dialog should be baked into any modern curriculum and a top-five priority for education leaders. So let’s tighten our definition and understanding of feedback so we can start to arrange learning around our most impactful tactic.

The term feedback is used easily in education. In practice, it means “any response to student effort.” Feedback might be automated guidance provided by an adaptive learning platform, written comments on a presentation or paper, a quick conversation during a classroom exercise, or a checkbox in an online gradebook indicating completion of some task. These are all potential examples of effective feedback depending on your definition of the term.

Oddly, at least in the long article linked above, Hattie doesn’t define ‘feedback.’ Grant Wiggins, another giant in the world of teaching effectiveness, writes that “even Hattie has ‘struggled to understand the concept.’” Wiggins goes on to nail a definition of feedback, arguing that feedback isn’t advice or evaluation or guidance. It isn’t “good job” or “maybe try and include more detail on Huck’s interaction with Jim on the raft.”

Wiggins writes, “feedback is goal-referenced; tangible and transparent; actionable; user-friendly (specific and personalized); timely; ongoing; and consistent.”

It’s the “goal-referenced” aspect to feedback that I want to focus on here. I think we all know that feedback should ideally be timely, consistent amongst students, continuous, personalized, and comprehensible to its intended audience (the student). This is why most educators continue to hope artificial intelligence like ETS’ Criterion or Pearson’s Intelligent Essay Assessor will ride in on white horse and save the day (it won’t).

But “goal-referenced”?

Herein lies a fundamental misconception and a major gap impacting the efficacy of any feedback we provide to students:

feedback isn’t feedback unless it is tied to a goal that the student understands and accepts.

That is, feedback will only be useful if it addresses a goal understood and valued by the both feedback creator and feedback receiver. Otherwise it’s noise or worse.

As educators we think in learning outcomes and course objectives, career readiness and lifelong learning. None of these concepts cross your average student’s mind on a regular basis. We have at least two choices, as educators: we can understand current student goals and attempt to align our feedback to those goals. Or we can influence our students’ goals so that they align with the feedback we want to provide.

Let’s consider part one of the first case: understanding current student goals to see if our desired feedback aligns. In subsequent articles we’ll dig further into these assumptions, hoping to create a pragmatic and effective strategy for making feedback as impactful and efficient as possible.

We can talk about course objectives, even place them prominently on our syllabi, but students glaze at these chunks of text, wanting only to know “will this be on the test?” They care about getting a decent grade, building credits towards their degree, and graduating on time. They care about convenience (location and schedule). Students will routinely enroll in a section taught by a monster of a professor (as learned through word of mouth and faculty review sites) if it fits their schedule. So what if the professor is incoherent, insults students in class, grades without rhyme or reason, and assigns a ton of reading? The class is at 2pm M/W!

None of this will be a surprise to teachers with a diverse student population (defining ‘diverse’ to mean students with different ability levels, motivation levels, experience etc). But the implications are serious, and deeply impact how educators must think about feedback if they accept Grant Wiggins’ (and many others) requirement that feedback be tied to agreed upon goals.

For many students, a B- might be a fantastic goal. Rarely so for an educator. We want every student to strive for the ‘top,’ where an ‘A’ grade is necessary but not sufficient. In reality, a majority of students have goals around effort required. They are constantly performing a calculus to analyze hours of studying vs. expected grade.

Another goal for students is to develop a relationship with their professors. And before we discount this goal and it’s implication that becomes judge, jury, and executioner of a perverse beauty contest, think about Gallup’s slam-dunk research that shows graduates are 1.9x more likely to be engaged at work if they had a professor who “cared about [them] as a person” while in university. If you read RMP with an open mind, you’ll find a huge percentage of reviews, positive and negative, refer to the relationship the student felt with their professor. Did they like the professor? Did the professor like them? Was the professor funny and entertaining and fair? The bar is so very low here that it does not take much to make a student feel that their professor “cares” about them; knowing a student’s first name can go a very long way.

The goals we hope students have (earn a good degree, go to class, minimize student debt, get a good job, get involved at school) aren’t ones they actually share on a week-to-week basis. “Getting a good job” is probably a distant fifth or sixth place goal for students, even with the national media focus on employability, STEM etc. These goals are easily put aside in favor of shorter term challenges and opportunities like surviving the Chem final and getting into a party on Thursday night.

What other goals do students have in the real world? Perhaps competition with other students? Keeping their GPA above the requirement to continue financial aid and scholarships? Which of these goals provides any shared mission with our learning outcomes and course objectives?

The only potential overlap between most academic feedback and the student goals we’ve sketched out here is grades. Students care about feedback that directly impacts their grade; but most ‘good’ feedback does NOT focus on grades directly. Helping a student create a more nuanced argument indirectly connects to their grade, but often the work/reward ratio is too much for students to really execute. So most feedback goes streaming into the ether, words like so many lemmings to a tragic end.

In the next article in this series we’ll dig into the implications of recognizing students’ real goals and begin to consider strategies for getting students to align their goals with ours.