Monday, May 30, 2016

Specifications Grading in Biological Anthropology: Part 1 - Learning Goals

This is the first in a series of posts about creating, implementing, and modifying a specifications grading system. You can find them all by clicking on the "specifications grading" label below.

Introduction to Biological Anthropology is one of my biggest classes. About 10% of students here at UMM take the class to fulfill their science GenEd. It's one of my favorite classes to teach, but for years I've tinkered with it, trying to improve student outcomes. I've flipped the class, given students exam questions ahead of time, and instituted a specifications grading system. The result: a noticeable improvement in student learning and student grades, although the failure rate continues to be high.

The specifications grading system caused a bit of a stir on campus when I rolled it out in Spring 2015. Students expressed their concerns to other faculty at the beginning of the semester, only to come back at the end to tell them how much they loved the grading system and would like to see it implemented in more classes. Unsurprisingly, a number of those faculty members then asked me "what do you mean by specifications grading?"

So, without further ado...

What do I mean by specifications grading?

Specifications grading systems are based on a book by Linda Nilson. At its heart, specifications grading is a mastery system: students show mastery of a concept or skill in order to pass an assessment. The more skills or concepts are mastered, the more assessments passed, the higher grade the student earns. It reminds me of my high school chemistry class: there were a number of self-paced units, each with associated lab work, homework, and an exam. You had to pass the exam with at least a C to move on to the next unit. Your final grade was entirely dependent on how many exams/units you passed, not on your percentage grade on the exam. If you only did a couple units, you'd fail. If you did ten or more, you'd get an A. (Incidentally, that analogy worked well for explaining the system to my students, even though I wasn't using a self-paced unit structure.)

In a standard, non-specifications grading system, a student passes the class by averaging at least a 60% in total points. If a class has any "easy points", such as points for attendance, this means a student could pass the class without truly mastering any of the core concepts or skills. Sure, if they do that for a lot of classes they'll get suspended, but we all want our students to leave our classroom with a basic understanding of our field, right?

Specifications grading makes it very clear, to both students and professor, what constitutes a passing-level understanding of the class material, what constitutes a C-level understanding, what constitutes a B-level understanding, etc., etc., and then ensures students reach that level of understanding in order to earn that grade. This increases both academic rigor and accountability. It also ensures that any student moving on to the next class in a sequence has the necessary skills to do well.
from Dee Fink https://www.deefinkandassociates.com/GuidetoCourseDesignAug05.pdf

I love it.

How did I develop my course?

In creating the new BioAnth course, I used a combination of Nilson's book and the work of Dee Fink on designing courses for significant learning. (I highly recommend following that link to an excellent PDF guide to course development.) The two approaches work very well together. I began, as Fink suggests, thinking about my learning goals. He recommends six different categories of learning goals, as seen in the figure above. Mine were as follows:

Learning Goals for Intro to Biological Anthropology:

Long-term:
  • A year or more after this class is over, I hope that all students will be able to critically evaluate claims about human nature, understanding that our genes are expressed through our social and physical environment, and recognizing that much of what we consider “inherent” to our biology is actually a reflection of our culture, because we have evolved to be flexible rather than to follow rigid rules of behavior. (In other words, when you sit down to Thanksgiving dinner with your least-favorite cousin, you should be able to intelligently debunk the latest media claims about race, gender, or overall human nature.)
Foundational Knowledge: Students should be able to explain in their own words and give examples of the following key concepts:
  • Scientific method in the historical sciences (fields such as anthropology, evolutionary biology, ecology, and astronomy) D
  • Culture D
  • Evolutionary theory D
  • The influence of both environment and genetics (humans as "naturenurtural") C
  • The human adaptation/niche B
Application: Students should be able to:
  • work "hands-on" with materials that tell us about human adaptation and evolution (fossils, biometric data, etc.) D
  • create and evaluate scientific arguments and evidence C
  • critically evaluate scientific arguments about fundamental human nature (based on race, gender, our hunter-gatherer past, etc.) B
  • find and be able to summarize multiple perspectives on key topics like race, gender, and culture B
Human Dimension Over the course of the semester, students should:
  • identify and list what about themselves is shared with all people around the world, and what is unique to their cultural perspective. C
  • develop their feelings of empathy for, interest in, and connection to people around the world. C
  • recognize where their own cultural filters are affecting their interpretation of the world around them. B
  • develop their skepticism for claims that biologically-based differences between people of different races, genders, etc., override culturally and contextually determined differences. A
Integration: Students should take from this class a set of ideas and skills that can be applied outside of the course, including:
  • Students should be able to apply evolutionary theory to all the topics in this course, as well as topics in other relevant courses (such as biology, philosophy, pseudo-scientific arguments in political science and economics, etc.) C
  • Discussions of human diversity and evolution that take place in other classes (history, political science, psychology, etc.) should be informed by the anthropological understanding of human differences in a broader context. B
  • Science never takes place in a cultural vacuum. Students should recognize that the cultural context of scientific endeavors affects interpretations, even on topics that are less emotionally charged than defining ourselves as a species. A
 

After creating these learning goals, I went back and added grade levels to each (shown in red above). In order to get a D in the class, students need to reach only those learning goals that say "D". In order to get a B in the class, students need to reach those learning goals that say "B", but also those that say "C" and "D". Those goals that say "A" need only be mastered by students who want the top grade (but those students will also have to master the "D", "C", and "B" goals).

Some things to consider when creating learning goals for specifications grading systems:

1) As Nilson discusses in her book, the learning goals must be testable and the way they will be evaluated should be clearly stated. Note that under "Foundational Knowledge" I don't say "students should know...", I say "students should be able to explain in their own words and give examples of...". No surprise, on exams I ask students to define and give examples of these concepts.

2) The D-level or passing learning goals are a mixture of truly fundamental concepts for the whole field (like evolution or culture) and those skills/concepts that are most easily grasped. There are some skills/concepts that I would love to require of all students (for example, the ability to critically evaluate claims about biological determinism), but my experience with this class in previous iterations convinces me that requiring all students to master a skill of that complexity would, essentially, lead to a 90% failure rate.

3) But, but...but all of my students should master all of the material! Yes, yes, I know you feel that way, but you have to let it go. It was truly painful for me to admit that not all of my students would be able to, say, develop an understanding of the influence of cultural context on science and therefore not even require it for students below the A level. The truth is, students with lower grades weren't mastering this material anyway. Because of the way I structure my class, all students get the same readings and lectures, so they're still exposed to the concepts. At least this way I know that a student passing my class really understands the basic concept of evolution, even if they haven't mastered the more complex arguments, while before they may have passed through without understanding either.


In my next post, I'll talk about how I structured the grading system and some of the changes I'm making this coming year to avoid the (inevitable) problems that come with doing anything new.

Sunday, May 22, 2016

Flipped Classrooms: Student Evaluations and Cui Bono?

An article by Kevin Werbach, "Flip-Flop: The Realities of Blended Teaching", came across my twitter feed. Now that I'm back from sabbatical, I've been meaning to write a series of posts on my flipped Biological Anthropology class. I wrote about my first attempts here. Last year, I used a specifications grading system (based on the Linda Nilson book), which is essentially a mastery grading scheme.

Like Werbach, I found my student evaluations went down after flipping the classroom, despite overwhelmingly positive unsolicited feedback from students. I haven't pulled out the evaluations and looked at the exact distribution of scores, but I can guess what I would find: a bimodal distribution in which the average score for the higher curve went up, but the average score for the lower curve dropped dramatically.

There were two things I wanted to accomplish by flipping the classroom and using specifications grading: 1) to improve overall comprehension of major concepts in Biological Anthropology; and 2) decrease the 15% failure rate for the class. The new system definitely accomplished goal #1; the students shows far greater mastery of the concepts, both during mid-semester evaluations and during the cumulative final. This translated into higher grades, as well, not because I changed my standards but because the students earned them.

Previous classes had a grade distribution more or less like this:
15% As, 30% Bs, 25% Cs, 15% Ds, 15% Fs.

This past year, the grade distribution looked more like:
25% As, 35% Bs, 25% Cs, 0 Ds, 15% Fs.

Students who had struggled to get Cs and Ds were earning Bs and Cs. It was glorious. But...that 15% of the class still failed. And to the extent that I got complaints from students about the grading system or flipped classroom, it was from the students getting Fs (and to some extent Cs). Almost every complaint could be boiled down to a variation on the following: "I've gotten Ds in my other classes without having to show up, read, finish assignments, or study. I'm frustrated that you require me to learn the basic material in order to pass the class." To be honest, though, I didn't get many complaints, because those F students were the ones who hardly ever showed up to class.

I don't want to mischaracterize my students. They are the opposite of entitled. I find them responsible, engaged, and mature. They face a number of barriers to success, including personal illness, family obligations, long work hours, disabilities, or under-preparedness for college. And, yes, some of them are just not doing the work. (I asked one young lady where her missing lab partner could be found. She said, "Oh, he just plays video games and gets high all day." OK. Good to know.) The systemic barriers many students face can be mitigated by specifications grading, but they can't be eliminated entirely.*

I discussed the problem of active learning and student motivation in another post, where I suggested faculty who rely on student evaluations for promotion/tenure/better jobs might want to avoid flipped classrooms because unmotivated students will ding your evaluations for forcing them to engage in the classroom. After my Biological Anthropology experience, I stand by that. I'm convinced that at least 75% of my students had a better educational experience as a result of the flipped classroom and specifications grading system: they learned more, were more engaged, and were more interested in the subject. But that remaining 15-25% were not reachable, not because I didn't try, but because they either faced systemic barriers that prevented success, or because they didn't want to be reached.

I've been forced to conclude that flipped classrooms are an advantage to higher-achieving students and that they can greatly benefit middle-level students who struggle with concepts but are willing and able to engage in the classroom. As a professor, however, it's often the "ones who got away" who haunt me. I originally flipped my classroom with those students in mind. Ironically, while the experience didn't help them, it helped me learn to let go, to realize I have done everything I can to provide students opportunities for success. At some point, I've reached the end of what I can do within the classroom.
 ______________
* This is a topic for another blog post, and because we're a small school I don't want to go into details, but I did look carefully at the grades of non-traditional students, students of color, students whose first language was not English, student parents, and students with registered disabilities. None of the students who fell into one or more of those categories failed the class, and their grade improvement was similar to that of the class as a whole.