Monday, June 6, 2016

Specifications Grading: Part 2 - Assessment

This is the second in a series of posts about creating, implementing, and modifying a specifications grading system. You can find them all by clicking on the "specifications grading" label below. Or you can follow these links: Part 1

In my last post, I laid out my class goals for Intro to Biological Anthropology and assigned a "grade level" to each. The next step is determining how to assess whether or not the students have achieved each goal. Again, I lean heavily on Linda Nilson's book and the work of Dee Fink on designing courses for significant learning.

Step one: matching assessments with goals

I literally just go through my learning goals and decide what type of assessment I will use for each one. I won't bore you with all the details, but here are some examples with the learning goal in black, the grade level in red, and the assessments in blue:

Foundational Knowledge: Students should be able to explain in their own words and give examples of the following key concepts:
  • Evolutionary theory D 
    • Multiple choice questions: ex., which of these is an example of natural selection?
    • Short essays: ex., define evolution and explain why it is not the same as natural selection 
    • In-class activities: Four forces of evolution; Pedigree analysis; Creating cladograms
Application: Students should be able to:
  • work "hands-on" with materials that tell us about human adaptation and evolution (fossils, biometric data, etc.) D
    • Lab write-ups and worksheets on fossils, primate skulls, and human osteology
    • Lab practicums
  • critically evaluate scientific arguments about fundamental human nature (based on race, gender, our hunter-gatherer past, etc.) B
    • Longer essay: ex., if your local newspaper published an opinion column that claimed males are better at math than females, how would you critique this from an anthropological perspective? Make sure you use actual data and examples from class or readings.
    •  In-class activities: Data on difference; Clines and population variation; What does it mean to be human?; Race is in the cards; Discussion: Race, housing, and systemic inequality; Discussion: sex, gender, and violence
    • Extra-credit opportunities: Six words project; #BlackLivesMatter reading and summary; campus speakers as available 

You get the idea. A few things to point out:
  • Not all assessments need to be exams or even graded. I count extra-credit opportunities or in-class assignments where the students are engaged with the topic and write a short reflection. Reading the reflections helps me revisit concepts that didn't get across the first time.
  • Higher grade-level goals have more complicated assessments, such as essays vs. multiple-choice questions. YMMV.
  • The more important the issue (from my perspective) the more ways it gets assessed in the class. Note how many things are listed under "critically evaluate scientific arguments about fundamental human nature", and I left off a bunch more. Even though only B students are asked to master that goal, all students are exposed to it again and again.
I had already taught this class several times before implementing the specifications grading system, so I had a whole list of activities, labs, and exams that already worked for me. Mapping them onto the learning goals, like above, helped me see where I had blanks and forced me to dump/modify some things that weren't serving a purpose.

Step two: develop a grading system (KISS)

Once I knew how I wanted to assess whether or not students had mastered a particular learning goal, I had to structure those assessments into a coherent grading system. Here's where I part ways a bit with Nilson's book. Something I wish I had known before I started: KISS

I'll dedicate another blog post to how badly structured my specifications grading system was the first time I tried it. Here's the new, improved structure:

To earn a D, a student must show mastery of the concepts in all "Comprehension Checks". These are short (one-paragraph) essays asking for definitions and examples -- or applications -- of basic class concepts. They are assigned every other week. 

To earn a C, a student must pass the requirements for a D, as well as answer 80% of questions correctly on all five "Basic Knowledge Checks". These are computer-generated multiple-choice exams, for which they get the questions ahead of time. (That doesn't help their grades as much as you might think.)

To earn a B, a student must pass the requirements for a D and a C, as well as achieve an 80% on three midterms. The midterms are also computer-generated exams for which they have the questions ahead of time, but they include essay questions and require skill application (like correctly calculating and interpreting Hardy-Weinberg, or pedigree analysis, or reading a cline map, etc.).

To earn an A, a student must pass the requirements for a D, C, and B, as well as show mastery of the concepts in all "Mastery Essays". Like Comprehension Checks, Mastery Essays are relatively short (2-3 paragraph) essays assigned every other week. Unlike Comprehension Checks, Mastery Essays move beyond defining or giving examples of class concepts and instead require students to grapple with more complex topics (for example: "In your class textbook, Jonathan Marks claims there is no such thing as a value-neutral primate taxonomy. What does he mean by that? Do you agree or not? Why?").

In addition, 20% of the grade for all students is based on three lab practicums which are graded on a simple percentage scale. That is, if they correctly identify 80% of the bones on the human osteology exam, they get 80% of the total points. Last year, I tried to use a specifications grading system for the lab, as well, but it was a logistic nightmare. I might revisit it as a possibility in the future.

My goal is to make the class impossible to pass unless a student has mastered the basic concepts. I want to encourage students, however, to aim for a B over a C, so the requirements for a B are not really that much greater than the requirements for a C. Before the specifications grading system, I tried to pitch exams so the average grade was about 80%. Under this system, I'd like more students to fall on the B side of that line than the C, assuming they're in that middle area of what once was the standard bell curve.

To get an A, though, requires a more complex level of engagement with the material. The "mastery essays" pull from a variety of sources and themes within the class, and they really do separate the best students from those who are struggling with the material.

Next time, I'll talk about troubleshooting the grading system, re-dos, and the grading load.

No comments:

Post a Comment