Assuming these comments are representative of these students’ comments for this assignment (and also that their comments are distributed throughout the entire assignment and submitted on time), they would obtain the following evaluations for their body of comments:
- Allison: Meets expectations
Alison’s comments reveal interpretation of the text and demonstrate her understanding of concepts through analogy and synthesis of multiple concepts. Her responses are thoughtful explanations with substantiated claims and/or concrete examples. She also poses a profound question that goes beyond the material covered in the text. Finally, she applies understanding of graphical representation to explain the relationship between concepts. - Beth: Improvement needed
While Beth asks possibly insightful questions, she does not elaborate on thought process. She demonstrates superficial reading, but no thoughtful reading or interpretation of the text. When responding to other students’ questions, she demonstrates some thought but does not really address the question posed. - Cory: Deficient
Cory’s comments have no real substance and do not demonstrate any thoughtful reading or interpretation of the text. His questions do not explicitly identify points of confusion. Moreover, his comments are not backed up by any reasoning or assumptions.
Perusall uses a machine learning algorithm that is designed to save instructors time by using linguistic features of the text to create a predictive model for the score a human instructor would give. In other words, instead of trying to figure out a set of rules to measure these things, we create a "training set" consisting of a large number of comments along with grades given by multiple expert human graders that are grading according to the rubric, and then create an algorithm that combines the linguistic features to best predict the scores given by the expert human graders. What we found in our validation work is that Perusall agreed with the expert human graders about as often as two humans agreed with each other!
Instructors are always in full control of how students are evaluated; you can always override the score that Perusall suggests for any individual comment or the overall score for any assignment. (To this end, Perusall will not show students their scores until you are ready to release them to students.) To view how Perusall calculated an assignment score, you can click on any score in the Gradebook and change the score if needed.