How scoring works

Each comment or question in Perusall is automatically scored by the system and rated as below expectations, meets expectations, or exceeds expectations. In your course settings, you decide how many points each of these categories should be worth (by default 0, 1, and 2 points, respectively).

You can also set the following parameters in your course settings:

  • How many comments/questions by each student will be taken into account when computing their grade for the assignment (by default, 7). Perusall will score all of the students' comments and questions and use only the average score of their top-scoring annotations when computing their score.
  • The maximum penalty for failing to evenly distribute annotations evenly throughout the chapter (by default, 10%).
  • The final score scale (by default, 0-3). Students' final scores for each assignment are rescaled to this scale and then rounded to the nearest integer.

If you set a "post-deadline reply window" in your course settings, students can respond to existing comments or questions for full credit for a certain amount of time after the deadline. (This is to encourage students to respond to each other's questions and comments even when the initial question or comment is made close to the deadline.) However, note that students cannot earn more credit after the deadline by entering responses than they have already earned from their work before the deadline. (This check exists to prevent students from simply starting the assignment after the deadline and earning full credit by simply responding to other students' questions or comments.)

If you set a "late annotation period" in your course settings, Perusall will automatically award partial credit for annotations made after the deadline based on when the comment was made. For example, comments made three-quarters through the late annotation period will receive 25% of the credit they would have received if made on time.

Each assignment is graded automatically as students are working. Scores for each assignment will show up in your gradebook, but will not appear to students until you have released the scores.

Scoring example

Suppose a particular assignment requires 4 comments for full credit, and a student submits 5 comments, with scores of 0, 0, 1, 2, and 2. Perusall will consider only the 4 best comments, so only the scores 0, 1, 2, and 2 will be used for computing the score. The average quality score is (0 + 1 + 2 + 2) / 4 = 1.25.

Now the distribution penalty is applied. Suppose that the student's distribution score was 4/5, and the maximum possible distribution penalty is 10% (the Perusall default). This means that the average quality score of 1.25 will be reduced by 1.25 * (1 - 4/5) * (10%) = 0.025, to 1.225.

Finally, this score of 1.225 (out of a maximum of 2, which would be obtained by quality scores of 2 across the board, and no distribution penalty) is rescaled to the final assignment score scale of 0-3 through the calculation 1.225 / 2 * 3 = 1.8375. This is rounded to the nearest integer, so this student would receive a final assignment score of 2.