Peer Review

Implementation and Timing: The WHEN, WHERE, and HOW

Peer reviews can be implemented at any point during the course, however it is highly recommended to introduce peer reviews at the beginning.  Early exposure allows opportunities for students to practice the process, improve their ability to give feedback, and experience academic success. Peer reviews are versatile in implementation and can be used in online or in-person courses, both in lecture or laboratory classes.

Initial Planning Considerations

• Consider the course and module learning objectives.
• Think about the skills students should get out of a peer review activity in alignment with the course/module learning objectives.
• Pick one or two areas where students could provide feedback to their peers (general writing, clarity of writing, accuracy of code, design iteration, presentation skills, etc.).
• Consider if peer review will be a graded activity. For example, will students turn in drafts/images with peer comments for points? Will students reflect on their work and what they learned after completing peer reviews?
• If it is a graded peer review assignment, build rubrics that assess the identified areas (see Creating Rubrics for Peer Reviews for additional information regarding rubrics)
• Use a tool such as CATME to assist in forming groups for peer review activities. CATME can also be set so students can provide feedback to peer group members.
• If working with teams of students, you might consider giving them some time to create rapport among group members before asking them to review their teammates.

Implementation Considerations

• Take some class time to teach and model peer review.  Explain the focus and purpose of the peer review activity. This shows students you value this experience and they might value it as a result.
• Discuss/demonstrate how students can provide feedback that is both positive and constructive. For example, provide a few sample feedback statements using the rubric. Make some general statements and some specific statements and discuss why using rubric language and specificity benefits students.
• Introduce the rubric to students by conducting a short mini-lesson or presentation, if it is a graded peer review activity.
  • Allow students to anonymously score the presentation using the rubric. Create a Google form or live poll to see the results of their assessments in real-time.
  • Refer back to the scores from students and talk through each score. Use the rubric vocabulary so that students can hear the thought process and reasoning behind each score.

Creating Rubrics for Peer Reviews

Practitioners and researchers agree that the most effective peer reviews are accompanied by a detailed rubric that students can refer to as they prepare to review each other’s work [3]. Instructors will need to invest time upfront to create detailed rubrics. However, the time needed to guide students afterward will be minimal as they will have a standard of good performance to compare their classmates’ work to, as well as their own. This also means that students will be able to practice self-evaluation as they go through the course and be able to improve their performance from it [4].  

The best peer review rubrics use clear and simple language to detail the meaning behind scores. When giving students a space for comments about their classmate’s performance, include examples of acceptable comments. Some researchers have recommended taking classroom time to elicit the students’ participation in creating the rubric in a short workshop, thus finding areas for growth and improvement that the students are interested in [5, 6]. The following is an example of how one might “crowd-source” a rubric for a group project with the help of students:

• Begin with an open-ended conversation: lead a discussion with the class as a whole, asking student teams to develop and share a list of things they dislike about group work. Present them with an existing rubric to spark the conversation, while guiding them to the learning objectives.
• Reach a consensus: identify the problems or areas for growth the students want to tackle and invite them to help develop criteria that they feel comfortable with to evaluate their work as well as the work of others.
• Construct the rubric: finalize the document using the developed criteria and make it available for students to access (e.g., post it online or send it through email).
• Test and improve: Once the students have used the rubric to review their peers, discuss the process with the class to uncover challenges the students found while using it. If any changes are needed, discuss possible improvements and update the rubric for the next assignment.

Eliciting the help of the learners to develop the rubric can easily be done for activities other than group projects, such as oral presentations or papers. Making the students feel included in the process will help them engage and reflect on the reviews they receive [5].

Content-based Peer Review Ideas 

When developing a rubric, it is necessary to determine the skill or topic to be assessed.  For example, if students are asked to give feedback on a prototype of an autonomous aerial vehicle, the rubric criteria should measure areas such as, and not limited to: design, functionality, understanding of the science behind flight. Designing a rubric for the written report or proposal of the vehicle would not apply at this time. In some cases, students can be provided with feedback sentence stems to help them begin the process.

• Written assignments: Students can be assigned partners to review their written work, such as essays or reports, and give each other constructive feedback on their arguments, critical thinking, supporting evidence, writing style, etc.  Consider dividing the assignment into stages so that they receive feedback at every stage [7]. Some examples of peer-reviewing a written assignment are included below. This is also beneficial in understanding what high-quality feedback looks like.
  • Something that I loved in this draft and had not considered in my writing is…
  • I would have enjoyed knowing more about…
  • A detail that I found missing is…
  • A minor tweak that could improve the quality of this draft is…
• Oral presentations: Peer reviews can be implemented to help students improve their communication and presentation skills. Students can be asked to give peer reviews based on criteria related to the learning objectives of the activity, such as content delivery [7].
• Problem-Based Learning Teams or Group Projects: Periodical peer reviews can be established to help team members identify ways in which they can improve their teamwork skills, help maintain an equal division of labor, and encourage all team members to actively participate in the project [8].

A peer review rubric example

The following is a modified example of a very general rubric for peer review. The areas for review were Involvement in Critical Analysis, Participation in team discussions, and Etiquette in Dialogue with Peers. In this example rubric the students use a five-point Likert’s scale to rate their classmates’ performance in these areas, where each score is explained with examples of the type of performance the student should exemplify. Follow-up questions can also be used in addition to the rubric to expand on their peer’s performance or provide a space for the student to give helpful constructive comments to their classmate [6].

Criteria	Unsatisfactory = 1-2	Limited = 3	Proficient = 4	Exemplary = 5
Involvement in Critical Analysis	Classmate shows little or no evidence of completing or understanding assignments. Their contributions are largely personal opinions (e.g., “I agree” or “Great idea”) without support of concepts from the theory, outside resources, relevant research, or specific real-life applications.	The student’s contributions repeat and summarize basic, correct information, but do not link readings to outside references, relevant research, or specific real-life applications and do not consider alternative perspectives or connections between ideas.	The student displays an understanding of the required readings and underlying theoretical concepts, including the correct use of terminology.	Classmate displays an excellent understanding of the required readings and underlying concepts, including correct use of terminology. Their contributions integrate an outside resource, relevant research, or specific real-life application (work experience, prior coursework, etc.) to support important points.
Participation in team discussions	Classmates do not contribute to team discussions or conversations around assignments.	The student sometimes contributes to ongoing conversations with affirming statements or references to relevant research and asks related questions	The student constantly contributes to the class’s ongoing conversations about assignments, evidenced by affirming statements or references to relevant research. Constantly asks related questions, or makes an oppositional statement supported by any personal experience or related research.	The student actively stimulates and sustains further discussion by building on peers’ responses, including building a focused argument around a specific issue, asking a new related question, or making an oppositional statement supported by personal experience or related research.
Etiquette in Dialogue with Peers	Written or verbal reviews show disrespect for the viewpoints of others.	Some of the student’s interactions with peers show respect and interest in the viewpoints of others.	Most of the student’s written or verbal interactions with peers show respect and interest in the viewpoints of others.	All written and verbal interactions with classmates show utmost respect and sensitivity to the viewpoints of others.