Peer Review

Overview and Introduction: The WHAT and WHO

Peer reviews are collaborative opportunities for students to work in pairs or small groups to assess the work of their peers and provide quality feedback on the work [1]. Peer reviews can be used as formative assessments, or assessments that do not have grades assigned to them and instead used as frequent opportunities to provide feedback so that students can target specific areas that need refining [2]. Peer review activities differ from “peer assessments” in both their execution and their final value to the students. In a peer review activity students provide feedback on their classmates’ work, pointing out strengths and suggesting possible areas for improvements, while also receiving feedback themselves. In a peer assessment activity, students rate or score each other on their performance but only for the purpose of grading. Though both may use similar methods, your students will benefit much more from receiving feedback through constructive peer reviews than a grade from a peer assessment.

Whether written or spoken, well thought-out peer reviews can become an essential part of any course assessment experience. Peer reviews can be applied in a variety of courses and be scaled up or down depending on section size. They are useful for students at all levels of undergraduate and graduate education.

Implementation and Timing: The WHEN, WHERE, and HOW

Peer reviews can be implemented at any point during the course, however it is highly recommended to introduce peer reviews at the beginning. Early exposure allows opportunities for students to practice the process, improve their ability to give feedback, and experience academic success. Peer reviews are versatile in implementation and can be used in online or in-person courses, both in lecture or laboratory classes.

Initial Planning Considerations

- 1. Consider the course and module learning objectives.
  2. Think about the skills students should get out of a peer review activity in alignment with the course/module learning objectives.
  3. Pick one or two areas where students could provide feedback to their peers (general writing, clarity of writing, accuracy of code, design iteration, presentation skills, etc.).
  4. Consider if peer review will be a graded activity. For example, will students turn in drafts/images with peer comments for points? Will students reflect on their own work and what they learned after completing peer reviews?
  5. If it is a graded peer review assignment, build rubrics that assess the identified areas (see Creating rubrics for Peer Reviews for additional information regarding rubrics)
  6. Use a tool such as CATME to assist in forming groups for peer review activities. CATME can also be set so students can provide feedback to peer group members.
  7. If working with teams of students, you might consider giving them some time to create rapport among group members before asking them to review their teammates.

Implementation Considerations

- - Take some class time to teach and model peer review. Explain the focus and purpose of the peer review activity. This shows students you value this experience and they might value it as a result.
  - Discuss/demonstrate how students can provide feedback that is both positive and constructive. For example, provide a few sample feedback statements using the rubric. Make some of them general statements and some specific statements and discuss why using rubric language and specificity benefit students.
  - Introduce the rubric to students by conducting a short mini lesson or presentation, if it is a graded peer review activity.
    - - Allow students to anonymously score the presentation using the rubric. Create a google form or live poll to see results of their assessments in real time.
        
        Refer back to the scores from students and talk through each score. Use the rubric vocabulary so that students can hear the thought process and reasoning behind each score.
  - Know that things will be messy but the benefits far outweigh the initial disorderliness which will improve with iteration(s).
  - For large courses, consider utilizing UGTAs and TAs to support peer review student groups.
  - For online courses, consider recording a video at the Hub studio to model a peer review.
  - Canvas discussion boards could also be used for peer reviews. See these other remote peer review strategies.

Creating Rubrics for Peer Reviews

Practitioners and researchers agree that the most effective peer reviews are accompanied by a detailed rubric that students can refer to as they prepare to review each other’s work [3]. Instructors will need to invest time upfront to create detailed rubrics. However, the time needed to guide students afterwards will be minimal as they will have a standard of good performance to compare their classmates’ work to, as well as their own. This also means that students will be able to practice self-evaluation as they go through the course, and be able to improve their own performance from it [4].

The best peer review rubrics use clear and simple language to detail the meaning behind scores. When giving students a space for comments about their classmate’s performance, include examples of acceptable comments. In fact, some researchers have recommended taking classroom time to elicit the students’ participation in creating the rubric in a short workshop, thus finding areas for growth and improvement that the students are interested in [5, 6]. The following is an example of how one might “crowd-source” a rubric for a group project with the help of students:

- - Begin with an open-ended conversation: lead a discussion with the class as a whole, asking student teams to develop and share a list of things they dislike about group work. Present them with an existing rubric to spark the conversation, while guiding them to the learning objectives.
  - Reach a consensus: identify the problems or areas for growth the students want to tackle, and invite them to help develop criteria that they feel comfortable with to evaluate their own work as well as the work of others.
  - Construct the rubric: finalize the document using the developed criteria and make it available for students to access (e.g., post it online or send it through email).
  - Test and improve: Once the students have used the rubric to review their peers, discuss the process with the class to uncover challenges the students found while using it. If any changes are needed, discuss possible improvements and update the rubric for the next assignment.

Eliciting the help of the learners to develop the rubric can easily be done for activities other than group projects, such as oral presentations or papers. Making the students feel included in the process will help them engage and reflect on the reviews they receive [5].

Content-based Peer Review Ideas

When developing a rubric, it is necessary to determine the skill or topic to be assessed. For example, if students are asked to give feedback on a prototype of an autonomous aerial vehicle, the rubric criteria should measure areas such as, and not limited to: design, functionality, understanding of the science behind flight. Designing a rubric for the written report or proposal of the vehicle would not apply at this time. In some cases, students can be provided with feedback sentence stems to help them begin the process.

- - Written assignments: Students can be assigned partners to review their written work, such as essays or reports, and give each other constructive feedback on their arguments, critical thinking, supporting evidence, writing style, etc. Consider dividing the assignment into stages so that they receive feedback at every stage [7]. Some examples for peer-reviewing a written assignment are included below. This is also beneficial in understanding what high quality feedback looks like.
    - - Something that I loved in this draft and had not considered in my own writing is…
        
        I would have enjoyed knowing more about…
        
        A detail that I found missing is…
        
        A minor tweak that could improve the quality of this draft is…
  - Oral presentations: Peer reviews can be implemented to help students improve their communication and presentation skills. Students can be asked to give peer reviews based on criteria related to the learning objectives of the activity, such as content delivery [7].
  - Problem Based Learning teams or Group Projects: Periodical peer reviews can be established to help team members identify ways in which they can improve their teamwork skills, help maintain an equal division of labor, and encourage all team members to actively participate in the project [8].

A peer review rubric example

The following is a modified example of a very general rubric for peer review. The areas for review were Involvement in Critical Analysis, Participation in team discussions, and Etiquette in Dialogue with Peers. In this example rubric the students use a five-point Likert’s scale to rate their classmates’ performance in these areas, where each score is explained with examples of the type of performance the student should exemplify. Follow up questions can also be used in addition to the rubric in order to expand on their peer’s performance, or provide a space for the student to give helpful constructive comments to their classmate [6].

*Criteria*	Unsatisfactory = 1-2	Limited = 3	Proficient = 4	Exemplary = 5
*Involvement in Critical Analysis*	Classmate shows little or no evidence of completing or understanding assignments. Their contributions are largely personal opinions (e.g., “I agree” or “Great idea”) without support of concepts from the theory, outside resources, relevant research, or specific real-life application.	The student’s contributions repeat and summarize basic, correct information, but do not link readings to outside references, relevant research or specific real-life application and do not consider alternative perspectives or connections between ideas.	The student displays an understanding of the required readings and underlying theoretical concepts, including correct use of terminology.	Classmate displays an excellent understanding of the required readings and underlying concepts, including correct use of terminology. Their contributions integrate an outside resource, or relevant research, or specific real-life application (work experience, prior coursework, etc.) to support important points.
*Participation in team discussions*	Classmate does not contribute to team discussions or conversations around assignments.	The student sometimes contributes to ongoing conversations with affirming statements or references to relevant research, and asks related questions	The student constantly contributes to the class’s ongoing conversations about assignment, evidenced by affirming statements or references to relevant research. Constantly asks related questions, or makes an oppositional statement supported by any personal experience or related research.	The student actively stimulates and sustains further discussion by building on peers’ responses, including building a focused argument around a specific issue, or asking a new related question, or making an oppositional statement supported by personal experience or related research.
*Etiquette in Dialogue with Peers*	Written or verbal reviews show disrespect for the viewpoints of others.	Some of the student’s interactions with peers show respect and interest in the viewpoints of others.	Most of the student’s written or verbal interactions with peers show respect and interest in the viewpoints of others.	All written and verbal interactions with classmates show utmost respect and sensitivity to the viewpoints of others.

Rationale and Research: The WHY

Learning how to give feedback to others, and receiving feedback on their own work, are invaluable professional skills that your students will learn as they actively participate in the review process. Peer reviews have been found to improve responsibility, involvement, and student performance as well as professional skills including etiquette, communication, and listening [9]. Peer review of one another’s work enhances performance and helps strengthen the students’ evaluation skills [10]. This becomes more important if one of your objectives is to help your students develop life-long learning skills, where self-assessment will be key. In short, the peer review is a formative and constructive experience where your students can find areas to improve on and bridge the gap between their current performance and their best work.

Additional Resources and References

Interested in learning more? Here are additional readings on academic integrity topics as well as citations and links to articles referenced in this document.

References

[1] M A. Conde, L. Sanchez-Gonzalez, V. Matellan-Olivera and F. J. Rodriguez-Lera, “Application of Peer Review Techniques in Engineering Education,” International Journal of Engineering Education , vol. 33, pp. 918–926, 2017.

[2] Carnegie Mellon University, “Formative vs summative assessment – eberly center – carnegie Mellon University,” Formative vs Summative Assessment – Eberly Center – Carnegie Mellon University. [Online]. Available: https://www.cmu.edu/teaching/assessment/basics/formative-summative.html. [Accessed: 20-Apr-2023].

[3] D. J. Nicol and D. Macfarlane‐Dick, “Formative assessment and self‐regulated learning: A model and seven principles of good feedback practice,” Studies in Higher Education, vol. 31, no. 2, pp. 199–218, 2006.

[4] C. Lundquist, M. A. Skoglund, K. Granstrom, and T. Glad, “Insights from implementing a system for Peer Review,” IEEE Transactions on Education, vol. 56, no. 3, pp. 261–267, 2013.

[5] J. Sharp and M. Dyrud, “Two perspectives on Peer Review,” 2009 Annual Conference & Exposition Proceedings, pp. 14.1287.1–14.1287.19, Jun. 2009.

[6] J. Lax and A. Fentiman, “Improving graduate student oral presentations through peer review,” 2016 ASEE Annual Conference & Exposition Proceedings, Jun. 2016.

[7] K. Kuder and N. Gnanapragasam, “Implementing peer-reviews in Civil Engineering Laboratories,” 2011 ASEE Annual Conference & Exposition Proceedings, pp. 22.820.1–22.820.15, Jun. 2011.

[8] E. de Graaff and G. Saunders-Smits, “Peer evaluation and peer review to support PBL teamwork,” 2011 2nd International Conference on Wireless Communication, Vehicular Technology, Information Theory and Aerospace & Electronic Systems Technology (Wireless VITAE), 2011.

[9] Pisoni, G., Gaio, L., & Rossi, A. (2019). Investigating soft skills development through peer reviews assessments in an entrepreneurship course. 2019 IEEE International Symposium on Multimedia (ISM), 219–2195. https://doi.org/10.1109/ism46123.2019.00065

[10] J. Sharp and M. Dyrud, “Two perspectives on Peer Review,” 2009 Annual Conference & Exposition Proceedings, pp. 22.820.1–22.820.15, Jun. 2011.