CATME: Team Formation and Peer Evaluation

< Back to Quick Reference Guides

Implementation and Timing: The WHEN, WHERE, and HOW

Instructors can use CATME to form teams when a project is first introduced. The team formation questionnaire should only take students 15 minutes or so to complete, and should be assigned in advance of when students are expected to begin working in groups on the project. This will allow the instructors enough time to use CATME to form teams prior to class.

The CATME peer evaluations should be disseminated to students at the end of the semester once the project work is completed to assess student performance. These peer evaluations should take students 15-30 minutes to complete. For longer-term projects, it is recommended that instructors distribute one or more additional peer evaluations while students are working on the project. This provides instructors an opportunity to identify and troubleshoot problems with teams before the end of the project. It also allows students to obtain feedback on how their team members perceive their contributions and performance.

ASU Instructors who have not used CATME will first need to request a CATME account. ASU has an institutional license so your account should be approved within a business day or so. Upon approval, instructors should be able to go to catme.org to log into their accounts. 

Student accounts are automatically created when the instructor creates the class and imports their student information, which includes student names and email addresses. The first time students use CATME, they will need to reset their password in order to set up their account. Students will need to go to catme.org and click the links that says ‘Forgot my password’ and follow the instructions. Once they set up their password, they will be able to successfully log-in to CATME and complete any future surveys.

Creating a Class

Creating a class is the first step before the Team-Maker and Peer Evaluation surveys can be used. Upon logging into your CATME account, you will need to select the icon in the upper right corner that says ‘Create a Class’. Follow the instructions provided by CATME and fill out the necessary information. 

CATME Team-Maker

The purpose of the CATME Team-Maker is to optimize the formation of teams based on information provided by students such as their schedule, previous experience, gender, race, and more. Once you have created the class, you can select the icon that says ‘Add Activity’. There are two options with brief tutorials: ‘Team-Maker’ and ‘CATME Bars [Peer Evaluation]’. First you will need to use the ‘Team-Maker’ survey to create teams.

For the Team-Maker survey, you will be asked to fill out information and specify which survey questions you will include. Potential question criteria include:

• • Schedule
  • Demographics (gender, race/ethnicity)
  • Grade-Point Average (GPA)
  • Prerequisite courses
  • Skills (software, hands-on, writing, shop)
  • Discipline or sub-discipline
  • Leadership preferences
  • Detail oriented or big-picture
  • Extracurriculars (fraternity, sorority, sports interest)

CATME includes default options such as Gender, Race, Schedule Availability, etc. that will inform a scoring heuristic to optimize student-team matching. For example, research suggests that it is important for female students not to be “outnumbered” by male students in a given team.

After the selection of the survey questions to be included, CATME will ask instructors to upload a .csv file with information on students. Instructors can use Excel or Sheets to create a file with students’ names, email addresses, and ID. It is required to follow the Example CSV file provided by CATME. Make sure to label 4 distinct columns in the first row: first, last, email, id. Then provide the subsequent student information in the following roles under the appropriate column. Note that a 5th column can be added called ‘team’ that includes the team number the student has been assigned to.

Once students have created their accounts and completed the Team-Maker survey, the data can be used to create teams. Log into your CATME account, click the link to your class, then click the link to the Team-Maker activity that was created, and then click the link ‘View Data and Teams’. From here the instructor can follow the step-by-step instructions. Instructors have the capacity to change the default settings on the various parameters (e.g. Gender, Race, etc.). 

Instructors can also view comments from students. This is a great way for students to let their instructor know if they want to work with a classmate or not work with a specific individual. Instructors can also manually assign students to specific teams or separate students based on student feedback. Once the instructor is content with the specified parameters, they can then have CATME make the teams according to the scoring heuristics. These teams can then be saved and used in future CATME Peer Evaluation surveys. 

CATME Peer Evaluations

The purpose of the CATME Peer Evaluation surveys is for the instructor to gather information on team dynamics and performance. Additionally, it’s an opportunity for students to provide their team members with feedback on their performance. Similar to the Team-Maker survey, the instructor will need to click the icon that says ‘Add Activity’ to set up the survey. There are various options the instructor can choose from in setting up their survey.

• • Rater Practice. CATME recommends that students first participate in rater practice first before completing their actual peer evaluations, but this option is at the discretion of the instructor. 
  • Survey Questions. By default CATME assesses the following five questions although instructors are given the option to add/delete additional questions.
  • Peer-to-Peer Comments. Instructors have the option to enable or disable peer-to-peer comments. If peer-to-peer comments are enabled, instructors can specify whether to release these comments and whether to make the comments anonymous. Instructors should consider which option makes the most sense for their course. Generally speaking, enabling peer-to-peer comments elicits more detailed information that can explain CATME scores and releasing peer-to-peer comments allows students to provide each other with more detailed feedback on their scores. This video explains the benefits of requiring peer-to-peer comments. 

Once the Peer Evaluation survey has been completed by students and closed, instructors can access the results. CATME provides all of the raw data and summarizes the results into two Adjusted Factor Scores: one that includes self-evaluation and another that excludes the self-evaluation. The adjusted factor score attempts to show the contribution of a given student relative to all members of the team. Generally speaking a student that scores at or above a 1.00 are likely solid contributors with high-performers typically receiving a max score of 1.05. A score below a 1.00 may indicate a student is not equally contributing. Instructors should be aware of “Exceptional Conditions” that can also influence scores. Very low scores and certain exceptional conditions may require instructor intervention such as follow-up discussions with specific group members, and/or possible reduction of grades.

Rationale and Research: The WHY

While project-based learning has been shown to support the development of a wide-range of skills, they can be challenging to facilitate. Whether it is a student who does not contribute their fair share of work or a team with team dynamic challenges, problems can arise that lead to negative group experiences. Fortunately, there are excellent tools available, like CATME, that provide instructors with valuable data and insights which they can use to more effectively form and monitor teams in project-based learning courses.

Additional Resources and References

For further information and or assistance, there are a number of instructor videos and tutorials available on CATME’s website.