TLDR;

• A complete assessment-scheme and procedure to run design-critique in a classroom setting.
• From individual interview to group presentation.
• A gamified presentation session.
• Intended to develop creative thinking and collaboration for IT students.
• Use Playing Card and Google Form to evaluate each other design works.

Problems & Motivation

In the realm of teaching design-based subject in higher education/university, one of the most challenging aspects is the assessment of students’ design assignments. Traditionally, this has been done through individual interviews. While this method allows instructors to understand a student’s thought process, it has its limitations. It can be time-consuming, it may not fully capture the collaborative nature of design work, and it may put students who are less comfortable with one-on-one interviews at a disadvantage.

To address these challenges, I propose a shift from individual interviews to group-based design critiques. Design critiques are a staple in design disciplines and involve a systematic examination and discussion of a design work. This method encourages peer learning and collaboration, essential skills in the real-world design process.

By implementing design critiques in a design unit (i.e., object-oriented design and programming/OOD), we can assess students’ understanding of design principles in a more holistic and collaborative manner. This approach not only mirrors professional design practices but also fosters a learning environment where ideas are shared, critiqued, and iteratively improved.

In essence, the motivation behind this shift is to enhance the learning experience in my current OOD unit, making it more engaging, collaborative, and reflective of industry practices. By doing so, I aim to equip my students with the necessary skills and knowledge to excel in their future design-related endeavours, especially in software engineering field. I hope this learning design can be applied in other learning contexts.

Tools

• Google form for students to score the presenting groups. → this requires some admin work after presentation week.
• Playing cards (52-cards) for critique participation.
• Google spreadsheet for TAs to score the presenting groups and to track students’ card usage.
• Briefing slide that contains presentation rules.

General agenda:

Assume it will run in a 3 hours lab. Start with 15 minutes introduction re-explain rules & boundaries. Each students should hold two cards. The whole design session will take 2.5h. Assume each lab has 6 groups, which means each group has 25 minutes. 1 group = 4 students. 15 minutes on final evaluation. Faciliated with a Google Form to give students a place to take notes & rate team.

1. [mins 00:05 to 00:10] briefing & rules explanation.
2. [mins 00:10 to 00:35] Group A
   1. Presentation (10 mins)
   2. Discussion (10 mins)
   3. Evaluation (5 mins): guided by teachers
3. [mins 00:35 to 01:00] Group B
   1. Presentation (10 mins)
   2. Discussion (10 mins)
   3. Evaluation (5 mins)
4. [mins 01:00 to 01:25] Group C
5. [mins 01:25 to 01:50] Group D
6. [mins 01:50 to 02:15] Group E
7. [mins 02:15 to 02:40] Group F
8. [mins 02:40 to 02:55] Wrap up & taking notes on the card counts (with attendance).

Due to time constraints, each group should not cover all requirements, and they have to select. But, they can cover some overlapping requirements. For example, the assignment has 4 major requirements: Group A-B-C can cover REQ1-2 and Group D-E-F cover REQ3-4.

Tasks & responsibilities:

Teacher’s tasks:

Assume there are two teachers in a design lab:

Teacher A will take a role as a Moderator:

• Ensures discussion flows smoothly.
• Tracks time.
• Tracks students’ participation cards.

Teacher B will take a role as a Commentator:

• Provides final critique on the design.
• Takes notes for a group-project marking later.
• Gives a presentation score.

Ideally, a senior teacher who has more expertise in the subject/content takes Commentator role. But, teacher’s may swap role in the middle of the session. In a situation where there is only one teacher, extra preparation might be required (e.g., use stop watch with time laps).

Student as a presenter

Each group will present their work:

• Each group (25 minutes): 10 minutes presentation, 10 minutes discussion, 5 minutes evaluation (suggestions/improvements) from teacher (optional if time allows).

It is highly recommended each presenters answer or response to the critique. Other team members may add more information in the discussion if required.

Student as an audience

Each audience has 2 cards to critique throughout the whole lab session. Each “participation” card is worth 5pt. Students who don’t exhaust/use their cards will receive penalty (5pt penalty for each unused card). Once a student has exhausted all cards, this student must report it to the teacher who has Moderator role at anytime (e.g., in between presentation). Alternatively, Moderator can tally this process. This part contributes 10pt of individual participation. To add a slight moderation, optionally, we can use Black card to give a positive comment, and Red card to give a negative comment.

A presenting group can only receive 4 critiques quota from audiences, to ensure discussion adheres to the time limit. Each critique may take around 2.5 minutes, with some discussion & elaboration from presenters. In those critiques, they must consist of at least 1 poor design comments. In a case where 4 quota has not met, teacher should sacrifice their 5 minutes evaluation to give this opportunity for students to talk & discuss. Beyond this 15 minutes (10 discussion + 5 evaluation), several passive students may not have a chance to use their card, and they will lose their participation score. It is intended to encourage active participations. Please ensure this is well communicated at debriefing!

After a presentation, audience and teacher rate the presentation:

Poor = 5 pt, Okay = 8pt, Good = 10pt. For teacher’s score, double the points (10, 16, and 20 respectively). Teacher has Extremely Poor option = 2pt.

Rationale: I want to ensure that there is always a point, even though a presentation may be extremely poor.

1. Quality of the delivery: flow and clarity of the design diagram, which can be assisted with code examples (Poor/Okay/Good) = 10pt (tutor = 20pt max).
2. Quality of the content: relatedness to the requirements and adherence to design principles (Poor/Okay/Good) = 10pt (tutor = 20pt max)
3. Quality of the responses/discussion: Each presenter responses to the critique adequately (Poor/Okay/Good) = 10pt (tutor = 20pt max)

In total, it will be 30pt from peers, and will be averaged depending on the number of respondents. Then, 60pt from tutors that results in total 90pt maximum. Lastly, we have 10pt for individual participation that results in total of 100pt.

Grading

In summary, individual scores can be calculated as follows:

• Peer’s score: 30pt (group)
• Lab tutor’s score: 60pt (group)
• Participation’s score: 10pt (individual)

Total = 100pt. No attendance = no score.

This design critique should contribute to 50% out of 100% of a total assignment score. Another 50% from group project’s artefact/system which will be marked by teachers as usual, following marking rubric. Let’s take a look at an example below.

A group receives 20 responses from peers/audience, with a sum total score 500pt [avg. Good, Good, Okay]. So, 500 divide by 20 students = 25pt, so the group score is 25 out of 30 total score. Teacher might be quite critical but in general it was pretty good, and gives a score 48pt out of 60 total [Good, Okay, Okay]. So, this group, will receive 73pt out of 90pt. Remember, we still have 10pt for individual’s score where each student who actively participates as required will get this score. Assume this student has actively participating (10pt), so 73pt + 10pt = 83pt. Hence, 83pt/100pt X 50% = 41.5% for this student’s design critique score. If their group project’s score is considerably good, 40% out of 50%, this student will get 81.5% out of 100% in this assignment. In most Australian university, it will be considered as a high-disctinction score.

Potential scenarios:

Here are some scenarios that could happen for implementing this ‘Design Critiques’ in the classroom:

• All students are being so nice to each other:
  • Scenario: each group gets 30% score automatically [rate everyone as Good] because students may feel bad if they give other students bad scores.
  • Mitigation: TAs has more weight in assessing this presentation (i.e., double points). It shall not be an issue, but breaks the assessment integrity. Consider clarify this in the briefing acti
• Sickness or other circumstances [with notice & clear reason]:
  • Scenario: A student cannot attend the session due to sickness or other emergencies.
  • Mitigation: Replace with individual interview when student is available (next week), but students will be assessed using a similar criteria. Audience’s scoring system (30%) will be fully replaced with teacher’s score (manual input). So, teachers will have 90% score. Participant’s score will be automatically granted (10%).
• Students who do not attend a session [without notice]:
  • Scenario: A student just did not attend the session, and does not have special consideration.
  • Mitigation: No mark on this. But, they can still get a score from the normal work if there’s any evidence they have contributed in the code (i.e., with maximum 50%). To pass without attending design critique session is possible, but extremely difficult.
• Students leave a class in the middle of the session:
  • Scenario: Perhaps it is caused by an emergency or other unexpected circumstances. They may feel they are not obliged to stay in the activity after they have done the required participations (i.e., giving critiques).
  • Mitigation: If this student has presented and contributed to design activity (i.e., exhausting all critique cards), they may leave without any consequences. Otherwise, student must apply special consideration to be interviewed later.

Considerations: Benefits & Limitations

➕Benefits:

1. Students can learn from each other’s work/design. In individual interview, teachers need to repeat similar/common errors several time.
2. Simulate real-world design process.
3. Practice creative and design thinking, specifically being objective on the design work.
4. Gamification that may encourage active participations.
5. Reduce the complexity of individual contributions scaling, and truly assess students based on design critique discussion.

➖Limitations:

1. The quality of discussion is highly dependent on students’ knowledge and engagement.
2. Students may not aware what is good and what is bad design. A clear criteria of design is required at student’s hand to adequately critique peer’s design.
3. May increase cognitive load for teachers (time management & scoring)
4. May increase students’ anxiety who may feel humiliated for poor design. Teachers must clarify the scope and use positive language in their comments/providing feedback.
5. Students who do not have good presentation/language skills may be at disadvantage. Recommend use more visual aids to guide the conversation.
6. This assesment will emphasise on group work. So,students who prefer individual work (hate group work/ prefer solo/cannot handle criticism/carry the whole team) will dislike this assessment format. Individual interview can be arranged, but there should be an extra consequence (otherwise everyone will request interview). Work hard !== higher scores.

Delivery Tips

Lastly, I want to share some ideas/tips to implement this ‘Design Critiques’ in your classroom.

1. Students:
   1. If you are a presenter, tries to think how can you get a lot of feedback from your peers to improve your design/work.
   2. How to critique a design:
      1. Recommended pattern: State current design + potential problem(s) + give alternative design + “what do you think?”
      2. Example: “I think your design is good to address X, but it may have a problem with Y, you may consider doing Z. What do you think?”
   3. If you don’t have any ideas to critique, consider asking analytical questions that do not end up with just “Yes/No” answer.
      1. “In comparison to X, why did you think your design is better/supporting?”
      2. “Have you considered X principle in Y? Why or Why not?”
      3. “How did you solve the problem X, considering Y?”.
      4. “I have X problem and I have tried your design, but issue Y rises, how did you solve this?”
2. Teachers:
   1. Disengaged students will make conversation less meaningful. If none talks within first 1 minute, teachers may start the discussion with just one idea to spark the conversation. In any case students begin to talk, always prioritise them. If none talks, teacher may keep providing critiques. You can also remind them that they may lose 10% of the mark for not contributing.
   2. Any critique is valid as it is part of a learning process. They may say nonsense, not being too critical on the design, and some discussion may not be too interesting. Please hold yourself, and use the 5 minutes evaluation to adjust & clarify. Ensure that this is well communicated in the briefing.