An interview with
Silvia Bartolic
Wanting to share the result of students’ work with their peers, Silvia Bartolic, faculty member at the Department of Sociology, set out to try this type of assignment with her first- and second-year students. As she tried it out herself, she came across a few challenges and had some learnings along the way.
Please tell us about your project.
This last term, in my 200 class, I had a lot of first-year and second-year students. I thought, let’s see what’s happening. So, I had them do a review of a draft of their work, in groups. So it was groups reviewing groups, and each group had to review two projects. It was a formative review, but I asked them to grade the draft. They gave each project a grade out of 10, and I said to the class was your grade on the draft, which is 10 percent would be the average of the two reviews.
We trained them. We went through the assignment. We talked about the rubric, so we did this whole session on the assignment, and we did a session on peer review. We really made sure they knew what they were doing.
Doing the peer review exercise showed me where they didn’t understand the assignment, and that it was probably the first time they bothered to look at the rubric. It really kind of focused, “Oh, that’s what we need. That’s how it’s going to be graded.” I didn’t expect that to come out, but it really helped them understand, and helped them focus on the assessment piece, like what is a good quality piece.
What motivated you to initiate the project?
Originally, I had this great class on relationships, and I had fantastic projects at the end of term; really well researched. Sometimes they were papers, sometimes they were websites, but really awesome projects — but I was the only one reading them.
I was trying to find a way of sharing this content knowledge back to the students. At the same time, our program outcomes in sociology are, they need to be able to read research and critically reflect, and speak from a sociological awareness, and we wanted to prepare them for the workforce. I was thinking these are great skills, like reviewing, being able to assess other people. You’re going to have to do this in life when with your job.
How did you do it?
I started to try peer review. And the very first time I did it, it was not the best.
It was summative review on videos. I had the class watch all the videos, so they were watching 25 videos, and the feedback from students was that it took way too much time. It was overwhelming. It was fun, but it was way too much time for us.
From there, I played around with formative peer review. I still do some summative where they actually have to give a grade. I use formative. I’ve done it on the spot in the classroom following a presentation. We just break out and do it right there. I’ve done it as homework, I’ve done individual reviews of groups, and now I’ve moved to groups reviewing groups. I’ve tried a whole bunch of things.
In the process of all that I collect student perspective data. That’s what I call it, so I had a survey asking various questions, how they feel about using peer review and also team member reviews. Overwhelmingly, the students thought this was important for their future, but overwhelmingly they were like, “We are not prepared to do this. The instructor should be grading, why are you making us do this?”
My next step then was, okay, it’s important to now train them how to peer review. I had one of my TAs put together a workshop on how to do peer review, and we started really by selling it to the students; this is why we’re asking you to peer review. Here is why it’s important, and then here is one method, here is a process.
What we had asked them to do, and what I continue to do is summarize the work, and then start with giving some critical feedback, really focusing on constructive feedback.
[After the workshop students] felt more prepared to do [the peer review], but still the issue was “I can’t give a grade. I’m not confident to give a grade.”
So, I would say. “You have to think about that rubric.” And they did, then I asked my TA at the time, “Can you please grade the drafts?” And I did a little comparison in my grade book, the grades are very similar. On average, there was a .3 difference in grading. Very similar, so students can grade, as long as you have a very specific rubric.
Continue reading ▼
Did you have the support you needed for the project? Is there additional support you wish you had had to help you achieve your goals?
I think when I started it was probably just an idea I heard at some workshop at the CTLT. To be honest, I was like, “Oh, peer review. That’s a good idea.” Let’s try this. It never was a scholarship of teaching and learning project. It was not meant to be an empirical study. I collected data because I was just curious, is this something the students like? Should I be doing this? It wasn’t really a planned study at any point.
In terms of support, I didn’t ask for it. I worked with Isabeau Iqbal [Senior Educational Consultant at the CTLT] a lot, so looking up references, and thinking about what other people were doing. Once I started, then I got more curious and read more of the literature, and talked it through with her. She’s been amazing. She’s seen other projects across campus, which I wouldn’t know of. She’s connected me to people that are doing peer review. What was really cool was, one person she connected me with has now connected me with someone else. It’s kind of like, it’s growing.
What were some of the key outcomes of the project?
I think that modeling has really come through by trying to assess somebody else. I think maybe it’s harder to accept criticism in your own work, but it’s easier to see, “Oh, someone is doing that. I get it.” When you’re not putting up the barrier of “I don’t want to be hurt by this judgment,” so to speak.
I think I want to look at that, and I just read something about not using judgment in peer review. Rather than asking them, “What stood out as positive? What could they improve?” That’s still judgment based. But rather asking them to describe things, like can you circle the thesis? Can you describe the thesis? And by giving that to the person who wrote it, you can see if they’re on, “Did I sufficiently do this?”
How did the project impact learners or how you teach?
I think the big one was they learned what a good example of something was. Whether it’s a presentation, or putting something in to a website, or a video, or that’s how you write a paper.
Well, I think if I’m doing peer review, it’s definitely made me think of my assessment in a deeper way. What am I trying to get them to learn in this project? Why am I making them do all this stuff? It’s really made me think about my objectives and am I meeting them, or is this just extra work?
I think that’s made me make my rubrics clearer for the students so that it’s more student centered. It’s moving away from doing it quickly to doing it thoughtfully. I really think about if I’m asking them to do this, does this make sense? Not just to me, because if I’m the only one grading, then I can go by a rubric and it doesn’t have to be perfect because I’m grading and I can adjust it in my head, but if you’re asking them it really has to be clear.
What are some of the lessons that you have learned that you would share with your colleagues?
If they’re going to do peer review, the first thing I would ask myself is, why do I want to do it? Why in this particular course or this assignment? How is it going to relate to the learning outcomes? On which assignment, how do I need to restructure that assignment? I’ve had to break mine apart a lot more to make it user-friendly, so to speak.
Then for my group work I never have partners anymore. Especially, because I do team member reviews, I always want to have at least three to four people. If somebody really isn’t pulling their weight, then multiple people will tell me that. Especially if I’m going to change their grade because they’re not pulling their weight. Then I don’t want it to be one person against the other.
What are the plans for this work? How do you plan on sustaining what you have created through the project?
I would really like to do a proper study, where I want to look at what students are learning, but I want to be able to actually, not just their perspectives, because that’s the data I have, what do they think, but I want to see if I can actually measure learning. I feel like that needs to be a real study, and maybe with several people, not just me but other departments, or instructors. I don’t know if it matters what department, just to see.
Yeah, I think I need to collect a few interested parties. I want to try to find a formal way to access whether they learn anything from the process of reviewing. That is where I need resources. I need a conversation with maybe somebody that does evaluation. Like maybe Adriana [Briseño-Garzon, Senior Manager, Research & Evaluations], to give me ideas on how to assess any learning that might happen, that is not just their perspective.
How UBC faculty have incorporated Student Peer Assessment
Silvia Bartolic
Silvia introduced SPA as a way of sharing her sociology students’ work with their peers. She explains the challenges and learnings she found along the way.
James Charbonneau
Initially a community-building exercise, James explains how peer evaluation in his physics class evolved into student peer review platform ComPAIR, and the importance of safety in peer assessment.
Kevin Chong
One of the foundational pieces of creative writing is peer review. Traditionally run in small workshops, Kevin shares how he brought peer review across to large lecture classes and its importance to developing writers.
Peter Graf
Personalized feedback for student learning can be challenging to deliver in large classes. But beyond that, Peter sees peer assessment as an opportunity for students to develop important critical reading and self-assessment skills.
Misuzu Kazama
It’s far more common for peer review to be applied to writing tasks than spoken ones. Can language students give each other good feedback on a spoken assessment? Misuzu developed a project with real-world context to find out.
Kelly Allison & Marie Nightbird
Interpersonal communication is a key skill for social work students. After using informal peer feedback to develop those skills for many years, Kelly and Marie share how they formalized the process to gather insights and improve the student experience.
Jennifer Walsh Marr
From a starting point of investigating accountability in group work, Jennifer’s peer assessment project led to more student-centred teaching, and a better sense of community for her students.
