Guidance and Structure
Each paper session is 50 minutes and has two parts:
-
Reviewer’s Card Activity (10 minutes). The room forms groups and takes a position on the paper based on the reading done before class, and the discussion they have during the activity.
-
Presentation and Discussion (40 minutes). The student who is serving as paper lead presents the paper and leads the discussion as one combined session, pausing to discuss each part of the paper as you get to it, the way a typical reading group works.
Reviewer’s Card Activity
Judging whether a paper’s evidence supports its claim is the core skill of this course, and the Reviewer’s Card Activity is how you practice it. Every paper session uses the same activity so the structure becomes second nature. It also means your discussion never starts from a silent room, because every group arrives holding a position they have already argued about. Blank cards are provided by the instructor at the start of class. The paper lead’s job is to run the activity clearly and keep it on time.
Procedure
Announce each transition out loud. Groups will run over if you do not. Use a timer to keep time if you need to.
Step 1. Form groups (30 seconds). Split the room into groups of five. Hand each group one blank Reviewer’s Card.
Step 2. Individual thinking, silent (2 minutes). Each student individually writes down, on their own paper, one sentence stating what they take the paper’s main claim to be, and one sentence naming what they see as the biggest weakness.
Step 3. Group discussion (3 minutes). Each group reconvenes and takes turns comparing what they wrote, working toward a shared position, accept or reject. Tell them explicitly that they do not have to agree, and that if they cannot agree on the decision they should note the split on the card and explain the discourse.
Step 4. Fill out the card (2 minutes). Instruct the groups to fill out their card. Remind them to write all group member names at the top.
Step 5. Share out (2 minutes). Go around the room. Each group gives one sentence: First, their accept or reject decision and then the reason. Keep this abridged.
Step 6. Collect the cards (30 seconds). Collect all cards and confirm every card has names on it. Cards are submitted to the instructor at the end of the exercise.
Reviewer’s Card template
The card is a single page. The instructor prints and distributes them.
┌────────────────────────────────────────────────────────────────┐
│ REVIEWER'S CARD │
│ CSCI 6907/4907 · Modern GenAI in Software Development │
│ │
│ Paper: ____________________________________ Date: _________ │
│ │
│ Group members: _______________________________________________│
│ _______________________________________________________________│
│ │
├────────────────────────────────────────────────────────────────┤
│ 1. THE CONTRIBUTION │
│ In one sentence, what does this paper contribute? │
│ │
│ ____________________________________________________________ │
│ ____________________________________________________________ │
│ │
├────────────────────────────────────────────────────────────────┤
│ 2. STRONGEST EVIDENCE │
│ What is the single strongest piece of evidence for it? │
│ │
│ ____________________________________________________________ │
│ ____________________________________________________________ │
│ │
├────────────────────────────────────────────────────────────────┤
│ 3. BIGGEST THREAT TO VALIDITY │
│ What most threatens the paper's conclusion? │
│ │
│ ____________________________________________________________ │
│ ____________________________________________________________ │
│ │
├────────────────────────────────────────────────────────────────┤
│ 4. ACCEPT OR REJECT? ☐ Accept ☐ Reject ☐ Split │
│ One sentence: why? │
│ │
│ ____________________________________________________________ │
│ ____________________________________________________________ │
└────────────────────────────────────────────────────────────────┘
Presentation and Discussion
Assume the room has read the paper. Your job is not to summarize it for people who haven’t. Your job is to explain it well enough that the room can evaluate it and discuss.
The presentation and the discussion are one session performed in tandem as one exercise. You present a part of the paper, then open it up for discussion, then move to the next part. Intro, {discussion}, method, {discussion}, results, {discussion}, and so on. The difference from a lecture is that you are not the one supplying the answers, but rather facilitating the larger group discussion. Your discussion questions go on the slide where you will ask them.
Aim for 7 content slides, plus one opening slide with the Reviewer’s Card prompts. Your speaking time across the 7 content slides should total about 15 minutes; the remaining 25 minutes is discussion.
Recommended Outline
| Slide | Content |
|---|---|
| 1 | Reviewer’s Card prompts (displayed during the activity) |
| 2 | Title, authors, affiliations, venue, year |
| 3 | The question the authors set out to answer, and why it matters |
| 4 | Background needed to follow the paper |
| 5 | Method: what the authors actually did |
| 6 | Results: the two or three key findings that carry the argument |
| 7 | Strengths and limitations |
| 8 | Open questions, discussion, and implications |
1. Reviewer’s Card prompts. The four prompts from the Reviewer’s Card (see above). Put this slide up when you start the activity and leave it up for the duration of the activity so groups can see what they are filling in.
2. Title, authors, affiliations, venue, year. Say who wrote this and where it appeared. Venue and year matter in a field moving this fast: a 2022 result about code completion and a 2026 result about agents are answering different questions. If the paper is a preprint, which many are, say so.
3. The question the authors set out to answer, and why it matters. Spend no more than two minutes here, but do not skip it. Try to state the problem in a straightforward manner, then say what makes the work non-trivial and who would care about the answer.
4. Background needed to follow the paper. Only what the room needs, and no more. Do not worry about getting too into the weeds here. This is also where you give a roadmap: tell the audience where the talk is going so they can follow the structure rather than reconstructing it.
5. Method: what the authors actually did. This slide usually deserves the most time, both presenting and discussing. The class cannot judge a finding without understanding how it was produced, and this is the slide the discussion will depend on. Cover who or what was studied, what was measured, and how. Slow down and take your time here, however avoid spending too much time on specific technical details.
6. Results: the findings the paper’s argument rests on. Pick the two or three key findings that support the paper’s claim, and skip the rest. Use the authors’ own figures where possible. State each result plainly before interpreting it, and be clear about what the results do and do not mean where relevant.
7. Strengths and limitations. Be specific and be fair. It is tempting to focus on flaws, but naming what the paper does well is part of the analysis. Frame weaknesses as openings for future work rather than as failures. Distinguish limitations the authors acknowledged from ones they did not.
8. Open questions, discussion, and implications. What follows if this result is true? What did the paper not settle? This is where you say what you think, not just what the authors said. Close by asking what the follow-up study should be, and give the room your own answer last.
Presenting the paper in the order the authors wrote it is fine. What matters is allocating time by importance rather than by section length. Related work rarely needs more than a sentence. If a section does not support the argument, say so and move on.
Not every paper fits this outline. If your paper builds a system rather than running a study, slides 5 and 6 become the design (what they built and which decision it turns on) and the evaluation (what they compared against and what that comparison establishes). If your paper argues a position rather than reporting results, they become the argument (the claim and the chain of reasoning) and the evidence it rests on.
An example shape for 40 minutes
| Time | Segment | What happens | |—|—|—| | 0:00–3:00 | Open (slide 2) | Title and venue. Report the accept/reject split from the cards in one sentence and name the objection that came up most often. | | 3:00–8:00 | Intro and motivation (slides 3 and 4) | Present the question the authors set out to answer and the background needed to follow the paper, then discuss: is the contribution what the authors say it is? Get the room to agree on what is actually being claimed before arguing about whether it holds. | | 8:00–18:00 | Methods (slide 5) | Present what the authors actually did, then discuss: does the design support the claim? What would you have done differently? This segment should get the most time. | | 18:00–26:00 | Findings (slide 6) | Present the two or three key results, then discuss: what do they establish, and what do they not? | | 26:00–32:00 | Limitations (slide 7) | Present strengths and limitations, then discuss: is the most common objection from the cards fatal to the paper’s claim, or does some weaker version survive it? | | 32:00–37:00 | Implications (slide 8) | Present open questions and implications, then discuss: if this is true, what follows? Who should change what they do? | | 37:00–40:00 | Close | Ask what the follow-up study should be. Give the room your own answer last. | Adapt the shape to your paper. A position paper has no method to interrogate, so the evidence segment becomes a question about whether the argument holds and where it breaks. A systems paper turns it into a question about whether the design was right and whether the evaluation supports the claim.
Advice on Giving the Presentation
Organize each section as big ideas, then details, then a one-line summary. People cannot absorb detail they have no place to put. Say what the section is about before you go into it.
Slides are a guide for your talk, not a transcript of it. If a slide contains full sentences you intend to read aloud, cut them down.
Use the paper’s figures. A good slide is mostly a figure with a short caption. Dense bullet lists are hard to follow and hard to speak from.
Calibrate for a mixed audience. This class includes students from different backgrounds and different levels of software engineering experience. Assume general computing knowledge. Do not assume familiarity with specific methods, tools, or subfields, and define terms you have gotten used to.
Practice once out loud with a timer. Fifteen minutes is shorter than it sounds. If you run long in practice, cut a slide rather than talking faster.
Preparing Discussion Questions
Prepare at least four discussion questions. Each one goes on the slide where you will ask it.
A question is worth asking if two well-prepared people who both read the paper could reasonably disagree about the answer. If the question has one correct answer, it belongs in your presentation, not the discussion.
Two questions that do not work:
“Did the authors do a good job?”
Too vague to answer. Nobody knows what kind of response is wanted, so nobody speaks.
“How many developers participated in the study?”
One correct answer, and it is in the paper. This is presentation content, not a discussion question.
Two questions that do work:
“METR found that developers were 19% slower with AI tools but believed they were 20% faster. If you were advising a company that had to set policy based on one of those two numbers, which would you pick, and why?”
Forces a choice between two defensible positions, and both sides have real arguments. Answering requires taking a position on what the study actually measured.
“Suppose the effect the authors found is real, but the mechanism is not the one they propose. What is the most plausible alternative explanation, and what would you measure to tell the two apart?”
Asks the room to reason about study design rather than recall results. It works on almost any empirical paper, and it produces concrete answers rather than general skepticism.
Running the room
When nobody answers, wait. Count silently to ten before rephrasing. Most presenters fill the silence after three seconds, which teaches the room that waiting works. If ten seconds pass, call on someone specific: “Group three, you said the sample was unrepresentative. Say more.”
Follow up before moving on. When someone answers, ask a second question about their answer before going to the next item on your list. Cycling mechanically through prepared questions produces four short exchanges instead of one real conversation.
When one person dominates, say “let’s hear from someone who hasn’t spoken yet.”
When a thread stalls, move on. Do not try to revive a question that is not working. This is why you prepared four and only need three.
When you disagree with someone, ask rather than assert. “What would have to be true for that to hold?” works better than telling them they are wrong.
Do not steer toward a conclusion you decided on in advance. Your job is to open the conversation up, not to land it where you already were.
Citation
If your presentation includes text, figures, data, or ideas from anywhere other than the assigned paper, cite the source. This includes other papers, blog posts, talks, and any figure or illustration you did not create yourself including graphics made by AI tools. See the academic integrity and generative AI policies.