Katkat
Back

Defend Your Own Research Paper: Experimental Setup, Practicality and Later Work

CohereMachine Learning Engineer · Onsite
MediumMachine Learning EngineerOnsiteMachine Learning

Last updated

A research paper round where you present your own published paper to an interviewer who already knows it and questions its weaknesses. Prepare to judge whether the experimental setup was reasonable, whether the results are practical, and what the field has learned since publication, all with specific, honest self-critique.

In a research paper round, you present one of your own published papers. Assume the interviewer already knows the paper well. They may skip your prepared slides entirely and go straight to its weaknesses. Expect questions such as:

  • Was your experimental setup reasonable? Where is it weakest?
  • Are the results practical, that is, would they hold up and be useful outside the paper's setting?
  • What major developments has the field seen since the paper was published, and how do they affect your conclusions?

This round is not a reading-group walkthrough of the paper's content. Prepare the presentation and your answers so that they hold up under this kind of questioning.

Clarifying Questions

  • How long is the round, and how is it split between presentation and questions?
  • Should the paper be related to the team's area, and is co-authored work acceptable if you explain your own contribution?
  • Are slides expected, or is a whiteboard discussion equally acceptable?

Hints

Hint 1

Critique it first: Before the interview, list every choice in your experimental setup that a skeptical reviewer would question, and decide what evidence you have for or against each one.

Hint 2

Since publication: For the question about later developments, look for later work that changed a baseline, a benchmark, a scale or an assumption your paper relied on.

Follow-up questions

  • If you could rerun one experiment with unlimited compute, which would you choose, and what result would change your mind?
  • Which later result most threatens your paper's main claim, and how would you respond to it?
  • How would the method or its lessons apply to post-training a large language model?
  • If you wrote the paper today, what would you change in the evaluation?