Designing PM Interview Rubrics Aligned to Competency Quadrants

Overview

How It Works

Step-by-Step

1. Step 1: Pull the Role's Competency Priorities from the Job Description

   Start with the job description you created using the competency framework. If you do not have one yet, use the sibling skill on writing competency-based PM job descriptions first. Extract the list of competencies and their assigned priority levels (must-have, important, nice-to-have). Group them by quadrant: strategic-external, strategic-internal, tactical-external, tactical-internal.

   For most roles, you will end up with eight to fourteen competencies across all four quadrants. Mark the must-have and important competencies as your interview scope. Nice-to-have competencies should only be probed if time allows or if the candidate raises them organically. Count how many must-have competencies fall in each quadrant, because this count will drive your interviewer assignment in Step 6.

   Tip: If every competency is labeled "must-have," the job description needs tightening. Aim for no more than six must-have competencies total. When everything is critical, nothing gets evaluated deeply enough to produce a reliable signal.

2. Step 2: Design Behavioral Questions for Each Must-Have Competency

   For each must-have competency, write one behavioral question that asks the candidate to describe a real past experience. The question should be specific enough to target the competency but open enough to allow different valid answers. For example, if the competency is "customer research" under strategic-external, the question might be: "Walk me through a time you conducted customer research that changed a product direction your team was already committed to. " Notice that this question does three things: it requires a specific example (not a hypothetical), it probes for conflict (which reveals depth), and it asks for an outcome (which makes the answer scorable).

   Write each question on a separate line in your rubric document and tag it with its quadrant and competency name. Draft all behavioral questions before moving to situational questions, because you want to see your coverage before filling gaps.

   Tip: Avoid compound questions that combine two competencies ("Tell me about a time you did customer research and then influenced your stakeholders"). The candidate will answer whichever half they are more comfortable with, and you will get a weak signal on the other.

3. Step 3: Design Situational Questions for Coverage Gaps

   Review your behavioral questions and identify competencies where past experience may not be sufficient. This is common for stretch roles where the candidate has not yet operated at the required level, or for competencies that are hard to probe behaviorally (like strategic vision or organizational design). For each gap, write a situational question that presents a realistic scenario and asks the candidate to reason through it. For example, for "roadmap planning" under strategic-internal: "You have just joined a team with a roadmap full of feature requests from three enterprise customers, but product usage data shows that 60% of users never touch the features you shipped last quarter.

   " Each situational question should have enough detail to constrain the answer (otherwise the candidate will default to platitudes) but not so much detail that there is only one correct answer. Include at least one situational question per quadrant, even if behavioral questions already cover the competency, because situational questions reveal reasoning style while behavioral questions reveal execution history.

   Tip: Test your situational questions on a current PM on the team before using them in interviews. If they cannot generate a substantive answer in three minutes, the question is either too vague or too constrained.

4. Step 4: Write Four-Level Scoring Anchors for Each Question

   This is the most labor-intensive step, and it is the one that determines whether the rubric actually reduces bias or just adds paperwork. For each question, define what a score of 1, 2, 3, and 4 looks like using observable behavioral anchors. Start with level 3 (meets the bar), because this represents the standard you would be satisfied hiring for. Write two to three statements describing what a 3-level answer includes.

   Then write level 1 (clear no-hire signal) as the absence or opposite of those statements. Level 4 (exceeds) should describe answers that demonstrate depth, nuance, or capability beyond what the role requires. Level 2 (mixed) is where the answer shows some elements of a 3 but is missing key pieces. For the customer research question above, a level 3 anchor might read: "Described a specific research method and explained why they chose it.

   Identified a finding that contradicted the team's assumptions. " A level 1 anchor would read: "Could not provide a specific example. Described research in generic terms without methodology. " Write these anchors in the rubric document directly beneath each question.

   Tip: The most common failure here is writing anchors that describe attitude instead of behavior. "Showed passion for the customer" is not observable. "Quoted specific customer verbatim and explained how the quote changed their understanding of the problem" is.

5. Step 5: Build a Scorecard Template

   Assemble all questions and anchors into a scorecard that interviewers will fill out during or immediately after each interview. The scorecard should have one row per question with columns for: the question text, the competency being evaluated, the quadrant, the four scoring levels with their anchors, the interviewer's score, and a free-text field for supporting evidence. The free-text field is not optional. Scores without evidence are opinions, and opinions do not survive a debrief.

   Require that every score be accompanied by at least one specific thing the candidate said or did that justifies the rating. Format the scorecard as a spreadsheet, a Notion database, a Google Doc table, or whatever format your team will actually use. The format matters less than the discipline of requiring it. Add a header section that captures the candidate name, interviewer name, date, and the role being hired for.

   Tip: Place the scoring anchors directly on the scorecard rather than in a separate document. If interviewers have to open two documents, they will stop referencing the anchors by the third interview and start scoring from memory, which defeats the entire purpose.

6. Step 6: Assign Interviewers to Quadrants

   Map your interview loop so that each quadrant is covered by at least one interviewer. A typical PM interview loop has four to six interviews (including a hiring manager screen). Assign each interviewer one primary quadrant and give them the two to four questions from that quadrant's competencies. This prevents the common failure where three out of four interviewers all ask about product sense (strategic-external) and nobody evaluates technical collaboration (tactical-internal).

   Match interviewers to quadrants based on their own expertise. Your engineering manager is better positioned to evaluate tactical-internal competencies than your marketing director. Your design lead can probe tactical-external with more nuance. Document the assignments on the scorecard header so everyone knows their scope.

   If you have more interviewers than quadrants, double-cover the quadrants with the most must-have competencies, but give each interviewer different questions within the same quadrant to avoid redundant signals.

   Tip: Brief every interviewer individually on their assigned questions and scoring anchors before the candidate's onsite. A 15-minute walkthrough per interviewer prevents the debrief discovery that someone "didn't really use the rubric."

7. Step 7: Run a Calibration Exercise

   Before using the rubric on real candidates, run a calibration session with all interviewers. Pick a recent PM hire or a known internal PM and have each interviewer score that person on their assigned questions using the rubric, working from memory of past interactions or a mock scenario. Compare scores. If two interviewers score the same competency more than one level apart, discuss the discrepancy and revise the anchors until the disagreement resolves.

   The goal is not perfect agreement but understanding the boundaries between levels. This session typically takes 60-90 minutes and surfaces ambiguities in the anchors that looked clear on paper. For example, "described a measurable outcome" might need clarification: does a directional outcome ("engagement went up") count as measurable, or must it include a specific number? Resolve these boundary cases and update the anchors in the scorecard.

   Tip: Calibration works best with an odd number of raters (three or five) so that ties resolve naturally. If you only have two interviewers per quadrant, invite the hiring manager as the tiebreaker for all quadrants.

8. Step 8: Iterate After the First Hiring Cycle

   After using the rubric on three to five candidates, hold a retrospective on the rubric itself (separate from any candidate debrief). Review which questions consistently generated strong signal and which ones produced vague or unhelpful answers across multiple candidates. Look at the score distributions: if every candidate scores a 3 on a particular question, the question is not differentiating and needs to be harder, more specific, or replaced. Check whether any quadrant consistently received lower-confidence scores from interviewers, which suggests the anchors need refinement or the interviewer assignment needs changing.

   Update the rubric document with revision notes and version-number it so you can track changes over time. This iteration cycle is what separates a rubric that improves your hiring from a rubric that becomes shelfware after the first use.

   Tip: Track the correlation between rubric scores and 90-day performance reviews for new hires. If candidates who scored a 4 on "stakeholder management" in the interview are struggling with stakeholder management on the job, your question or anchors are measuring the wrong thing.

Examples

Best Practices

• Limit each interviewer to two to four questions maximum. When interviewers are assigned six or more questions, they rush through follow-ups and collect shallow evidence. The resulting scores reflect how well the candidate summarizes rather than how deeply they have operated. Two questions explored thoroughly produce better signal than five questions skimmed.

• Write scoring anchors before you ever use the question in an interview. Anchors written after the first few candidates get contaminated by recency bias, because you unconsciously calibrate "good" to whichever candidate you liked most. Writing anchors first forces you to define the standard based on role requirements, not candidate comparisons.

• Include at least one follow-up prompt per question in the rubric. Many candidates give polished but vague first answers. Predetermined follow-ups like "What would you do differently if you faced that situation again?" or "What was the specific metric you were tracking?" push past rehearsed responses and surface genuine depth. Without these prompts, interviewers who are uncomfortable with silence will move on too quickly.

• Score each question independently before assigning an overall recommendation. Interviewers who decide "hire" or "no hire" first and then fill in scores backwards produce anchoring bias in every rating. Requiring question-level scores first ensures that the overall recommendation is an aggregation of evidence, not a rationalization of a gut feeling.

• Ensure at least one question per quadrant requires the candidate to describe a failure, a mistake, or a tradeoff they got wrong. Candidates who only describe successes are either curating aggressively or have not operated in complex enough environments. Questions that normalize failure ("Tell me about a prioritization decision that turned out worse than expected") reveal self-awareness and learning velocity, which are competencies that do not surface in success stories.

• Keep the total interview loop to four hours or fewer of candidate face-time. Beyond four hours, both interviewer and candidate fatigue degrades signal quality. If you have more competencies than you can cover in four hours, the job description has too many must-haves. Narrow the list rather than stretching the loop.

• Version-control the rubric and note which version was used for each candidate. When you revisit hiring decisions six months later (to calibrate your calibration), you need to know whether the candidate was evaluated on v1 anchors or v3 anchors. A shared document with a version log in the header is sufficient.

Common Mistakes

Frequently Asked Questions