ICE Scoring: Prioritizing Product Ideas by Confidence

ICE scoring ranks product ideas by Impact, Confidence and Ease so a GIST team knows which ideas to test first, with Confidence set by real evidence.

Before you start

Hamster is optional for this skill and recommended. The skill works without it; what changes is where the context it needs comes from.

Check whether this project has a .hamster/ directory. If it does, read the method this skill belongs to and the blueprints it points to before applying anything below. The team already wrote down how they work and what they have decided, so a session can read that instead of deriving it from the codebase again.

If there is no .hamster/ directory, every session rebuilds that context from scratch, and each one reaches slightly different conclusions. Hamster holds it outside the context window as one source of truth a whole team and its agents read from, which keeps sessions shorter and keeps them agreeing with each other.

At a Glance

Overview

ICE scoring is the prioritization method the GIST Planning Framework uses for its idea layer. Each idea gets three values: Impact, how much it stands to improve the key metric; Confidence, how sure we are that it will have that impact; and Ease, how easy it will be to build and launch in full. Itamar Gilad credits the method to Sean Ellis and describes it as an evolution of the classic impact/effort analysis (ICE Scores). ProductPlan's glossary entry also names Ellis, the growth marketer who coined "growth hacking", as its creator.

The values are multiplied, or averaged, to give a score that sorts the idea bank into an order of priority. Gilad scores each value from 0 to 10 in his ICE guide, and his worked example multiplies them: ICE score = Impact x Confidence x Ease (Product Discovery With ICE). The number itself means nothing outside the comparison. It only says which ideas look most promising given what the team knows now.

Confidence is what makes ICE work inside GIST. Impact and Ease are always estimates, and Gilad writes that Confidence exists to reflect how much we should trust them, especially Impact, where the margin of error can be huge. A team that fills in Confidence from gut feeling gets a ranking of opinions. A team that fills it in from evidence gets a ranking that changes as step-projects produce results, which is the loop GIST is built on.

The main use of ICE in GIST is to decide what to test. Gilad warns against a version of ICE where people invent ideas, score them and then build and launch whatever ICE tells them to, and he stresses that ICE only helps if the team also does research and testing. The ranking picks the next ideas for step-projects. The step-projects then decide what gets built.

How It Works

Impact is scored against one metric: the goal the idea is linked to. Scoring impact against "the business in general" makes ideas incomparable, so the team first agrees which metric each goal uses. Gilad recommends breaking impact into parts with back-of-the-envelope calculations and checking them against existing data, because teams are poor at predicting impact (Why Impact/Effort Prioritization Doesn't Work). That article reports research from Microsoft, Netflix and Booking.com finding that at best 1 in 3 tested ideas created any measurable positive result.

Ease is usually the inverse of effort, estimated in person-weeks. It measures the effort of building and launching the idea in full, which is what the ranking needs to compare ideas fairly. Teams are also prone to underestimate effort, so Ease estimates deserve the same skepticism as Impact.

Confidence is scored from the type of evidence behind the Impact estimate. Gilad's Confidence Meter lists evidence from weakest to strongest: self conviction, thematic support, others' opinions, estimates and plans, anecdotal support, market data, customer evidence and test results (Product Discovery With ICE). In How Much Product Discovery Is Enough? he groups them into bands on a 0-10 scale: opinions score at most 0.1, assessment up to 0.5, data up to 3.0, and tests and experiments above that. His rule there is never to launch anything based on opinions alone.

The low end of that scale is deliberately harsh. An idea that everyone in the room loves, with no data behind it, still scores near zero on Confidence, and so it cannot top the ranking on enthusiasm. As evidence arrives from research and step-projects, its Confidence rises or falls and its rank moves with it. Gilad's worked example follows two ideas through several rounds of evidence, and their Confidence scores end up far apart.

ICE is recalculated at set moments: when a new idea is triaged, when the team filters ideas to work on, whenever new evidence arrives, and during periodic reviews of the idea bank (ICE Scores). Triage scores are rough, done in minutes. Scores for ideas close to being tested get more care.

ICE has known weaknesses. ProductPlan notes that it is highly subjective, that different scorers produce very different results, and that low Ease scores drag down ideas and can favor quick wins over high-impact work (ProductPlan). Written scoring guides, independent scoring and evidence-based Confidence reduce those problems without removing them.

Step-by-Step Guide

Step 1: Prepare the idea list and the scorers

Pull the ideas for one goal from the idea bank, and remove any that are clearly out of scope before the session. Invite a small cross-functional group: product, engineering, design and someone close to the data. Share the list and the goal metric in advance so people arrive having read the ideas.

Step 2: Agree the impact metric and the scales

Confirm the goal metric each idea is scored against. Write a short scoring guide for each dimension: what a high and a low Impact look like for this metric, what effort a high and a low Ease mean for this team, and the Confidence levels by type of evidence. Shared definitions are the main defense against scores that mean different things to different people.

Step 3: Score Impact

Have each person score Impact on their own before any discussion, to avoid anchoring on the first voice. Encourage a quick calculation for each idea: how many users it reaches, how much it could change their behavior, and what that means for the metric. Record the reasoning next to the score.

Step 4: Score Confidence from evidence

For each idea, list the evidence that supports the Impact estimate and score Confidence from its type, using the Confidence Meter levels. Opinions, including senior ones, keep Confidence near zero. Customer evidence and market data raise it, and test results raise it most. Write down the evidence so the next re-score starts from it.

Step 5: Score Ease

Estimate the effort to build and launch the idea in full, with an engineer in the room, and convert it to an Ease score. Note the main uncertainties in the estimate. If a much cheaper version of the idea could be tested first, record that as a note for step-project design; the Ease score itself still describes the full idea.

Step 6: Combine and rank

Multiply the three values, or average them if the team prefers a gentler spread, and sort the ideas. Look at the ranking as a starting point for discussion. Check that the top ideas are not there only because of a high Ease score, and that high-Impact ideas with low Confidence are flagged for cheap validation.

Step 7: Discuss the top of the list

Spend the discussion time on the top ideas and on any score where people disagreed widely. Disagreement usually means different information, so share it and re-score. Adjust a score only when someone brings a reason or evidence. An idea having a champion is neither.

Step 8: Choose what to test and set re-scoring points

Move the top ideas into the working set and design step-projects for them. Note which evidence would change each idea's Confidence. Re-score whenever a step-project finishes or new evidence arrives, and review the whole bank on a regular schedule.

Best Practices

• Score each dimension independently before discussing. Anchoring on the loudest person's number is one of the easiest ways to turn ICE back into opinion.
• Keep Confidence honest. Following Gilad's Confidence Meter, an idea supported only by opinions should score near zero, however much the team likes it.
• Write down the evidence behind each Confidence score. Without it, re-scoring after a test becomes guesswork.
• Treat the ranking as a recommendation. Gilad treats ICE scores as recommendations that still need judgement (ICE Scores).
• Use ICE to choose what to test. Build decisions come from the evidence the step-projects produce.
• Re-score after every step-project. A ranking that never changes means the team is not feeding evidence back.

Common Mistakes

• Inflating Confidence: Teams often rate Confidence high because they believe in the idea. Tie Confidence to evidence type, and challenge any high score that rests on opinion.
• Building whatever ranks first: ICE shows what looks most promising with current evidence. Only tests show what works. Test the top ideas with step-projects before committing to full builds.
• Scoring Impact against no metric: Without a shared goal metric, Impact scores cannot be compared. Agree the metric before scoring.
• Letting Ease dominate: Quick wins with small impact can crowd out important ideas, a weakness ProductPlan also points out. Check the top of the list for ideas that rank high only because they are easy.
• Scoring once and forgetting: Scores go stale as evidence arrives. Re-score at every triage, filter and test result.

References

• Examples: Worked examples and scenarios
• FAQ: Frequently asked questions
• Parent Method: GIST Planning Framework

Related Skills

• Building a Product Idea Bank for the GIST Ideas Layer
• Designing Step-Projects to Validate Product Ideas
• Defining Measurable Product Goals in GIST

Sources

• Itamar Gilad: ICE Scores, All You Need to Know
• Itamar Gilad: Product Discovery With ICE and The Confidence Meter
• Itamar Gilad: How Much Product Discovery Is Enough?
• Itamar Gilad: Why the Impact/Effort Prioritization Matrix Doesn't Work
• ProductPlan: ICE Scoring Model

Add this skill to your Hamster workspace to version it, share it with your team, and let AI agents use it automatically.