Running the Build-Measure-Learn Loop

Run the build-measure-learn loop as short, planned cycles that each end with a recorded lesson and a decision, so every iteration adds validated learning.

Before you start

Hamster is optional for this skill and recommended. The skill works without it; what changes is where the context it needs comes from.

Check whether this project has a .hamster/ directory. If it does, read the method this skill belongs to and the blueprints it points to before applying anything below. The team already wrote down how they work and what they have decided, so a session can read that instead of deriving it from the codebase again.

If there is no .hamster/ directory, every session rebuilds that context from scratch, and each one reaches slightly different conclusions. Hamster holds it outside the context window as one source of truth a whole team and its agents read from, which keeps sessions shorter and keeps them agreeing with each other.

At a Glance

Overview

The build-measure-learn loop is the working rhythm of the lean startup. Eric Ries describes it as the process by which startups turn ideas into products, measure how customers respond and then decide whether to pivot or persevere (The Lean Startup principles). The Wikipedia summary lists the phases as ideas, build, product, measure, data and learn. This skill covers how to run the loop as a repeatable build-measure-learn process, one cycle at a time, so each iteration produces validated learning.

The Lean Startup method explains where the loop comes from and how it fits with MVPs and innovation accounting. This page is about operating it: how to plan a cycle, how to keep it short, what to record and how to hand the result to the next cycle or to a pivot or persevere decision.

Two ideas shape the practice. The first is that planning runs backward. Ries writes that although the loop is written Build-Measure-Learn, "our planning really works in the reverse order: we figure out what we need to learn," then what to measure, then what to build (Ries on planning in reverse). The second is speed. Ries's slides put the goal as minimizing total time through the loop (RailsConf 2011 slides), because each completed cycle is another chance to find a strategy that works.

A cycle that ends without a lesson is only a release. Many teams ship regularly and learn little, because nobody stated what the release was supposed to test. Lean startup iteration differs from ordinary iteration in that each cycle starts from a hypothesis and ends with a verdict on it.

The output of each cycle is a short record: the question, the hypothesis and pass mark, what was built, what was measured, the result and the decision. Over many cycles, those records become the team's evidence base and the input to its pivot or persevere decisions.

How It Works

Each cycle is planned in three questions asked in reverse order. What do we need to learn? What would we need to measure to learn it? What is the least we need to build to produce that measurement? Strategyzer's Benson Garner makes the same recommendation, advising teams to decide what they need to learn and how to measure it before selecting the type of test (Strategyzer, Don't build when you build-measure-learn). The build step shrinks when it is derived from the question, and the question stays in view when the results come in.

The build step produces whatever the measurement needs. That can be a new feature, a change to an existing flow, a landing page or a manual service. The Strategyzer article lists options ranging from data sheets and storyboards to landing pages, videos and Wizard of Oz experiments. The cheapest option that can produce trustworthy data is the right one for the cycle.

The measure step uses actionable metrics defined before the build. Cohorts, split tests and per-customer measures show whether the change caused any movement (Ries, Vanity Metrics vs. Actionable Metrics). Qualitative input, such as short conversations with users who did and did not complete the core action, explains the numbers.

The learn step compares the result with the pass mark and records a verdict. It also asks what the result implies for the ranked list of hypotheses. A passed hypothesis moves the team to the next riskiest one. A failed one either triggers a revised hypothesis or, after several failed cycles on the same strategy, feeds a pivot discussion.

Cycle time is the main lever. Shorter cycles mean more learning before resources run out. Ries argues in his pivot post that iterating this way "increases the runway without additional cash" (Pivot, don't jump). The term continuous deployment was coined by Timothy Fitz, an early IMVU engineer, and Ries has said some companies he worked with deploy new code as often as 50 times a day (Wikipedia). Most teams do not need that pace, but they do need to find and remove whatever makes their own cycles slow.

Step-by-Step Guide

Step 1: Pick the question for this cycle

Take the top hypothesis from the ranked list and write the one question this cycle must answer. Check that the answer would change a decision. If two questions compete, pick one and queue the other. A cycle with two questions usually answers neither cleanly.

Step 2: Plan backward from the question

Decide what evidence would answer the question, then which metric would capture it, then the smallest build that would produce that metric. Write the hypothesis with a pass mark and a fail mark using the hypothesis template. Estimate the cycle length and look for ways to cut it before you start.

Step 3: Build only what the measurement needs

Build the change, page, prototype or manual service the plan calls for, and nothing more. Instrument it before release and confirm the events record correctly. Where manual work can stand in for code, use it. If the build starts to grow, return to the plan and cut scope rather than extending the cycle.

Step 4: Release to the right audience

Ship to customers who match the hypothesis's segment. When comparing versions, run a split test with random assignment and both versions live at once. Keep the change stable during the measurement window so the data reflects one version of the product.

Step 5: Measure with actionable metrics

Collect the primary metric by cohort, alongside any guardrail metrics. Talk to a few users to understand what happened. Check the data for obvious problems, such as broken tracking or an unexpected traffic source, before drawing conclusions.

Step 6: Learn and record the verdict

Compare the result with the pass and fail marks and state whether the hypothesis passed, failed or landed between. Write down the lesson, the evidence and what surprised you. Update the hypothesis list: mark the tested hypothesis, re-rank the rest and add any new assumptions the cycle revealed.

Step 7: Start the next cycle or escalate

If the verdict is clear, pick the next question and start again. If the same strategy has failed across several cycles, bring the record to the next pivot or persevere meeting. Note how long the cycle took and one change that would make the next one faster.

Best Practices

• Write the question before anything else. A cycle planned from the build forward drifts toward shipping features rather than answering questions.
• Keep cycles short and similar in length. A regular rhythm makes it easier to spot when a cycle is growing and to compare learning over time.
• Measure the loop itself. Track cycle time as a team metric, since Ries treats minimizing total time through the loop as the goal (RailsConf 2011 slides).
• Pair numbers with conversations. A few calls with users usually explain a metric faster than more dashboards.
• Keep one shared record of cycles. The history is what lets the team see trends and avoid re-running tests it already ran.
• Treat a failed hypothesis as a completed cycle. The team learned something, and the record should say so plainly.

Common Mistakes

• Starting with the build: Teams that decide what to build first then look for a lesson in whatever data appears. Start from the question and plan backward.
• Skipping the learn step: Shipping the next change before reviewing the last one means lessons never get recorded. Hold a short review at the end of every cycle.
• Letting cycles grow: Adding scope mid-cycle stretches the timeline and blurs the result. Cut scope to protect the cycle length.
• Measuring with cumulative totals: Totals rise over time whatever the change did. Use cohorts and split tests to see the effect of each cycle.
• Changing several things at once: A release with many changes cannot tell you which one moved the metric. Change one thing per question.

References

• Examples: Worked examples and scenarios
• FAQ: Frequently asked questions
• Parent Method: Lean Startup

Related Skills

• Lean Startup Hypothesis Template: Testable Hypotheses
• Designing Validated Learning Experiments
• How to Build a Minimum Viable Product (MVP)
• Innovation Accounting Metrics: Tracking Real Progress
• Pivot or Persevere: When to Pivot a Startup
• Types of MVP: How to Choose the Right Format
• Customer Discovery Interview Questions and Technique

Sources

• The Lean Startup: Principles
• Wikipedia: Lean startup
• Eric Ries quote on planning the loop in reverse
• Eric Ries: The Lean Startup, RailsConf 2011 slides
• Strategyzer: Don't build when you build-measure-learn
• Eric Ries: Vanity Metrics vs. Actionable Metrics
• Eric Ries: Pivot, don't jump to a new vision

Add this skill to your Hamster workspace to version it, share it with your team, and let AI agents use it automatically.