What is a growth experiment, and how do you run one?

What is a growth experiment, and how is it different from an A/B test?

A growth experiment is a test you run on purpose to find out if one change grows a number you care about. It has four parts: one change, one number, a set length, and a clear keep-or-stop result at the end.

The point is to stop guessing. If you change five things in one month, you will not know which one moved your sign-ups. If you have ten customers, an experiment can be as simple as sending 30 personal emails with a new offer and counting how many people book a call.

An A/B test is one kind of growth experiment. It splits people into two groups at the same time and shows each group a different version. A growth experiment is the wider idea: any planned test of one change against one number, with or without a split.

A/B tests need a lot of traffic to give a trustworthy answer. Most early-stage startups do not have it yet. That is fine. Compare your number before and after the change instead, as long as you change only one thing and run it for full weeks.

• What it tests

  Growth experiment:

  Any one change: a channel, an offer, a message, a price

  A/B test:

  Two versions of the same thing, shown at the same time

• How it compares

  Growth experiment:

  Before vs after, or version A vs B

  A/B test:

  Group A vs group B, split at random

• Traffic needed

  Growth experiment:

  Works with small numbers

  A/B test:

  Needs lots of visitors to be trusted

• Best for

  Growth experiment:

  Early-stage startups finding what works

  A/B test:

  Products with steady traffic fine-tuning pages

How do I write a hypothesis and fill in a growth experiment template?

Use this sentence: "If we [make this change], then [this number] will [go up or down] because [this reason]." That format comes from Appcues' growth experiment template, and the reason at the end is the most important part.

The reason forces you to explain why you think it will work. With it, even a failed test teaches you something, because you learn that your reason was wrong. Hold the problem steady and expect to change the how many times.

Write it down before you start, with the number you expect. If you decide what counts as a win after the results come in, you will talk yourself into a win every time.

Then put it into a template. Copy the table below into a doc or a spreadsheet and fill in one row per test.

• Weak: "Let's try a new homepage."
• Strong: "If we change the homepage headline to name the exact problem we solve, then free sign-ups per week will go up, because visitors will know right away the product is for them."
• Real: At Theanna, Nomiki Petrolla, the founder, guessed people were canceling because they did not know what to do after signing up. She tested live group onboarding, and churn dropped from 12% to 7%. Her rule: track the numbers tied to the guesses you are making.

• Goal

  What to write:

  The one goal this test supports

  Example:

  Get our first 10 paying customers

• Hypothesis

  What to write:

  If we..., then..., because...

  Example:

  If we offer a free setup call on the pricing page, then paid sign-ups will go up, because people are unsure how to start

• Number to watch

  What to write:

  One number, from a tool you trust

  Example:

  New paid customers per week, from Stripe

• Starting number

  What to write:

  Where the number is today

  Example:

  1 new paid customer per week

• Win line

  What to write:

  What result means keep

  Example:

  3 or more per week by the end

• Length

  What to write:

  Start and end date

  Example:

  3 weeks

• ICE score

  What to write:

  Impact x Confidence x Ease

  Example:

  7 x 6 x 8 = 336

• Weekly check-ins

  What to write:

  The number each week and a short note

  Example:

  Week 1: 2. Week 2: 2. Week 3: 4

• Result

  What to write:

  Keep, stop or adjust, and why

  Example:

  Keep. Setup calls turned into paid customers

How do I run a growth experiment, step by step?

Pick one goal, write a hypothesis, choose your number, set the length, run it without changing anything else, then keep it or stop it.

1. 01

   Pick the one goal that matters most right now

   Choose one goal for this stage: more sign-ups, first paying customers, more revenue, or more sales calls.

2. 02

   Write down your starting number

   Before you change anything, record where the number is today, like 12 sign-ups a week. Without a starting point you cannot tell if something worked.

3. 03

   Write the hypothesis

   Use "If we..., then..., because..." and write the result you expect. Keep it to one change so you know what caused the result.

4. 04

   Pick the length and the win line

   Decide how long it will run and what result means keep. For example, run it for three weeks, and keep it if sign-ups go up by a clear amount over your starting number.

5. 05

   Launch it and leave it alone

   Do the change and do not tweak it halfway through. If you change the test in the middle, you cannot trust the result.

6. 06

   Check the number every week

   Look at your number once a week against where it started. Write a short note on what you see, but do not end the test early because of one good or bad week. As Nomiki Petrolla, Theanna's founder, puts it: if you see a bad number, do not panic and do not kill it. Wait for the full picture.

7. 07

   Decide: keep, stop or adjust

   On the end date, compare the result to your win line. Keep what worked and make it part of how you run the business. Stop what did not. If it was close, change one thing and run it again.

8. 08

   Write down what you learned

   Save the hypothesis, the result and one sentence on why. Over time this list becomes your own playbook.

What are some growth experiment examples for early-stage startups?

Good early-stage experiments are small, quick to set up, and tied to sign-ups, first customers, revenue or sales calls. Here are examples you can start this week. Each one is a single change with a single number to watch.

• A real one: Nomiki Petrolla ran Meta ads to a free webinar for Theanna. She made the signup form easier. Registrations jumped from 148 to 653, but the share who showed up fell from 32% to 9.3%. She almost stopped it. Then the sales came in: 24 buyers instead of 14, and day-one revenue of $6,578 against $5,994 in ad spend. She had been watching show rate when the number that mattered was ad spend per sale.
• Another: before building Theanna's AI agent, the team shipped a simple fake one. People used it so much they broke it, and that told them to build the real thing.
• Start with the experiments closest to money. For most early startups that means talking to more of the right people.
• Brian Balfour's Bullseye Framework suggests testing a few channels cheaply, then putting your effort into the one that shows promise.

• More sign-ups

  Experiment:

  Rewrite the homepage headline to name the exact problem you solve

  Number to watch:

  Sign-ups per week (Google Analytics)

• First customers

  Experiment:

  Send 30 personal messages to people who match your best customer, with one clear ask

  Number to watch:

  Replies and calls booked

• First customers

  Experiment:

  Offer a founder-led onboarding call to every new sign-up

  Number to watch:

  Sign-ups who become paying customers

• More revenue

  Experiment:

  Add a yearly plan next to the monthly plan

  Number to watch:

  Revenue per new customer (Stripe)

• More sales calls

  Experiment:

  Put a booking link in your email signature and on the pricing page

  Number to watch:

  Calls booked per week (Calendly)

What is an ICE score, and how do I prioritise experiments?

An ICE score ranks experiment ideas by Impact, Confidence and Ease, each scored from 1 to 10. Sean Ellis, a well-known growth leader, uses it to sort ideas, and says the best test to run is one that scores high on all three.

Impact is how much it could move your number. Confidence is how strong your evidence is. Ease is how simple it is to set up. Multiply the three scores together and rank your ideas from highest to lowest.

Treat ICE as a tie-breaker, not a rule. The scores are your own estimates, so a small gap means little. ICE also ignores how many people an idea reaches. If reach varies a lot between ideas, the RICE model adds it as a fourth score.

• Personal outreach to 30 ideal customers

  Impact:

  8

  Confidence:

  7

  Ease:

  8

  ICE score:

  448

• New homepage headline

  Impact:

  6

  Confidence:

  6

  Ease:

  9

  ICE score:

  324

• Yearly plan on pricing page

  Impact:

  5

  Confidence:

  5

  Ease:

  7

  ICE score:

  175

• Paid ads on a new channel

  Impact:

  7

  Confidence:

  3

  Ease:

  4

  ICE score:

  84

How long should a growth experiment run, and how many should I run at once?

Run each experiment for full weeks, never just a few days, and pick the end date before you start. Weekday and weekend visitors act differently, so a test that skips part of a week can fool you.

Slow numbers like paid customers often need more weeks. Fast numbers like replies can show an answer sooner. Appcues warns that stopping too early gives misleading results, and running too long lets other things muddy them.

On how many: you can run several at the same time as long as each one has its own number and they do not touch the same thing. Two tests on the same pricing page will mix up each other's results. Two tests on different channels will not. Sean Ellis has described his team picking three to five ideas to test each week. Theanna recommends running 5 to 10 at the same time, so you learn faster and one failure does not stall you.

What should I do when most of my growth experiments fail?

Expect most experiments to fail, and keep going. Even large companies with big testing teams lose most of their tests. Optimizely found that only 12% of experiments produce a clear win on the main number, and GrowthBook puts the industry average success rate at about 33%.

A failed test is still useful if you wrote down the reason in your hypothesis. You learn which reason was wrong, and your next idea gets better. Sean Ellis has said the tests that did not work are often what lead to the big breakthrough ideas.

• Stop the test on the end date. Do not let a dead test keep running.
• Write one sentence on why you think it did not work.
• Check if the change was too small to move the number. Optimizely found that tests making bigger changes deliver the strongest gains.
• Pick the next idea from your ranked list and launch it the same week.