Go to app

Reading results and picking a winner

How to read an experiment's lift and variant numbers, how EarlyBird picks the winner, and what happens when it's applied.

Written by the EarlyBird teamBETA

Last verified Nov 4, 2026

Watch the video walkthrough (opens in a new tab)

Each experiment has a results card that compares your variants side by side. Here's how to read it and how the winner is chosen.

Open the results

  • For one bidder: open the bidder and click the A/B Tests tab. The current test is at the top, and older ones are under Past experiments.
  • For all bidders: click Analytics → Experiments, then click a row.

The card starts with a line such as "Long vs short letter started on … with 60 proposals test size."

Choose a winning metric

You pick the winning metric when you set up the test. It decides which variant wins.

  • Reply rate: the share of proposals where the client messaged you. It's the closest to new work, so it's the default and the best choice for most tests.
  • View rate: the share of proposals the client opened. Use it for changes clients see before they open your proposal, such as boosting or your first line. You'll also get an answer sooner, since views come before replies.

Read the lift

The two boxes at the top show View rate lift and Reply rate lift, each marked "Best variant vs control". Lift compares the best variant with your control:

  • +25.0% means the best variant's rate is a quarter higher than the control's. For example, a 10% reply rate against the control's 8%.
  • A negative number, such as -12.5%, means every variant did worse than the control.
  • +∞ (5.0%) means the control has no views or replies yet, and the best variant has a 5.0% rate.
  • – – means there's nothing to compare yet.

Read the variant table

ColumnWhat it shows
VariantThe letter and name, with a control tag on your baseline and a winner tag once the test is completed
BidsProposals this variant sent
ViewsProposals the client opened
RepliesProposals where the client messaged you
View rateViews divided by bids
Reply rateReplies divided by bids

Your bidder sends with each variant in turn, A, B, C and back to A, so they get about the same number of bids.

Click a variant, or ⋯ → Preview, to see the settings it sends with: the Bidder algorithm, Cover letter, Answers, Terms settings, Boosting, Advanced settings and Other settings. Use it to check that your variants differ only in what you're testing.

How the winner is picked

When the test completes, the variant with the highest rate on your winning metric wins. If a variant ties with the control, the control wins, because there's no reason to change.

The winner is chosen on the numbers alone, and bigger tests give more reliable results. Aim for at least 10 proposals per variant, and treat a small lift from a small test as a hint, not proof.

Auto-apply the winner

If Automatically apply winning variant settings when experiment completes was on, the winning variant's settings become your bidder's settings when the test completes. Your bidder stays on or off as it was.

If it was off, nothing changes. To use the winner, copy its settings into the bidder's Settings tab yourself. Preview the winning variant to see exactly what to set.

Next steps

Start your next test from the new settings, and change one thing again. See Test ideas: what to test first.