A/B tests on emails, WhatsApp steps and broadcasts, with the winner sent
Try up to five versions of an email, a WhatsApp template or a broadcast. Split people evenly or by shares, or test first and send the winner to everyone after. You get a plain verdict on how sure the result is.
PLANSScale only
Built; the statistics were checked on their own and tests are written. A/B tests have not yet run on a live system with real traffic.
The problem it solves
- Two people argue about the subject line and nobody tests it.
- A test is run, but nobody can say whether 6 % against 4 % is real or luck.
- The losing version keeps going out because no one switched it off.
How it works
- Open an email or WhatsApp step, or a broadcast, and its A/B Test tab. Version A is the step itself; add B to E with what is different.
- A new email version starts as a copy of the step's email, so you change only what differs. A WhatsApp version is another approved template.
- Choose How to Test: Split Evenly, Split by Shares, or Test, Then Send the Winner with Winner By (replies, positive replies, clicks, opens or reads).
- With Test, Then Send the Winner, each version goes to People per Version in the Test (100). After Wait for Answers (48 hours) the best rate wins. A broadcast tests on 20 % of the audience and waits 24 hours.
- The winner goes to everyone after; losing versions are marked Lost the Test. The owner gets A/B Test Decided with the verdict.
Everything it does
Where you can test
- Send Email and Send WhatsApp steps of sequences and journeys
- WhatsApp and email broadcasts
- Random Split steps in journeys: paths A to D with shares (A and B 50 % each to start with). People and goals are counted per path
How to test
- Split Evenly (default); Split by Shares (Share of A 50 %, each version's share); Test, Then Send the Winner
- A person always gets the same version and the same path
- Steps: People per Version in the Test 100, Wait for Answers 48 hours
- Broadcasts: Test On 20 % of the audience, Wait for Answers 24 hours; the rest are held (Waiting for the Winner)
- At most five versions, A to E
Winner By
- Replies (default), Positive Replies (sorted Interested), Clicks (email links or WhatsApp buttons), Opens (email, when counted), Reads (WhatsApp)
- Only a measure the channel can count is accepted
- Free text sent inside the WhatsApp 24-hour window is not part of a test
The verdict
- Results table: version, sent, answers and rate, with the leader marked
- A clear winner needs at least 95 % sure and 30 people a version. Otherwise the best one still wins and the verdict says "a close call"
- No answers at all: A goes on to everyone
- Decided every hour by itself, or Decide Now
- Growth: Write with AI can also write version B of each email
Who uses it
Marketing UserAdds versions, sets how to test and reads the verdict.
Marketing ManagerDecides early with Decide Now when the numbers are enough.
Owner of the journey or broadcastGets the result.
Where you use it
- Web: the A/B Test tab of email and WhatsApp steps and of broadcasts
- Web: Outreach > Results, WhatsApp Results and Broadcast Results grouped by Variant
- Not in the phone app: this is done on the web. The SmartMy Marketing app covers chats, contacts, posts, leads and campaigns
What you control
Settings
- How to Test: Split Evenly
- Winner By: Replies
- People per Version in the Test: 100; Wait for Answers: 48 hours (24 for broadcasts)
- Test On: 20 % (broadcasts)
- Random Split: Path A 50 %, Path B 50 %
Notifications
- A/B Test Decided: to the owner of the journey, in-app and by email, at once
- A/B Test Decided, Broadcast: to the owner of the broadcast, in-app and by email, at once
- Each person can switch off the email channel; in-app always arrives
Questions
Can the same person get two versions?
No. Each person always gets the same version of a step, and the same path of a Random Split.What does "a close call" mean?
The best version won, but the numbers are not enough to be 95 % sure it is really better. It is still sent to everyone after.Can I pick the winner myself?
Press Decide Now: the best version so far wins and goes to everyone after.Is A/B testing in Growth?
No. A/B tests and Random Split are in Scale.Related features
OUTREACHEmail sequences over many mailboxes, with tracking and one-click unsubscribeCold and warm outreach as plain, personal emails. Each sequence sends from a pool of your mailboxes, a little at a time. It keeps follow-ups in the same thread and stops for a person when they reply. Every email carries an unsubscribe link and the one-click unsubscribe header.OUTREACHWhatsApp sequences: free text in the 24-hour window, over several numbersPut WhatsApp steps in the same sequence as your emails. Each step sends its approved template, or your own free text when the person wrote in the last 24 hours. New chats can be spread over several numbers within Meta's daily limits.CAMPAIGNSBroadcasts on WhatsApp and email, at a set speed, with follow-upsSend one approved WhatsApp template or one email to a whole audience, now or at a set time. The speed keeps within your number's limit and your mail server's rate. Afterwards, follow up only the people who did not read, click or reply.CAMPAIGNSJourneys: triggers, waits, If / Else, tags, tasks and goalsAutomate what happens after someone enters. Wait, send an email or WhatsApp message, and branch on a reply or a tag. Tag people, give someone a task, and end at your goal.OUTREACHReplies sorted by AI, and reply rulesEvery reply to a sequence or journey gets a category such as Interested, Not Now or Wrong Person. It is sorted by AI or by the button pressed. Your reply rules then start another sequence, give a tag, set a task or, with CRM, make a lead.AI AND SELLINGSequences written with AI, and a first line for each personDescribe your offer and who it is for, and the AI writes a whole outreach sequence with the waits between the emails. It also writes a personal opening line for every person on a lead list, from facts you already have.
See your inbox and campaigns on it.
Tell us which channels your customers use, how leads reach you and how many people answer them. We will connect a channel with you and run your first journey.