Split Testing Results for Bingo Card Creator
This page displays automatically recorded A/B testing results for Bingo Card
Creator. Some of them I thought were interesting enough to do a little mini-writeup
for. The other ones are minor tweaks or failed experiments. I mostly keep them
here a) to have a convenient interface to check on my tests-in-progress and b)
to remind folks that most A/B tests fail. That's the entire point of
A/B testing: failing your way forward.
Stats geeks: All stats are delayed by up to 24 hours. If you have a burning
urge to see the numbers, mouse over any percentage on this page and it will show
you how it was calculated.
Sign Up Page Banner: On Theme or Not?
I have a page which hawks a Halloween bingo activity that has a graphical design distinct from that of this site. I tested the control signup page, which uses this site's branding, against an alternative which uses elements of the mini-site branding to reassure users that they are in the same place.
Desired conversion: A free trial sign-up
Results:
Halloween header: 34.44% winner
Standard header: 32.49%
Significance: Not significant
A/B Test Results (Live)
| Test Name | Alternatives | Significance? |
|---|---|---|
| Landing Page Hide Header And Footer | No(38.80%) winner Yes(31.47%) |
90% |
| Identify Free Trial As Such | No(12.54%) winner Yes(11.94%) |
Not significant |
| Purchasing Options Table Border | true(9.97%) winner false(8.99%) |
Not significant |
| Purchasing Page Sidebar Button | On(8.74%) winner Off(8.47%) |
Not significant |
| Purchasing Page Safety Logo Sidebar | Off(8.88%) winner On(8.30%) |
Not significant |
| Signup Form Button Creative (Ie Fixed) | Start Making Bingo Cards(35.86%) winner Bt Txt(35.56%) |
Not significant |
| Landing Page User Count | Off(28.53%) winner On(28.22%) |
Not significant |
A/B Test Results (Completed)
| Test Name | Alternatives | Significance? |
|---|---|---|
| Google Checkout Offered | Inline(13.16%) winner None(7.87%) |
95% |
| Landing Page Menu | Off(28.52%) winner On(27.44%) |
95% |
| Trial Button Design | Try Now Online(15.30%) winner Try Online Free(14.50%) |
90% |
| Purchasing Page Safety Logo | Off(7.08%) winner On(6.34%) |
Not significant |
| Landing Page Button Creative (Ie Fixed) | Start Making Bingo Cards(26.72%) winner Bt Txt(25.38%) |
90% |
| Signup Form Button Creative | Bt Txt(36.82%) winner Start Making Bingo Cards(33.66%) |
90% |
| Landing Page Button Creative | Bt Txt(27.74%) winner Start Making Bingo Cards(25.80%) |
Not significant |
| Landing Page Sidebar Wording 3 | Featured(27.40%) winner Included(25.47%) |
90% |
| Front Page Screenshot 2 | Redone(0.81%) winner Traditional(0.49%) |
90% |
| Site Logo No Bots 2 | Classic(11.05%) winner 99designs(9.92%) |
99% |
| Offer Downloadable Software 2 | On(0.26%) winner Off(0.24%) |
Not significant |
| Offer Downloadable Software | false(0.00%) winner true(0.00%) |
Not significant |
| Front Page Screenshot | Redone(0.00%) winner Traditional(0.00%) |
Not significant |
| Site Logo No Bots | 99designs(0.00%) winner Classic(0.00%) |
Not significant |
| Location Of Javascript | Head(0.01%) winner End Of Body(0.01%) |
Not significant |
| Site Logo | 99designs(1.03%) winner Classic(0.97%) |
Not significant |
| Allow Incentivized Invites | Off(4.43%) winner On(4.35%) |
Not significant |
| Trial Card Limit | 15(1.52%) winner 12(0.65%) |
95% |
| Landing Page Sidebar Wording 2 | Included(0.00%) winner Featured(0.00%) |
Not significant |
| Landing Page Testimonial 2 | Glenda(25.12%) winner Dave(23.38%) |
90% |
| Horizontal Conversion Button Styling | Traditional(0.31%) winner Redone(0.28%) |
Not significant |
| Landing Page Direct Ask | Next Page(23.39%) winner On Page(23.22%) |
Not significant |
| Landing Page Button Test 3 | Traditional(15.34%) winner Redone(12.60%) |
90% |
| Landing Page Sidebar Wording | Included(17.16%) winner Featured(16.27%) |
Not significant |
| Landing Page Testimonial | Glenda(17.27%) winner Dave(16.17%) |
Not significant |
| Color Conversion Button Landing Page | Blue(18.64%) winner Red(16.59%) Purple(14.80%) Orange(11.07%) Green(10.00%) |
(Too early) |
| Conversion Button Styling Landing Page | Traditional(17.01%) winner Redone(14.14%) |
95% |
| Registration Use Lightbox Landing Page | (16.28%) winner Ibox&Width=528(15.02%) |
Not significant |
| Registration Use Lightbox | Ibox&Width=528(0.35%) winner (0.34%) |
Not significant |
| Landing Page Sidebar Links | On(18.48%) winner Off(17.80%) |
Not significant |
| Landing Page Full Spiel | Off(18.65%) winner On(17.46%) |
Not significant |
| Conversion Button Styling | Traditional(0.79%) winner Redone(0.72%) |
90% |
| Shopping Cart Paypal Verbiage | Off(3.51%) winner On(2.06%) |
90% |
| Instant Access In Shopping Cart | Off(4.13%) winner On(1.32%) |
99% |
| Suggestions On Dashboard | No(63.00%) winner Yes(56.93%) |
90% |
| Redo Guest Test | Guests Off(0.36%) winner Guests On(0.31%) |
Not significant |
| Signup Madlibs Versus Standard | Standard(27.03%) winner Madlibs(21.58%) |
95% |
| Order Of First Operation Suggestions Fixed | Create From Scratch First(54.34%) winner Use Premade Card First(53.82%) |
Not significant |
| Main Flow Button Styles | Image(59.25%) winner Html(56.18%) |
90% |
| Offer Reward For Survey | Yes(2.24%) winner No(0.94%) |
99% |
| Offer Survey | Yes(1.07%) winner No(0.93%) |
Not significant |
| Order Of First Operation Suggestions | Use Premade Card First(55.06%) winner Create From Scratch First(53.83%) |
Not significant |
| Express Path For New Trialers | Dashboard First(58.55%) winner Skip Dashboard(53.75%) |
99.9% |
| Main Flow Submit Buttons | Image(59.81%) winner Html(58.67%) |
Not significant |
| Lp Featured Or Baby Shower | Featured(19.55%) winner Shower(18.93%) |
90% |
| Get The Code Link | italics(3.53%) winner bold(3.09%) |
Not significant |
| Ask For Sale On Print Page | false(0.42%) winner true(0.26%) |
Not significant |
| Let Users Choose Fonts | false(84.41%) winner true(81.33%) |
95% |
| Let Users Pick Colors | false(83.56%) winner true(82.19%) |
Not significant |
| Lightbox On Landing Page | false(0.00%) winner true(0.00%) |
Not significant |
