Split Testing Results for Bingo Card Creator

This page displays automatically recorded A/B testing results for Bingo Card Creator. Some of them I thought were interesting enough to do a little mini-writeup for. The other ones are minor tweaks or failed experiments. I mostly keep them here a) to have a convenient interface to check on my tests-in-progress and b) to remind folks that most A/B tests fail. That's the entire point of A/B testing: failing your way forward.

Stats geeks: All stats are delayed by up to 24 hours. If you have a burning urge to see the numbers, mouse over any percentage on this page and it will show you how it was calculated.




Sign Up Page Banner: On Theme or Not?

I have a page which hawks a Halloween bingo activity that has a graphical design distinct from that of this site. I tested the control signup page, which uses this site's branding, against an alternative which uses elements of the mini-site branding to reassure users that they are in the same place.

Desired conversion: A free trial sign-up
Results:
Halloween header: 34.44% winner
Standard header: 32.49%
Significance: Not significant




A/B Test Results (Live)

Test Name Alternatives Significance?
Shopping Cart Paypal Verbiage On (33.33%) winner
Off (0.00%)
Not significant
Instant Access In Shopping Cart Off (25.00%) winner
On (0.00%)
Not significant
Suggestions On Dashboard Yes (66.67%) winner
No (60.00%)
Not significant
Redo Guest Test Guests Off (0.28%) winner
Guests On (0.25%)
Not significant

A/B Test Results (Completed)

Test Name Alternatives Significance?
Signup Madlibs Versus Standard Standard (27.03%) winner
Madlibs (21.58%)
95%
Order Of First Operation Suggestions Fixed Create From Scratch First (54.34%) winner
Use Premade Card First (53.82%)
Not significant
Main Flow Button Styles Image (59.25%) winner
Html (56.18%)
90%
Offer Reward For Survey Yes (2.24%) winner
No (0.94%)
99%
Offer Survey Yes (1.07%) winner
No (0.93%)
Not significant
Order Of First Operation Suggestions Use Premade Card First (55.06%) winner
Create From Scratch First (53.83%)
Not significant
Express Path For New Trialers Dashboard First (58.55%) winner
Skip Dashboard (53.75%)
99.9%
Main Flow Submit Buttons Image (59.81%) winner
Html (58.67%)
Not significant
Lp Featured Or Baby Shower Featured (19.55%) winner
Shower (18.93%)
90%
Get The Code Link italics (3.53%) winner
bold (3.09%)
Not significant
Ask For Sale On Print Page false (0.42%) winner
true (0.26%)
Not significant
Let Users Choose Fonts false (84.41%) winner
true (81.33%)
95%
Let Users Pick Colors false (83.56%) winner
true (82.19%)
Not significant
Lightbox On Landing Page false (0.00%) winner
true (0.00%)
Not significant