The Best Testing Fail of the Week: If It Worked Once It, It May Not Work Again

By Sally Hall

May 10, 2018


This week’s fail was informed by a major success. A change that boosted conversions on one page ended up causing a very different result when reproduced.

Less Distraction

A large clothing retailer conducted some A/B tests and determined that removing suggested items from its Cart page increased conversions. The suggestions ended up serving more as a distraction than a help. The brand decided to take that lesson to its product detail pages, and A/B tested a variation without any product suggestions there either. On the variation pages, they saw a 5% decrease in conversions, 75% drop in RichRelevance suggested item click-throughs, 3.7% increase in search requests, 6% decrease in product detail page views, and a 2.3% decrease in add to carts.

Glass Half Full

Because of the Cart page design results, this A/B test seemed like a no brainer. Had they taken results from a different A/B test and applied them here, this brand would have seen a major revenue loss. Luckily, because of A/B testing, they were able to learn what works and adjust accordingly.


Read More Best Testing Fails             Read More Case Studies

To learn more about SiteSpect, visit our website.




Sally Hall

Sally Hall

Sally Hall is an Optimization Consultant at SiteSpect, guiding SiteSpect users on the road to optimization. She has more than 10 years of experience as a web optimizer and testing manager for enterprise brands. She is based in Austin, Texas.

Suggested Posts

Subscribe to our blog: