The Best Testing Fail of the Week: If It Worked Once It, It May Not Work Again

May 10, 2018


This week’s fail was informed by a major success. A change that boosted conversions on one page ended up causing a very different result when reproduced.

Less Distraction

A large clothing retailer conducted some A/B tests and determined that removing suggested items from its Cart page increased conversions. The suggestions ended up serving more as a distraction than a help. The brand decided to take that lesson to its product detail pages, and A/B tested a variation without any product suggestions there either. On the variation pages, they saw a 5% decrease in conversions, 75% drop in RichRelevance suggested item click-throughs, 3.7% increase in search requests, 6% decrease in product detail page views, and a 2.3% decrease in add to carts.

Glass Half Full

Because of the Cart page design results, this A/B test seemed like a no brainer. Had they taken results from a different A/B test and applied them here, this brand would have seen a major revenue loss. Luckily, because of A/B testing, they were able to learn what works and adjust accordingly.


Read More Best Testing Fails             Read More Case Studies

To learn more about SiteSpect, visit our website.




Sally Hall is an Optimization Consultant at SiteSpect, guiding SiteSpect users on the road to optimization. She has more than 10 years of experience as a web optimizer and testing manager for enterprise brands. She is based in Austin, Texas.

Suggested Posts

Businessman working on laptop. Protection network security computer and safe your data concept. Digital crime by an anonymous hacker

SiteSpect Synopsis: Privacy & Personalization

Team work for graphic design working on wood table with computer tablet electronic pen and color chart

June Visual Editor Update

Rear view of businesswoman with curly hair drawing diagram on whiteboard when preparing for presentation

Optimization for Product Teams