ShopNBC Finds Hidden Treasures Through Multivariate Testing
ShopNBC.com is operated by ValueVision Media, a multi-channel, direct-to-consumer retailer that sells general merchandise to consumers through television, the Internet, and direct mail. Founded in 1990, the Company operates the third largest television home shopping network in the United States — ShopNBC — and is a leader in upscale television home shopping. ShopNBC.com offers customers a rich shopping experience filled with Web-Exclusive offers, customized search features, interactive buying guides, videos and auctions. ShopNBC also offers a top quality and rich TV-like experience thru their newly launched initiative, ShopNBC.TV — the excitement of live demonstration selling combined with a rich internet experience.
With a reputation for providing visitors with an exciting and efficient shopping experience, ShopNBC needed a way to quickly test site changes without disrupting their online store and without burdening valuable IT and marketing resources.
ShopNBC’s Internet marketing team turned to SiteSpect, both for its non-intrusive testing technology and professional services. SiteSpect enabled ShopNBC to design and implement A/B and multivariate tests for validating new creative ideas and optimizing promotional offers, product copy, and user interface elements.
- Using SiteSpect, ShopNBC’s Internet marketing team tested the impact of variations on copy, content, and layout changes without any involvement from their IT department.
- A recent test designed to optimize the product detail pages on ShopNBC.com resulted in a 16% increase in the average order value across all departments and product lines. The optimizations from this test have the potential to yield many millions of dollars in incremental revenue.
- Simultaneously testing multiple promotional offers enabled ShopNBC to quickly react to changes in customer demand and inventory availability.
- In addition to increased revenue through the optimization of ShopNBC.com, SiteSpect helped ShopNBC to avoid lost revenue from potential changes that, had they been implemented blindly, would have hurt sales.
It Starts with a Hypothesis
When ShopNBC first started testing, their goal was to answer a question whose answer they thought they already knew. The Internet marketing team’s initial goal with testing was to determine whether their instincts were correct. As their hypotheses became more complex, and they progressed from A/B testing to multivariate testing with SiteSpect, ShopNBC learned that combinations of variations could sometimes produce unexpected positive results.
The Power of Interaction Effects
What ShopNBC experienced was one of the benefits of Multivariate Testing: Interaction Effects. By testing multiple Web-site elements (“test factors”) and content variations simultaneously, the Interaction Effects reflected the impact that variations had on each other when presented in combination with each other. In many cases, these winning combinations far-outperformed each of the individual variations — the effect of interactions that occurred beyond the additive effects that ShopNBC had anticipated.
And The Winner Is . . . . .
ShopNBC recently designed a test to optimize the layout and content on the product detail pages of ShopNBC.com. Their hypothesis was that by drawing attention to key elements of the page, they could encourage more customers to initiate and complete purchases that they were considering. ShopNBC suspected that several key features were getting lost on the page and that they were losing sales as a result.
Working with SiteSpect’s Professional Services Team, ShopNBC identified five key factors that they hypothesized would influence the actions of users who were contemplating a purchase.
Variations that were tested on the product detail pages:
- Location and size of the Add-to-Cart Button
- Headlines identifying cross-sell items
- Styles of tabs that link users to product details
- Color, size, and style of Clearance and Limited Time pricing offers
- Highlighting of payment options for qualified buyers