Make website testing and optimization an ongoing priority, focusing not only on creative content elements, but also on page load speed.
With the help of SiteSpect's non-intrusive platform, Mozilla found an easy way to start testing and optimizing its site. As opposed to previous attempts with other products (including a homegrown solution), Mozilla is now able to run tests on an ongoing basis — at least two concurrent experiments at any given time. Mozilla's overall results with SiteSpect have been compelling. The first two A/B tests saw double-digit improvements to engagement of 33% on the First Run page and more than 50% on the What's New page.
In addition to enhancing visitor engagement, Mozilla wanted to increase its conversion rate on Firefox downloads. The conversion rate on the main Firefox product page (http://www.mozilla.com/firefox) was already high, but Mozilla wondered what it would take to boost that number even higher. To find out, a 32-level multivariate test was run on the following page factors:
To test how each element contributed to Firefox downloads, the SiteSpect test engine automatically produced 32 page variations, delivered them to end users, and measured behavior.
The test concluded that only one element positively impacted the download conversion rate: the download stats box. All other elements had either no effect or actually drove conversions down. In fact, the simplest overall page performed the best, boosting the conversion rate by 2.3% (99% confidence), increasing Firefox downloads by 1.7 million per year.
But why did the simpler variation perform better? One hypothesis was that more users converted because the page loaded faster. To test this hypothesis, Mozilla analyzed the distribution of page load times for visitors who downloaded Firefox and visitors who didn't.
The results showed that half of the 'downloaders' loaded the landing page in less than 2.10 seconds while half of the 'non-downloaders' loaded the landing page in less than 3.67 seconds, which is 75% slower.
Reducing the page load time by 2.2 seconds increased conversions by 15.4% (99.99% confidence), which translated into an additional 10 million downloads per year. Interestingly, when segmenting the results by browser, the increase in speed had an even more dramatic effect on users of Microsoft Internet Explorer (Firefox's target audience for conversion), who experience an increased conversion rate of 27.6%.
Clearly, speed influences end user behavior. By testing performance optimizations with SiteSpect, Mozilla was able to produce a winning combination of content that reduced page load time and boosted conversions dramatically.