Case Study
CreditCards.com Optimizes Lead Generation While Reducing Risk
CreditCards.com is the leading online marketing and publishing organization devoted to helping consumers make informed choices about credit cards. In 2018, over 30 million consumers have used our site to search for their next credit card. Our websites and products offer news, advice and tools – all designed to help consumers in their search for a new credit card, or to better manage their existing credit cards.
Challenge
With more than one million unique visitors to its site every month, CreditCards.com needed to find a way to optimize product offers from its partners for maximum lead generation as well as reduce the risk of rolling out new designs, all while not impacting its search engine optimization efforts (SEO).
Solution
CreditCards.com found the solution in SiteSpect, whose non-intrusive multivariate testing technology allowed the company to implement tests extremely quickly with no negative impact to organic search rankings.
“The first pilot test was up and running in ten minutes. Getting started with SiteSpect could not have been easier.”
John Stansbury, Director of Analytics and Testing, CreditCards.com
CreditCards.com is the leading online credit card marketplace, bringing consumers and card issuers together. The company’s mission is to provide consumers with the largest variety of credit card offers online, and to enable smart selection and use of cards by offering news, advice, features, and tools.
In support of that mission, the website acts as a free online resource where consumers can compare hundreds of credit card offers by category. More than one million unique visitors access CreditCards.com each month.
The site lists offers from leading credit card issuers and banks, each linked to a secure online application. Categories of cards offered range from low-interest credit cards, credit cards with rewards programs, airline credit cards, and cashback credit cards to instant-approval credit cards as well as prepaid debit cards, among others. The company also makes its content and tools available to several partners, such as NASDAQ.com, MSN, AOL, Kiplinger.com, Forbes.com, and TheStreet.com.
In Search of a Solution
CreditCards.com got its start in multivariate testing in 2007. John Stansbury, Director of Analytics and Testing, went to work looking for a testing solution that did not impact organic search engine rankings.
As most folks know, achieving and maintaining high organic search engine rankings requires a significant investment of time and resources, and companies should avoid making any changes that may prevent search engine spiders from fully crawling their websites. Unfortunately, content that is rendered solely through JavaScript is less likely to be crawled by search engines, thereby losing its SEO value – which makes JavaScript-based testing solutions sub-optimal.
SiteSpect automatically excludes search engine spiders from seeing and indexing test variations. Instead, when spiders visit, they are shown the original, non-test content as it exists on the web server. Thus, organic search engine rankings are never affected by SiteSpect.
Based on this transparency to search engines, and strong recommendations from reference calls, CreditCards.com chose SiteSpect’s enterprise solution for multivariate testing and site optimization. “The first pilot test was up and running in ten minutes,” Stansbury recalls. “Getting started with SiteSpect could not have been easier.”
“SiteSpect is a competitive differentiator for us because we can test what works with our issuing partners, such as marketing copy, card art, and offer copy.”
John Stansbury, Director of Analytics and Testing, CreditCards.com
Maybe a Little Too Easy
Stansbury cautions that testing with SiteSpect is so easy, he needed to take a step back. “We opened the floodgates and ran so many tests that it was a little difficult to accumulate statistically significant results and communicate the results of all those tests. So, we decided to focus on building the testing program itself,” Stansbury notes. As a result, CreditCards.com now uses a testing brief that specifies each new test as well as a reporting template that describes the variations, screen shots, results versus the control, etc. “Issuers in particular love seeing this report,” he says.
CreditCards.com typically runs two types of tests:
- Internal tests focusing on new tools, designs, and user experience
- External tests involving issuers and new card offers
Discovering What Works
“SiteSpect is a competitive differentiator for us because we can test what works with our issuing partners, such as marketing copy, card art, and offer copy,” Stansbury notes. “For example, we just completed a test on an interstitial page between our website and issuers’ sites. We wanted to test whether consumers preferred all the information in the same window, whether the timing of the handoff between websites impacted conversions, and the impact of different offer treatments. To our surprise, what we currently have implemented is exactly what consumers prefer. The control version won! We would have made a huge mistake changing anything without testing it. This was very validating and allows us to focus on other things. It takes the whole discussion on the effectiveness of that page off the table.”
Figure 1: The homepage of CreditCards.com, the leading online credit card marketplace.
. . . And What Doesn’t
One of the less frequently talked about benefits of testing is the mitigation of risk. “Not only do we use testing to optimize our offers from our issuing card partners, but we also use it to avoid introducing new designs that users just don’t like, thereby greatly reducing risk. This protects both the company and our customers,” Stansbury asserts. “For example, as marketers, we want to constantly update the design of the site – but every time we’ve tested new designs, they’ve failed. It would have cost us a lot of leads if we’d rolled out the new designs without testing them first.” Stansbury emphasizes that testing is not just about discovering what works, but also about what doesn’t.
On the Technical Side
Stansbury says that because SiteSpect allows CreditCards.com to use their own cookie ID, they can join data from SiteSpect to their backend database and see all the test results within the session.
“Another fantastic feature is SiteSpect WATTS®, which gives us a 360 degree view of our tests,” Stansbury concluded. “We easily populate Omniture SiteCatalyst variables with information about SiteSpect test factors and variables.”
SiteSpect Web Analytics Tag Transformation and Segmentation (WATTS) allows users to view and segment multivariate testing data within any major third-party analytics system. WATTS offers a 100% real-time capability for exposing more than 20 new visitor and session-level segments by dynamically injecting data into web analytics tags, URLs, cookies and metadata.
With SiteSpect WATTS, users can:
- View and segment the results of their A/B and multivariate tests within their existing third-party web analytics system.
- Transform existing page tags into new tags (to trial or migrate between systems).
- Dynamically inject tags into a site for instant “no-IT” web analytics implementation.
