Eric J. Hansen: Hi I'm Eric Hansen, CEO of SiteSpect, and I'm here today to talk with Mike Brown from Vegas.com. Mike, tell us a little bit about Vegas.com and what you do there.

Mike Brown: Sure. Vegas.com is the largest city website and we sell everything Vegas — any place to stay, any airline, anything to do.

Eric: OK. And it's Vegas.com and LasVegas.com, right?

Mike: This is correct.

Eric: Great. So how long as the Vegas.com company been testing and optimizing its websites?

Mike: I think we've been testing basically from day one. Initially it would be serial testing, "Let's change this one day and next week we'll do that." I've been there about six years, and shortly I came we launched a homegrown A/B tool. It was the best thing since sliced bread. In retrospect it was pretty limited. It could test two versions of something. It took weeks of software development to set it up. Unfortunately it messed with our SEO results while we were running it. So you wouldn't want to be testing everything all the time because the search engines couldn't crawl us then.

Eric: OK, I got you. So to start with, you basically built your own in-house testing system. Did you have some success with it? I would assume you had some initial success or you might have given up on testing all together.

Mike: Absolutely. We became rapidly addicted to this concept of iterating mini-iterations to get to change. It was very painful to do the iterations, and long.

Eric: Would you say that testing is something that's gotten a lot of attention in the company at different levels of the organization?

Mike: Absolutely. In fact, we've created an entire group, the optimization group that I oversee, that has dedicated staff and work flow. All they do is focus on the user experience. "How can we make it better? How can we optimize it?" It's been really successful to prioritize those resources for doing that.

Eric: When it comes to site optimization testing, generally speaking, what are your objectives? What are you trying to do? What are your goals with all this effort that you put into running tests?

Mike: Sure. Initially, it was very single focused, "How can we drive more sales? How can we get more customers?" Very hunter-gatherer. "How can I bag one more trophy?" Over time I think we've learned that that's a very short focused way of thinking about it, and it's more about solving users' problems. User experience is not the opposite of conversion. They're actually the same thing. So if you make it easier, faster, funner for the user to use your website, they're going to buy more stuff from you. It's totally symbiotic.

Eric: So what are some things that you've tested — parts of your site, or promotions, or offers, things like that?

Mike: Years ago, and I think it's still a factor but more so back then, there was a lot of concern about security. The common thinking was OK, tell people that you're secure down on your billing page. "You're secure. Your information isn't going to get hacked." One of our early wins was taking that kind of assurance and explaining how we were taking care of folks' security up on the phone, putting it on the home page, putting it in different layers, addressing a user need. "Am I going to get hacked? Am I going to have my credit card stolen?"

In that same vein, what do users need, we used our analytics products, our Coremetrics, our Tealeaf. We saw lots of users were doing hotel searches and they would put it in their cart, and then they would go back to search, and then they would look at some more and they'd put it in their cart. And they'd just keep pogo sticking back and forth. Or they would leave search entirely and go to a different part of the website and go look at our hotel detail pages.

So what was the problem? The problem was they weren't getting what they needed in search so they were going deeper into the fold to figure out the real price, or they were leaving the funnel entirely to go look at other content that wasn't there.

So addressing that need of bringing the content into the sales process, we really beefed up our hotel search results, put all of our media, all of our videos, all the pictures, and real final pricing, which was something nobody else in the industry was doing. You would see the daily rate but you wouldn't really see what taxes and fees were until you got deeper into it. That was a big risk for us. We didn't know if that was going to suddenly make us non-competitive because our prices were going to be a little bit higher because we were including fees and nobody else would.

So it was really important when we did that to roll it out with SiteSpect because we weren't going to bet the company on this theory that if you really told people the pricing that they'd buy more stuff. So we rolled it out slowly through SiteSpect, we kept the old version up and running. Both of those things, putting the content in the search process and the real pricing, were enormously successful. But we could track them uniquely and see which one did what and do it with low risk.

Eric: So good success, basically help people find what they're actually looking for. Seems like a novel idea maybe, but at the end of the day, when you go to a website generally you have some sort of a goal in mind. So help people get to that goal and you'll reap the rewards. So you are a SiteSpect customer, been using the tool for about three years.

Mike: A little more than three.

Eric: How did you first hear about SiteSpect and what attracted you to the offering?

Mike: A long time ago, Eric. I remember a Forrester report. It was when multivariate testing was first coming out. What is this multivariate stuff? I remember A/B. What's that? So I remember that Forrester report. I had a product manager do a vendor search. It was you guys (SiteSpect), and Offermatica, Optimost. Those two folks are still around, different incarnations, but even then they were JavaScript tag-based. One of them was full-service. We wanted to be experts at this. We didn't want to outsource it to somebody. But as we dug into the JavaScript solution, it really seemed like we wouldn't be getting out of that IT dependency that we were already in with our homegrown tool. There was a lot of setup for individual tests. And the thing that really tickled our fancy with you guys was this promise that "OK, you set it up and then IT doesn't get in your way." We loved that idea, we tested it and it was real. So that was the initial value prop on when we picked SiteSpect, and to this day it's one of the best things about it.

Eric: So the distinction, perhaps, or maybe the key distinction, we use this term "non-intrusive," so we don't require the tags and we don't require a tagged project. Would you say that that's the key differentiator for Vegas.com for the product?

Mike: Yeah. I would say 90, 95 percent of our tests, they are conceived of... actually ideas come throughout the company. But they're executed by Marketing, by Optimization, by Customer Experience. 95 percent of them don't need any software or QA. Really, really complex stuff, we want to have it QA'd. We want to make sure it works. But most of the stuff we do and some of our biggest wins have required nobody else but some basic web designer stuff that our own group supplies.

Eric: That's fantastic.

Next to Part 2 of the interview transcript