Part 4/6: 5 Questions You Need to Answer to Optimize an Experience

Welcome to installment 4 of my 6-part series, “5 Questions You Need to Answer to Optimize an Experience. In the first 3 blogs, I introduced my approach to optimization, and I walked through ways to answer the questions, Who is the user?, and What experience should you show a user? Today, we’re on question 3: How will you deliver the experience? Let’s dive in.

Many organizations inadvertently take for granted one of the most important steps in the optimization journey: deciding how to deliver an experience. For A/B testing, personalization, and product recommendations this can make all the difference in your conversion rates and customer experience. Your delivery method can affect time to market, the cost of implementation, and site performance. Ultimately, these all boil down to the return on investment you’ll see for each experience. But, most people don’t weigh the benefits of different delivery options.

What Do We Mean By, “How Will You Deliver the Experience”

First, let’s look at how an experience gets created. There are many factors in creating a user experience for a site: several teams working to build, release, A/B test, and analyze the experience. An organization might use any of the following methods to deliver an experience:

• Code directly into the existing site
• Use CMS
• Implement redirects to new pages
• Release experience via Feature Flag
• Use an A/B testing tool 
• Use a personalization tool

Each of these methods work to deliver an experience, and each has its own set of pros and cons. The right choice for the experience will depend on time and effort to implement, performance capabilities, and the intended duration of the experience.

Time and Effort to Implement

The first criteria to consider is the amount of time it takes to create an experience. Of course, the longer the production takes, the greater the cost, and the fewer experiences you can release. So, how can you maximize the customer experience while minimizing time and effort?

Simple changes can often be done in the visual editor of your optimization tool. For example, if you need to change the image on your homepage, your marketing team can deliver that experience via your A/B testing tool, rather than having your development team create a permanent coded change to your site. 

For more involved changes, it may be more efficient to work with your CMS and direct users to the new experience. A lot of SiteSpect users, for example, leverage a combination of CMS changes with our rewrite capabilities (our higher-performance alternative to redirects). This allows for quick changes to more complex experience variations. 

If your client-side tool requires a lot of JavaScript to function, using developers to build an experience may be the least up front effort for you.

As Product Managers, these are the criteria we consider when prioritizing items on a roadmap, but most of us typically only consider one means of delivering the experience to validate ideas (developers). But, understanding whether an experience could be delivered in a different way to achieve your goals quickly is critical. Validating fast and cheap is the key to prioritization and velocity. This is true for optimization teams, user experience teams, merchants, marketers, product teams, or any other team that builds customer experiences.

Performance 

There is a proven relationship between conversion and site speed. If your method for delivering an experience adds too much latency, it will negatively impact conversions. This will corrupt your data for both A/B testing and personalization since you won’t know whether a negative user behavior was based on the experience itself, or the lag it had in delivery — not to mention, it holds you back from maximizing conversion.

A classic example of a delivery method unintentionally hurting your data and experience is relying on redirects. With redirects, the multiple trips the browser needs to make back to the origin adds significant latency, where a simple rewrite returns the right experience the first time. Tag based A/B testing and personalization tools can often run into performance issues as well, especially at scale. So, it’s important to factor that performance impact into your prioritization strategy when determining how to deliver an experience.

A Real User Monitoring (RUM) tool will help you determine the performance impact of any given experience.

The Duration of the Experience

Finally, when determining how to deliver your experience you should consider how long it will be live. If it’s a quick promotion or temporary change with a definite end date, you may not want to invest a lot of time creating the experience, and rather look to your A/B testing or marketing tools. Ideally “temporary” experiences can leverage a template that can be reused. Otherwise, you would need to see a significant lift to justify the effort. However, if you are creating a long-term change, like a rebranding or redesign, it makes sense to invest more up front, and consider other delivery methods including phases of A/B testing and iteration, development, and functionality A/B testing.

Approaching the question, How will you deliver the experience? this way gives you greater opportunities for optimization. Join me next week as I tackle the question, Was the user’s experience any good?

To learn more about SiteSpect, visit our website.