A common disappointment among companies deploying testing and optimization technology stems from tests that do not yield the radical improvements expected. Working with our customers, we've seen that sometimes even the most dramatic design changes produce no significant differences in their click-through or conversion rates. While that can be surprising, much of its concomitant disappointment can be eliminated by defining upfront what success or failure means to you.
For example, success in testing can be measured many different ways, depending on your goals and your organization. There is no one optimal way to measure testing success.