How to A/B Test and Optimize Marathon Training

I’ve been training for the Boston Marathon all winter (you can read all about why over in my first blog), and over the course of these months I’ve realized how much that training shows up in other aspects of my life. In my job at SiteSpect — an optimization and A/B testing solution — I deliver a platform that helps organizations test and optimize their digital channels. In talking about what we do as a company, I realized I’ve done quite a bit of A/B testing and optimization of my own. In fact, it turns out my journey to a successful marathon is pretty similar to my clients’ journeys to improving their brands’ customer experiences. These past months I’ve been figuring out how to optimize my running and training for race day, test strategies, learn from failures, and come out with a defensible ROI.

Figure Out Your Metrics

My first step in optimizing my training for Boston was determining my metrics. Finishing time is usually the first one that jumps out — much like “purchase” might pop as a KPI for a retailer. But finishing time is only one piece of a larger puzzle. The metrics I actually care the most about are: Am I healthy and uninjured? Is my pace relatively steady throughout the run? Am I wearing the right clothes for the weather? Is the run enjoyable?

Just like a brand might measure the bounce rate on a particular landing page, I measured factors like how tired I was after 10 or so miles, how hot or cold I was, and if I had enough energy to push through. In the same way that conversions can drive a bottom line, these metrics will determine how I feel at the finish.

A/B Testing, 1-2-3

We had one hot weekend in February, where instead of the frigid temperatures we’d been used to it was above 50 degrees (practically tropical). Still, I wore my tights, thermal shirt, jacket, and hat, and by the end I was a whole new kind of miserable. Why? This was all part of my A/B testing strategy. Past years’ race-day temperatures have ranged from mid-40s to mid-70s, and training through winter can’t prepare you for that. So, I created variable conditions in order to learn how I would perform at different temperatures.

The same goes for nutrition. Any distance runner will tell you how important nutrition is to race day and long run success, but no distance runner can tell you what you want to know: what combination is going to work for you. There are as many types of running fuel as there are running shoes (that’s a lot), so the only way to know what works is to test.

Failing Big

Despite all the A/B testing and optimizing, you just can’t avoid the occasional failure. Failures every runner faces include: missing a training run for any number of reasons, cutting a run short, not fueling properly (see above), eating the wrong meal the night before a long run, not dressing appropriately for the weather, getting nagging injuries that call for rest. But what do all these failures lead to? A more likely success on race day.

Part of A/B testing and optimizing is learning what doesn’t work, so that you can discover what does. By nature, when you compare two variants with the intent to optimize, you hope that one performs better. That means one variant is going to lose. But that’s why we train, and that’s why we test: so that those failures happen in controlled circumstances where you can turn them into optimization.

See You Monday

We’ll see what happens on Marathon Monday. However it goes, it’s sure to be a memorable day. And because of everything I’ve learned at SiteSpect, I know I’m going into it prepared.