Experimentation Program Management: Ideas You Can Execute Immediately

Efficiently managing experimentation efforts is essential for campaign success. However, many organizations lack these efforts which leads to chaos and poor results. 

Experimentation programs are broken down into ecosystems with many sub-ecosystems embedded within. This is largely due to the multidisciplinary nature of the work. Think about an experimentation program as a machine. Within that machine, there are many functions and parts. All require regular care and maintenance otherwise, it has the potential to break down. 

In the business setting, different roles and teams are like the different functions and parts of the machine. No car engine is ever haphazardly put into a car, driven, and never maintained again, right? (That’s assuming you could even start the engine at all in the first place.) Well, it’s the same concept with experimentation programs.

This is where experimentation program management comes into play. Tactically, it usually breaks down into many different areas, as program management usually requires tackling from many different angles. And that’s what I’m going to cover in this article. As it’s heavily related to governance, we’re going to dive into this section first. 

Mapping Governance

There are five main governance models, and you can read more about those in my latest article here. A team must choose a model to start building – almost always with the goal of scaling the program over time. Sometimes teams can’t work toward what they want right away, so I always tell them to choose a current-state model and a future-state model (transition plans can always be sorted out later as needed). We need to know the key players and peripheral players in the game to start deeply understanding six primary concepts: who, what, when, where, why, and how. If we don’t understand these, it’s much more difficult to use other experimentation program management tactics effectively.

Once there’s a governance model, then a governance mapping workshop in a tool like Miro is extremely helpful. This tells us the players in the game within the organization. It’s just like mapping a regular org chart except it’s specific to the experimentation program, which often looks different. This is especially helpful for visual learners, who I find to be many people (including myself). Once we know the players, we can figure out their positions using RASCI charts.

Clarifying Roles with RASCI Charts

RASCI charts enter the scene now, and they tell us the position of each player in the governance model. This provides clarity in regard to who is doing what and who’s responsible for what at the end of the day. (Credit to Miro for the template.)

After that’s complete and we know the players and positions, now we have a structure to work from and build off of moving forward. We’ve applied much more organization and intention giving everyone more clarity and confidence. As a result, efficiency and effectiveness ultimately increase.

Governance mapping and RASCI charts are absolutely two critical starting points for program management that I would encourage most (if not every) team in CRO to keep in mind. From there, you should pile on other tactics, or even do it simultaneously to some extent. 

My Grab Bag of Go-To Tactics

By other tactics, here’s what I mean… there are several tactics that I pull from regularly (many are self-explanatory):

More Information About Two of My Favorites

Playbooks

Segueing into one of these tactics from governance and RASCIs, let’s talk about playbooks. Both governance maps and RASCIs should be rolled into your program’s playbook as assets that everyone can reference (because you’ll have awesome diagrams and charts from your workshops). 

Playbooks are like handbooks. Every program should have a playbook with documentation about its practices, rigor standards, processes, systems, etc. It should have everything. That’s soup to nuts from research, ideation, and hypothesis formatting to development practices, QA checklists, and statistics notes. That way there’s no question as to how things are done. It’s there for historical reference and onboarding. It’s for anyone to self-serve as much as possible at any time. It should be consistently updated as needed and considered a living document. It should be set up to serve CRO nerds in the weeds, higher-ups, and anyone who’s just peripherally involved. Farfetch has a great example here.

Reporting

Documentation consistency is almost always an improvement opportunity for teams. There should be templates for test reporting, newsletters, research readouts, etc. that are used consistently by all team members. This ensures efficiency, predictability, easier scalability, proper historical reporting, and proper data entry, and it also makes everyone’s lives easier.

Example of a robust individual test document template

Final Thoughts

Well, there you have it. Implement some of these tactics in your process, and you’ll level up right away. Know that successful experimentation program management takes time to build and is a continuous effort—it’s always critical to success. Teams should always start tackling this as quickly as possible with their other efforts. If you wait and don’t prioritize, you could end up paying the price later on.

Learn how SiteSpect can help you reach these goals. Book your personalized demo today to get started.