How to Cure Priority Paralysis: Using an Action Priority Matrix

Sometimes, having too much choice leaves you paralyzed. This is especially true if you work in optimization. Website stakeholders from all over your company are generating lots of A/B test ideas because they know that A/B testing live traffic yields powerful results. They’ve also heard the buzz that you and your optimization team have proven that you can run more A/B tests in less time.

It’s great to have all these ideas, but how do you filter the demands and separate the high-priority A/B tests from those that can wait? Don’t let yourself become overwhelmed by this challenge. Use these recognized strategies for getting your priorities straight.

Basic Prioritization Strategies

Avinash Kaushik, author of “Web Analytics 2.0,” notes that you should work on the money-makers first. On his website, Occam’s Razor, Kaushik says that “To have a direct impact faster, lower the bounce rate and reduce acquisition cost.” He cautions against obsessing about every change, but instead suggests that you focus on increasing profitability.

Rich Page offers a similar suggestion and takes it a step further. In his book, “Web Optimization: An Hour a Day,” Page advises you to organize your list of A/B test ideas and rate them in terms of both the likely conversion lift value and the difficulty of implementation (also known as LoE: Level of Effort). “The test ratings will be essential for helping you and your senior executives prioritize the order of your upcoming tests,” says Page.

The Action Priority Matrix

A diagramming technique called the Action Priority Matrix adds another dimension to the basic strategy and maps it on a grid. According to TimeAnalyzer, the principle behind using this technique is that you score each activity you want to complete on two scales: first on the impact the activity will have (the y-axis) and second on the effort involved (the x-axis). Plotting each activity on the Action Priority Matrix lets you see the projects that give the greatest return on your effort and helps you adopt the most effective approach for that activity:

Table 1: Action Priority Matrix, Source: TimeAnalyzer.com

• Quick Wins (High Impact, Low Effort): These are the most attractive projects and give you a good return for relatively little effort. Focus on these as much as you can.
• Major Projects (High Impact, High Effort): While these give good returns, they take a long time to complete, meaning that one Major Project can crowd out many Quick Wins. If you’re starting a Major Project, make sure you complete it quickly and efficiently and that you move on as soon as you can.
• Fill Ins (Low Impact, Low Effort): Do these if you’ve got spare time, but drop them if something better comes along.
• Hard Slogs (Low Impact, High Effort): Avoid these. Not only do they give low returns, they steal time better spent elsewhere.

To use the Action Priority Matrix, TimeAnalyzer suggests that you:

1. List the activities that you’d like to complete.
2. Score them on impact (from 0 for no impact to 10 for maximum impact) and then on effort involved (from 0 for no real effort to 10 for a very major effort).
3. Plot the activities on the Action Priority Matrix.
4. Finally, choose your activities wisely.

Prioritization Based on Importance

Another matrix that helps you make tough decisions objectively is explained in the guide Project Prioritization: A Structured Approach to Working on What Matters Most (PDF) by the Office of Quality Improvement at University of Wisconsin, Madison.

This matrix helps prioritize complex or unclear issues when there are multiple criteria for determining importance. The benefit of the matrix is that it provides a consistent method for evaluating options and takes some of the emotion out of the process by quantifying the decision with numeric rankings. This approach appeals to analytical leaders who value a data-driven approach to decision making.

The guide outlines five steps for creating a prioritization matrix based on importance:

1. Determine your criteria and rating scale. The guide provides generic criteria (see below), which you can supplement with marketing and product-specific criteria, such as New Release, Response to Competition, Senior Executive Mandate, and so on.
   

   Table 2: Prioritization Matrix, Source: “Project Prioritization: A Structured Approach to Working on What Matters Most” (PDF), Office of Quality Improvement at University of Wisconsin, Madison.

2. Weight the criteria based on your values, strategic direction, organizational goals, and available resources.Weighting examples:
   • Required Service or Product: Weight = 5
   • Strategic Alignment: Weight = 4
   • Value to Customer: Weight = 4
3. Create the matrix.
4. Score your projects and prioritize them based on the criteria. Review each project and rate it on each of the criteria. Next, multiply the rating for each criterion by its weight and record the weighted value. After evaluating the project against all of the criteria, add up the weighted values to determine the project’s total score.
5. Discuss the results and prioritize your list. After you score the projects, review the results with the stakeholders. Develop a master list of the prioritized projects and leave room for change that is bound to come.

Whatever strategy you choose, always consult with peers, colleagues, and stakeholders before deciding what A/B tests to run. Taking an ad-hoc or shotgun approach before researching strategic plans and requirements may undermine your credibility.

Are you undaunted by the deluge of A/B test requests? Share your prioritization techniques below!

Explore Case Studies

To learn more about SiteSpect, visit our website.