I’ve been pushing for more and more online makreting testing at my current job, and things are starting to move forward, which is very exciting. I’m a huge fan of testing, and there are tons of tools and resources that make online testing very easy. However, because it is becoming easier and easier to start tests, it is more and more important to establish a basic test plan prior to beginning any testing.

Testing is an important part of any online marketing initiatives. While the overall goal of testing is usually to increase conversions, which will ideally increase revenue, the real value in testing is the information you collect. What you learn from the success or failure of a test can be applied to your next test, and the one after that, and on and on. I’m finding that many people gauge conversion as the end-all-be-all metric of testing, which may often be the case. But, there are several other metrics that one can measure for success.

For instance, on a banner ad test, your CTR (click-through rate) may be your primary metric of success. An email test might also use CTR, or possibly turnover rate or open rates might be what you should be concerned with. And for landing pages, maybe lowering your bounce rate or improving your customer retention rate is the primary objective. Of course, all of these metrics will likely (and hopefully) help towards improving your conversion rate, but for the test at hand, CR may not be your top metric.

This is why it is so important to have a proper test plan prior to beginning any online marketing test. What the test plan does is establish what you are testing, how you will be testing, and what metrics you will be using to determine success or failure.

Test plans can be long, complicated documents with tons of detailed information, or they can be short, simple outlines with the high-level topics, but in any case there are 3 key bits of information that MUST be established for any test plan:

Hypothesis: Just like in scientific testing, the first thing that must be established is the hypothesis, or the preliminary question of what you are looking to learn. The hypothesis should be in question form, and be very short and simple to understand. For example: “Will changing our CTA button from red to green increase clickthrough rate?” or “Will moving the coupon code input field below the total price decrease cart abandonment?” As you can see, these are very short statements that explain what we are changing (button color or input field placement) and what we will measure (CTR or abandonment rate), without getting into the full details of how and what we need to do in order to make the test happen. Your hypothesis will drive all other information in your testing plan, and thus is the first and most important piece of information.

Prime Objective: This is where you can expand on your hypothesis with greater detail and state the specifics of what you are going to learn. For instance, using the button hypothesis above, your prime objective may look something like this: “Determine whether changing the bottom right call to action from a red color to a green color will increase or decrease click through rate on [Company Name’s] lead generation landing pages.” While it may seem like we are simply repeating the hypothesis, upon closer look, you’ll see we are providing additional details, such as where the CTA button is (bottom right) and what page we will be testing (lead generation page). This hypothesis is a pretty straightforward and simple question, so in this example, our Prime Objective is fairly simple as well, but the more complex the test, the more clarification your prime objective may need. The important thing to remember is your hypothesis should be kept as short and simple as possible (and always in question form), leaving the further detail to the prime objective section.

Metrics: Here is where you state the KPI (key performance indicator) or metric that you will gauge success or failure of your test. The metric you use should be able to answer your hypothesis. Again, using the button hypothesis example, our primary metric would be clickthrough rate. As mentioned above, our goal of any test may be to improve conversion rate, but that is not the TOP METRIC for our button test. Based on our hypothesis, CTR is really the only metric we need to look at to answer our question. Some tests may require more than one metric, and I always like to add any secondary metrics (if applicable) that may be of interest or may help in future tests, but most importantly and at minimum, the metric that addresses our hypothesis should be listed. And obviously this should be a metric that you are capable of recording and viewing the data.

A fourth bit of information that many consider to be necessary in your initial test plan is Test Period (duration of your test), but I look at a test plan as a working document, and your test period could change during the actual test (you may need to extend the time period to get significant results, or stop the test if you are seeing dramatic negative impact), and thus is not a requirement at the initial stage.

There are several other elements that are extremely helpful to have in a testing plan including Measurement System, Baseline Metric and User Path, but most (if not all) of this information will be driven by the 3 points detailed above. It’s important to note, I am not recommending that your test plan should ONLY have these three points as a complete and thorough test plan will only help with current and future tests. What I am recommending is that at minimum and first priority, your test plan needs to have these three points.

View a sample online optimization test plan here (PDF). As you can see from this example, your test plan does not have to be long and complex, but it should be thorough. And while by necessity it may have technical language, it is very helpful that this plan is understandable by any member of the organization.

I hope you found this article and the example plan helpful. Please feel free to contact us to find out if we can help you in your online marketing optimization needs.

Be Sociable, Share!

• 
•