In the old days, it was all about simple direct response marketing. Today, experimental design is allowing marketers to get trickier with their marketing initiatives – pinpointing what works and what doesn’t for their target audience. These new marketing and tracking schemes rely on mathematical formulas and specific variables.
What Traditional Testing Tells You
Traditional testing uses data mining to gather up data to present it to a marketing department for analysis. This method of analysis is “after-the-fact” and pretty easy to do. Variables like price, promotion, and message can all be looked at, and different combinations can be tried to improve results.
The resulting process is sometimes called “A/B split testing” because one variable is tested at a time. It’s effective, but it’s also slow and a lot of variables will be left out of the testing process. Plus, since it gives you results after-the-fact, you have to spend money to figure out what works. You may find that none of your variables panned out, in which case you wasted money on the initiative. This is where experimental design comes in.
What New Experimental Design Tells You
Experimental design, as a concept, isn’t new, but the way it’s being implemented in. Because computers can run infinitely complex algorithms, they can generate results that are based on an almost infinite number of variables.
This isn’t the same as data mining, however, because the variables are hypothesized prior to the marketing test.
Multiple possible combinations are set up and graphed on a grid. Then, using fractional factorial design, a subset of variables can be tested. Why a subset? Because, even if there are 3 different variables, yielding 16 different possible results, only 8 of those may make a difference in a sales context.
So, for example, an algorithm can be devised that will calculate crossover between different combinations of price, message, and promotion. If you have four different prices you want to test, three different promotions, and three main messages or benefits you want to test, the formula can determine how many of those combinations are basically “overlap,” telling you which variables would potentially make a difference in your sales.
Perhaps there are only 8 different variables to test. Instead of testing them one at a time, you test them all.
When the test is completed, you’ve essentially done 8 different A/B split tests and figured out which variable pulls in the most sales. But, that’s not the only thing you can test. You can also see which sales method results in the most net sales – net of returns.
By extending the test out 30 days (or however long your refund period is), you can hone in on the right message that creates “stickiness,” thereby increasing long-term sales and net profits.
Companies, like Winnipeg SEO Services take care of optimizing your website and various marketing channels so that you can conduct an effective test. After all, without a mailing list or website traffic, you’ve nothing to test your variables on.
Why Experimental Design
Experimental design’s biggest benefit is how it saves you time and money. When you test individual variables, you spend significant resources. You could also waste a lot of time if (or when) a percentage of those variables just don’t produce any measurable results.
There is one downside to using this model though. It relies on “main effects” models. In other words, it assumes that the impact of one variable will have negligible effects on another variable. And, while this is usually a reasonable assumption in direct marketing, it’s not always true.
And, the times when it’s not true tend to be times when it makes a fairly substantial difference.
Another thing you might miss with this method is how various marketing channels affect outcome. For example, if you’re prone to do email marketing, you may miss the impact of handing out your fliers or some other marketing material (like brochures) in person.
At the end of the day, marketing is mostly science, but like any science it doesn’t always give you concrete answers. It gives you variables to test. And, when those variables yield data, you can rely on it in the context of the experiment you designed.
At the same time, when variables aren’t known, there’s no method that can fully capture the impact they might have.
When your company’s main concern is saving money and optimizing results for known variables, this method usually produces superior results. However, it shouldn’t be the “be-all-end-all” of marketing. It’s one component of a successful marketing solution.
Kyle Gustin has been an SEO consultant for the past 6 years, specializing in high end SEO for businesses and professionals who deserve an edge.
Featured image credit: ShutterStock
