Test Analytics

Test Analytics

“Innovate or perish” has become a truism; an accepted part of the business lexicon. And to innovate, we must test—or sift through the options and find those worthy of full scale implementation.

Surprisingly, there has been very little innovation over the years when it comes to test analytics. Even though testing is vital to innovation, most companies use the same old same old method of analyzing test results. Statements like “sales were up 7% in the test region” are de rigueur. But what if the test region was already headed for a 6% sales increase? Then maybe the test was not all that successful, and maybe we need to keep looking for a better innovation.

Forecasting can greatly increase the sophistication of test analytics. Good forecasts detect trend and seasonality in the test market, and project them into the future. So, a great way to assess a test is to measure the degree to which it beats forecast, not just prior month. Even better, you can compare actual versus forecast for the test and non-test (i.e., control) regions. If the control regions were close to forecast, and the test region beat the forecast by a sizable margin, you have a clear winner.

The chart pictured above tells a powerful story. The lower green line is actual monthly sales for the control regions. The red line is forecasted sales for the same control regions. Notice how closely they track together. This means the forecasting method is accurate. The upper black line is actual sales for the test region. The teal line is the test region’s forecast. Notice how sales bumped up, then gradually came back down. This tells a clear story: the test generated a sizable but short-term jump in sales.

Click here for more information on the use of forecasting in innovation and test analytics.

For an advanced course on sales forecasting, including its role in test analytics, click here.