Testing – What’s the Big Idea?

Is yours the typical company launching 2-5 tests a month, struggling to eke out more from your marketing optimization program, and wasting critical marketing resources of your team and website traffic? That’s the result of not focusing in on the Big Idea!

How much should you be testing?

A mid-size company can easily handle 30-50 tests a month. The reason most companies never get there is because they waste so many cycles on what I call “slice & dice” optimization on poorly designed landing pages.

Let’s consider the following test which I found “in the wild” — and which I find ironic because it is for a service offering a marketplace of landing page designers.

Version A

 

Version B

Depending on how you want to define your testing variables these two landing pages have around a dozen changes. I hope this is not what you want from your landing page designers. You don’t need someone to create endless variations of every variable in order to succeed in your marketing optimization efforts.

Can you identify all of the variables being proposed for testing? (I’ll share with you my list next time.)

For now, let’s assume that for each of these variables on these landing pages, you test just 2 variations even if more may be warranted. I’ll show you what the problem with that approach is. (Note: I’ll make use of Google’s Website Optimizer “Test Duration Calculator” to estimate the numbers, but you could easily do this by hand or with a calculator or simple spreadsheet):

I don’t know the true stats for this page but they don’t really matter in order to illustrate the challenge.

Let’s assume the following:

  • For this test we have 12 variables and are testing 2 variations of each = 24 variations total
  • The page gets a 1000 page views a day (it is in beta after all).
  • We will graciously assign the page a 10% current conversion rate.
  • We expect to get a 30% lift in conversion.
  • We will assign 100% of our traffic to the page.

That means it take more than 108 days — over 3 months! — to complete this test for these simple landing pages. That’s a whole-heck-of-a-lot of visitors and a whole-heck-of-a-lot of time consumed to get one test completed.

Testing Landing Pages: A More Efficient Way

The way we teach testing, there are probably 3 variables worth testing (variables that communicate to a visitor) on this page. Let’s assume the same 2 variations for each, though to be frank, for one of the variables I would want at least 3 or 4 variations if we were doing this test for an actual client. But for the sake of simplicity let’s keep everything the same.

So now we would have:

  • We have 3 variables and are testing 2 variations for each = 6 variations total
  • The page gets a 1000 page views a day (it is in beta after all).
  • We will graciously assign the page a 10% current conversion rate.
  • We expect to get a 30% lift in conversion.
  • We will assign 100% of our traffic to the page.

This test would be over in just under 18 days, a scant 2 ½ weeks.

Which way seems more efficient?

Suffering from Conversion Optimization Fatique

Should you test for variables that seem to really matter to visitors versus testing virtually random variations of elements in the hope something gives you a little lift? You may achieve some gains — that’s why this practice is so common — but you’ll burn out waiting for the results. This is why so many marketing optimization efforts fizzle out over time.

Next time I will share the variables I found comparing these landing pages and discuss the 3 elements I would test on this current page.

In order to give you a leg up on identifying the variables on your own, I’ll give you a question as a framework. When was the the last time you looked at a page and said to yourself, “The layout is horizontal and not vertical, so gosh darn it, I can’t buy from this page?” Vertical vs. horizontal layout on landing pages could matter as a display of information issue if you are trying to change a lot of what is above and below the fold. But that really wasn’t the case in this example. It’s just a waste of time and effort unless or course you have no real idea what will move the needle for customers – and in that case ANY test is better than nothing. Maybe.

Instead of trying to test endless variations of minutiae we teach companies to look for the big ideas that impact customer experience and buying process. The smaller variations we can always come back to after the Big Ideas establish directionality.

Can you find the big ideas for optimization (or that should be tested) in this example?

Please share if you think others would benefit.