So I've only ever done A-B tests. Swipe various pages, find a winner, do a few tests on modifications until it's profitable, then just focus on testing traffic and optimization.
Recently I got my hands on some testing software, lpoptimizer, which if nothing else makes the whole A-B setup so much faster. It's making me want to do a whole lot more testing, especially to try my hand at multi-variate testing.
But I'm having trouble deciding how best to go about implementing it, at what stage of a campaign, or if even at all.
Let me give you an example.
Campaign #1
$1.X CPA
simple page
conversions: hundreds/day, pretty consistent conversion rates
Campaign #2
$30-60 CPS depending on specific product
tons of stuff on page.
conversions: volatile, sometimes 15 sales one day, 7 another. Somewhat averages out over a week though.
Campaign 1 would seem like a perfect fit for multi-variate testing once things are at least-break even.
Whereas Campaign 2 however I can't envision getting any sort of conclusive data if I just go off of conversions. I've run A-B tests in the past where I changed the wording of the main headline, or the main CTA button design and after a thousands spent over a few weeks there was no clear winner. What can be done in the case of a campaign like that?
Does it ever make sense to run multi-variate tests on just non copy elements like colors, buttons, and general design in hopes of finding the highest possible CTR? Or is that too optimistic the CR would remain unchanged?
I guess another way to phrase the issue with #2 would be "what's the most cost effective way to test and improve LPs for CPS products?", assuming you're only getting a dozen or so sales a day?
Would building squeeze pages to capture emails be the logical step to gather much more conversion data? I'm not terribly familiar with that side of things.
Apologies for the scatterbrained post. Got ideas bouncing everywhere.
I'm actually working on a case-study about multivariate testing just now, so watch this space!
In the two cases you've describe above, I'd say that Campaign #1 is a perfect candidate for multivariate testing. Particularly if you're using full factorial testing, multivariate requires a LOT of data, which you have. If you've got the budget and you're confident the campaign has promise, I wouldn't even say you need to wait for break-even.
Campaign #2 sounds like a nightmare for multivariate. I would stick with vanilla A/B testing there.
I'd strongly advise against testing just on CTR: even on non-copy elements, you might find that designer or colour influences CVR in a non-obvious way, and what optimises CTR actually reduces CVR in the long run. Colour psychology is very much a thing, for example.
However, building a squeeze page and doing multivariate testing on that might be a more effective way to go. You'd have to split-test the squeeze page against your current funnel first, though, and check it's actually an effective sales tactic! But if it is, then you're in a situation where multivariate could really make a difference.