Hey guys,
I'd like to know more about your philosophy on testing new offers, especially ones that don't have much of a track record.
In my mind I divide testing methods into two camps: Shallow and Deep Testing.
Shallow testing would be taking an angle that you're confident in and applying it to a large number of offers in order to find one that sticks. Conversely, deep testing would be using an offer you're very confident in and trying a ton of different angles to make one work.
I'm becoming more and more aware of how little significant data I collect before I make a decision on the viability of an offer. In fact, I think it's probably nearly impossible to get "true" data on something as simple as a short-form lead gen offer without spending hundreds of dollars to test across different nations, cities, region, interests, age rages, genders, landing pages, etc. In some regards I think that I'm taking a tiny portion of all significant data in an offer and hoping to get lucky in having applied the right angle early.
As an example, consider a testing process where you have ten age groups with ten images, two different headlines, two different ad copies, three nations, and five different landing pages. That's 6,000 different combinations of ads and landing pages. We'll assume that you need a minimum of 30 clicks to begin to reach significance on CTR and 30 clicks as well on CR (although this one is usually axed earlier). Just testing those combinations for CTR with significant data would be 90,000 clicks, or $13,500 with 15 cent clicks.
This is a bit extreme, but you can see my concern.
I'd love any thoughts on this. What does everything think about the fleeting nature of testing data?
ATG
test with tight demos known to convert or would want to convert....work that first if that works.....scale it up....
unless you are backed with a large budget.....the draw a wider net.....
but TBH the offer is critical....it either works or it doesn't.....thats the first step in building camp IMO
Personally I'm a fan of taking an offer that I know is proven and taking it to the dry cleaners. As opposed to putting recycling an angle, since most of the time a certain angle is built around a certain traffic source and offer.
I dont think there is a right or wrong way, but one area to take into account is the "lifespan" of an offer.
If your in the email submit area then you wont have the luxury of doing lots of testing. You need to quickly know whether the offer is still converting and then hone in on your profitable "targets".
I am not a fan of testing lots of combinations (20 images, 10 headlines, 3 body copies).
I prefer to start with a limited number of variables to find something that breaks even.
Often tightly targetted.
Then improve metrics by split testing and improving on the control.
Then expanding.