Home > POP / PPV / Redirect > Follow-along Campaigns

need some insights on lead impact... (with stats) (10)

03-28-2012 02:28 AM #1 harrypotter (Member)
need some insights on lead impact... (with stats)

pushing on lead impact right now and I am kinda confused over what is happening...

some insights or comments from fellow stackers would be GREAT!

please keep in mind:

conversions not a priority right now, just want to find lander with good CTR
split tests span from March 19 to March 27 (8 days)
user cap is set to 1 view per 6 days

split test #1: started off with a few landers... here are the stats:
name Views Clicks CTR Conv CR
landerA 190 8 4.21% 0 0.00%
landerB 188 7 3.72% 1 0.53%
landerC 188 9 4.79% 0 0.00%
landerD 185 4 2.16% 0 0.00%
landerE 184 3 1.63% 0 0.00%

So I stuck with landerB and did a bunch of split testing with the page... adding sound, timer, stuff like that

split test #2: version 2 beats version 1
name Views Clicks CTR Conv CR
landerB(2) 135 7 5.19% 0 0.00%
landerB 117 0 0.00% 0 0.00%

split test #3: version 2 beats version 3
name Views Clicks CTR Conv CR
landerB(3) 173 3 1.73% 0 0.00%
landerB(2) 163 3 1.84% 1 0.61%

split test #4: inconclusive, but version 4 looks like it will beat version 2
name Views Clicks CTR Conv CR
landerB(4) 135 5 3.70% 0 0.00%
landerB(2) 97 2 2.06% 0 0.00%

so i did the test again

split test #5: tested version 2 + 4 again, with version 5. version 4 wins
name Views Clicks CTR Conv CR
landerB(2) 231 5 2.16% 0 0.00%
landerB(4) 300 10 3.33% 1 0.33%
landerB(5) 334 8 2.40% 0 0.00%

split test #6: version 4 wins again
name Views Clicks CTR Conv CR
landerB(6) 163 4 2.45% 0 0.00%
landerB(4) 159 6 3.77% 1 0.63%

split test #7: version 4 wins again
name Views Clicks CTR Conv CR
landerB(7) 138 1 0.72% 0 0.00%
landerB(4) 122 6 4.92% 0 0.00%

split test #8: since there were two other landers from split test 1 that had potential, i decided to apply the tweaks made to landerB to both landerA and C to see what would happen...
name Views Clicks CTR Conv CR
landerA(2) 112 0 0.00% 0 0.00%
landerC(2) 111 0 0.00% 0 0.00%
landerB(4) 75 1 1.33% 0 0.00%

THIS IS THE WEIRD PART... no clicks. and for landerB(4) i don't know why CTR just went down so much all of the sudden

split test #9: so i tested version 4 again by itself to see if something was wrong...
name Views Clicks CTR Conv CR
landerB(4) 222 5 2.25% 0 0.00%
landerB(4) 225 4 1.78% 0 0.00%

questions:

1) CTR for landerB dropped from 3.72% to 0%, within a few days (split test 2 started right after split test 1). Is this "normal"? is this what you guys experience too?

and we know ppl are clicking because landerB(2) got 5 clicks in the same test duration

2) landerB(2) also dropped in CTR within a few days. see trend:

Views Clicks CTR Conv CR
split test #2 - 135 7 5.19% 0 0.00%
split test #3 - 163 3 1.84% 1 0.61%
split test #4 - 97 2 2.06% 0 0.00%
split test #5 - 231 5 2.16% 0 0.00%

3) landerB(4) AGAIN dropped in CTR within a few days and really the reason i posted this thread... look at the trend:

Views Clicks CTR Conv CR
split test #4 - 135 5 3.70% 0 0.00%
split test #5 - 300 10 3.33% 1 0.33%
split test #6 - 159 6 3.77% 1 0.63%
split test #7 - 122 6 4.92% 0 0.00%
split test #8 - 75 1 1.33% 0 0.00%
split test #9 - 222 5 2.25% 0 0.00%
split test #9 - 225 4 1.78% 0 0.00%

there seems to be a "steady decline" in CTR in just a few days (on several occasions) and so I wanted to ask if this is what you guys experience as well in lead impact.

technically, with the user cap set to 1 view per 6 days, I should NOT be getting repeated viewers...

oh i should mention... i checked on my competitors and they are not doing anything similar at all... so my landers shouldn't dry up so fast? don't know.

confused.

thanks in advanced for your insights

03-28-2012 08:20 AM #2 tijn (Moderator)

first off - 100 views is probably not enough to conclusively say 1 page is better then another.

check some of the posts by inversion about statistical significance.

especially test 8 - 75 views is too low.

re your questions

different people are online different times of the week. I suggest running the same test a bit longer to eradicate any influence due to day of the week. You can do this by setting the budget to 5 per day or 1 per hour on LI.

also - it really depends on your targets and the volume they have.

third - your freq cap is quit high. I normally set it to 48 or 72 hours.

but overall my gut feel is that its not enough or you ran the campaign for too short a time.

other views?

03-28-2012 04:00 PM #3 harrypotter (Member)

thanks for that tijn!

will need to find inversions' threads on statistical significance and take a read!

i was playing around with http://www.answersresearch.com/size.php and I ROUGHLY estimated what I needed... (which is probably wrong now that I think about it)

this WAS my thought process
- if i wanted an 85% "effective" confidence, then all I needed was a (95% confidence; ± 10% error) because the worst case scenario would be 85%... the tool told me I needed a sample size of 96

- if i wanted an 90% "effective" confidence, then all I needed was a (99% confidence; ± 9% error) because the worst case scenario would be 90%... the tool told me I needed a sample size of 205

lol, going to search for his threads now

--

regarding the frequency cap, 48 or 72 hours would get many repeated visitors seeing our landers right?

i inquired with lead impact and they said the frequency cap is based on an install machine

03-28-2012 06:44 PM #4 harrypotter (Member)

for other wonder where to find inversion's wisdom on stats, here are the ones i found:

http://stmforum.com/forum/showthread...ore-Guesswork/

http://stmforum.com/forum/showthread...l-Significance (This page isn't loading...)

http://stmforum.com/forum/showthread...-Ad-quot-Table

--

@tijn, i am following your tutorial (http://stmforum.com/forum/showthread...28Requested%29) while doing my campaign

so for my email submit campaigns, also shooting for
10% CTR (10 clicks in 100 views)
20% CR from lander (1 conversion in 5 clicks)

trying to hit the 10% CTR...

when you guys optimize your landers, how many views do you give the lander in an A/B split test before determining WINNER or LOSER?

03-28-2012 07:41 PM #5 Mr Green (Administrator)

^^ I keep split testing until I see a gap of more than roughly 15% between the landers. There is no set rule, I mean if one lander gets 10 leads and the other lander gets 2 leads after 5k impressions then you are good to make the call. But if you see 4 leads and 5 leads after 15k impressions, I would keep rotating.

03-28-2012 08:24 PM #6 harrypotter (Member)

(student always learning - please forgive me if my questions seem elementary or blunt...)

i see! so what about lower budget tests?

5k impressions at 0.015 = $75

if we were testing like an email submit offer with < $2.00 payouts, $75 is a lot? i don't know.

is there a MINIMUM amount of impressions/views you guys shoot for?

thanks for your insights!

03-29-2012 11:37 PM #7 tonyt2929 (Member)

I also had some questions of my own and would be grateful if some of the senior members could shed some light :-)

From my understanding There are going to be targets that eat up ur budget and never convert so Could we also do this same calculations at a target level for ppv?

Also I wonder about the conversion rate. What if let's say we are getting a 5% CTR on 100 views/impressions with 5 conversions. would we still kill the target/ LP if the did not reach our 10% target?

03-30-2012 04:01 AM #8 harrypotter (Member)

Originally Posted by tonyt2929

From my understanding There are going to be targets that eat up ur budget and never convert so Could we also do this same calculations at a target level for ppv?

I haven't heard of using CTR to gauge how well targets perform... but maybe the calculation can be done for CR, but the problem would be to find out WHAT IS the typical CR for a target to shoot for?

-> I don't think I have seen this figure reading through much of the forums

from the 40 day guide (http://stmforum.com/forum/showthread...arketing-Guide), this is how it's recommended to optimize targets:

There’s no exact science – over time you will develop your own rules and improve your gut feeling. For now, here are some general guidelines:

• 50% payout, no clicks - Pause any targets that have spent 50% of your offer payout and not received any clicks.
• 100% payout, no conversions: Pause any targets that have spent 100% of your offer payout and not received any conversions.

keep in mind the above is for lower payout offers though... things like email/zip submits

the rules have to be adjusted (with experience) to suit higher payout offers

--

Originally Posted by tonyt2929

Also I wonder about the conversion rate. What if let's say we are getting a 5% CTR on 100 views/impressions with 5 conversions. would we still kill the target/ LP if the did not reach our 10% target?

Ultimately conversions is what we are after and if you had a lander that converts even though the CTR isn't great, you would probably keep that since it's bringing in the MONEY and making the campaign profitable

think of it this way.

CTR could be high if you say next 10 person clicks will win $10000

but if you send it to a car insurance offer, there will probably be NO conversions

my 2 cents with my limited experience

03-30-2012 05:13 AM #9 tonyt2929 (Member)

Thanks for the tip. Anyone else has any more tips?

03-30-2012 05:15 AM #10 tonyt2929 (Member)

I also have not heard of killing targets by their CTR. On the 40 day guide it says to kill it in refrence to the spending but then again I thought it might apply too.

Home > POP / PPV / Redirect > Follow-along Campaigns