Brand lift or Sales Lift? Brand lift is a bit different because there are a lot more companies offering this due to more simplicity. The shortcoming of Brand Lift is every vendor has their own methodology when it comes to deciding what exactly is a marginal lift in brand awareness. I think the easiest option for a lower budget is using a FB/IG brand awareness campaign and looking at ad recall numbers (https://www.facebook.com/business/help/1029827880390718). However, we don't really know how they calculate this number and it would only be useful to benchmark between campaigns (if you trust the number). The best option right now is to find a vendor that polls the user who saw the ad, like Vizu (Nielsen). Unfortunately, to get a stat. significant read, you probably need a lot of results, and thus would need higher media spends.
I don't trust the black box of the ad recall stats they provide, so that's not really an option. Is Nielsen's data any less of a black box for this sort of data? And is it actually a decent sample? The older way Nielsen did things was biased because people had to sign up to be a part of it.
And yes, the challenge is that to get a statistically significant read, we need to throw a lot into a specific test, and that is hard to justify without benchmarks. Kind of a chicken and egg problem if your brand is large enough where it takes considerable display dollars to make a blip against your other traffic.
I've been curious if things like Adobe's Attribution econometric modeling tools help much here, but guessing that lack of data will be an issue there as well.
I agree with you 100%. No options are ideal and statistically correct which sucks... a lot. As you mentioned there will always be selection bias and a black box, just hopefully less, with Nielsen for example. Not many people in our space even understand these problems exist, so I think you already lead the pack in this regard. For the sales lift studies, the methodology seems less "iffy" since its one to one and measurable results. One can argue there's selection bias due to no cash data. I haven't seen any cheaper options to do this while minimizing error also.
My client is in the same chicken-egg problem and the way we approached it is: Going on blind is worse than the risk of a bad study. It's completely up to us to do due diligence and ask the right questions to find holes in the vendor's methodology and minimize the risk. We felt that the data gained will be accurate directionally, albeit skewed.
I've never had first hand experience with Adobe's attribution models but from what I gather, lack of data and similar to media mix modeling.
IMO, advancements in this space will be battling privacy concerns.
Yeah, privacy concerns are what will decide a lot of this. I'm very torn because as an end-user I definitely have gotten more nervous with the current state of tracking, but as an advertiser, these are arguably the toughest problems in the industry and what keep me up at night.
Have you found any good off-the-shelf tools to help with the incremental lift analysis that might be better suited for advertisers who are not as massive? Our current plan is to get DCM in place, pipe everything we can through there, and then let our Data Science team go nuts at trying to model it. Seems like that's our best bet since tools like Adometry or other dedicated attributions platforms don't make sense for us yet.
