The Dark Side of A/B Testing – Avoiding the Pitfalls

As product designers and marketers, we love the clarity that comes from A/B testing. We have an idea; we implement it; then, we let the customers decide whether it’s good or not by running a controlled experiment. We split the users into unbiased groups and watch one treatment overtake the other in a statistically significant, unbiased sample.

Thoroughly impressed by our own analytical rigor, we then scale up the winning treatment and move on. You guessed it: I’m about to poke holes in one of the most sacred practices in tech, A/B testing.

Continue reading

Getting a Return from Awareness Marketing, or Measuring the Immeasurable

An engineer by training, I’ve always been attracted to problem spaces that offer a feedback loop. Got something right? Observe a metric go up. Want to know which approach your customers like better? Run an A/B test, look at the numbers, clearly see the best path forward. You can make a lot of product and business decisions this way.

0 (1)

This approach served me well – until that memorable day. The day I was the one responsible for acquiring new customers. A hundred years of marketing wisdom tells us that multiple exposures are required to have someone buy from a brand for the first time. And yet, there’s so little science in determining the optimal media mix to actually drive sales.

Continue reading

Having Others Want to Follow You

01fa9740-8980-441f-a71d-78ffaa06a3b1-largeI was lucky to work for a wise man who taught me the best definition of leadership that I know of: being a leader means that others want to follow you.


Want, as in, they see something in you that makes them believe in you, in the words that are yet to come out of your mouth – only because they are yours.

Want, as in, voluntarily come to you asking for advice. Want, as in, trusting your intuition with no explanation necessary. Somewhat fanatical, a bit primal, it’s akin to blind trust – something that doesn’t usually happen at the workplace.

Continue reading

Incrementality in Marketing, or Marketing Economics 101

1.jpgIt’s easy to grow a consumer business: attract new customers, increase your sales. Just give your prospects money. That’s right, get on top of a nearby building and scream “Anyone want money? Take some of my money! I’ll pay for your first 5 purchases from me! And give you a $50 gift card!..”

What, you’re not compelled to try this?.. Good. You’re thinking about profitability – acquiring customers in a way that actually allows you to have a sustainable business instead of just burning your investors’ money.

And yet, the approach of giving prospects money so that they become customers is surprisingly common. Humans are amazing at optimizing for the goal that’s set for them; the CEO asked for new customers? Great, we’ll give each consumer $20 to give our product a shot!..

An obvious constraint must be added here, a profitability constraint. If we give a customer $20, we better be sure that in the time horizon we deem appropriate, we are going to get $20 back in profit from this customer. That is, that the long-term value (LTV) of this customer is equal to the cost of acquiring this customer.

Continue reading

End of my eBay Chapter

This is a copy of an email I sent to my colleagues at eBay.

My chapter at eBay is coming to an end. These past 3 years have been such a privilege for me – thank you so much for putting your trust in me. I’m proud and humbled by people that worked by my side – persistent, creative, bold. I’ve learned so much from you.

When I look back, I remember the days when CRM at eBay was just forming – with a myriad of uncoordinated campaigns and lots of technical debt, all we had was determination, grit, and a vision. We have achieved tremendous results since those days: we’ve grown Marketing by double digits each year – and found the balance between human and machine. Today, peers from Uber, LinkedIn, and Airbnb say that eBay has one of the top 3 Marketing platforms on the planet – both in terms of customer experience and technological sophistication. I know that in the coming years, you will take it even further.

Continue reading