Beware Of “Experts” Sharing Data…..

I guess part of being perceived as being an expert is sharing data and research—all of which supports the conclusions you want people to reach.

Don’t get me wrong, we share data on the results we’ve produced, we share data on research we have conducted.  But we are careful to position the context of that data and how it might be interpreted.

But too often, we consciously or unconsciously, we “lie with statistics.”

I read an interesting article in Forbes.  Largely, I agree with the author, but he cited an example that is very misleading.  I’m certain there was no bad intent, but it was an example that reinforced his basic premise–everything in buying is going digital, the B2B buying process is becoming consumerized.

He posed an example, under the heading, “Don’t Get Fooled By The Occasional Analog Win.”

The example was behavior he noticed on a train ride.  A SaaS sales person, over the course of 4 hours and multiple conversations closed a single deal worth $10K.  He contrasted that with another sales person sending out a promotional tweet for a logo creation service to 1.2 M followers on Instagram and Twitter.  After blasting those messages out, at the end of the train ride, he had converted 800 to orders totalling $16,000.

Clearly, the conclusion we are supposed to reach is a simple posting on Instagram and Twitter was far more productive (in those 4 hours) than a series of phone discussions with a single customer.

“Experts” that want you to drink the “Social Selling Kool Aid” will revel in this example.  They will say, “Social selling produced 60% more revenue in the same time, QED!”

But one should be skeptical about these examples.  One should start questioning and looking deeper.  Some thoughts I had:

1.  The $10K SaaS deal was ARR.  In reality, the deal was far greater than that, an important metric in SaaS is CLV-Customer Lifetime Value.  In reality, this deal was probably worth $10’s of thousands in CLV.  I’m not sure what the entire sales cycle was, we’re left to think it was 4 hours, but even if that was the tail end of the sales cycle, that’s a great deal.  Let’s be really conservative and imagine CLV at $25K.

To create $1M CLV, I have to close 40 deals.  If it’s $1M ARR that I want to hit, I have to close 100 deals (but the CLV for that is $2.5M).  On a CLV basis, for the sales person, that’s a little more than 3 deals a month.  On an ARR basis, that’s a little more than 8 deals a month—both probably achievable.

2.  By contrast the Instagram/Twitter Deal was $16K, based on the description of the company, it looks like that is one-time revenue.  The results aren’t stunning.  Out of 1.2 M messages, 800 closed—or 0.067%, for an average transaction of $20 per customer.

One starts doing the math.  If I want to produce $1M in orders, I have to acquire 50,000 customers.  At the 0.067% conversion rate I must reach out to about 75M customers.  What’s this mean–can I go to the same 1M+ customers 75 times (clearly there will be significant fatigue in those customers receiving messages.  Do I cast a wider net, but the yields are likely to plummet.  How do I effectively reach those 75M customers?   And at what cost?

Now, all of a sudden, things look a little different.  Perhaps the “old school” method actually looks much more intriguing.

In fairness, it’s impossible to ascribe “goodness” to either case, though that’s clearly the implication the author is trying to make.  The situations are so different.  The selling/buying process, the target customers, the decision-making processes are very different.   For example the SaaS deal may have been a complex sale with multiple decision-makers and a higher level of risk.  The other deal is a very transactional deal, with very low risk, and likely to be a single decision-maker  (Brent and Nick, I know you are reading this, how many $20 deals require 6.8 😉 )

It’s impossible to tell which is better than the other, because these aren’t comparable situations.

Yet we are constantly presented with these false choices or alternatives.

And too often, we get sucked into believing them, often in very dangerous ways.

As consumers of this kind of data, it’s always important to be skeptical of the data and conclusions.  Make sure you understand the underlying context and assumptions.  Make sure those are relevant to you, representative of your business, customers and markets.  Be skeptical of claims of outrageous results, while they may be true, they may only apply to a very special case which may not be representative of the situations you face.

As experts, while it’s fun to present sexy compelling data, but if we are to be responsible and to engage followers in a meaningful way, we have to be careful how we position and present the data.  Thinking of customers a gullible and presenting data that, while true, may be very misrepresentative of what they might expect.  Be careful, it will always come back to bite you.

Afterword:  To help you understand and think about these issues both more deeply and broadly, I highly recommend reading the following:

Factfulness by Hans Rosling:  A stunning analysis of much of what we think of as true about world data on medicine, environment, economics, and how we look at problems in the world.  Lessons can easily be translated to what we do every day.

How To Lie With Statistics by Darrell Huff:  An absolute classic.  This was originally published in the 1940’s but is a fun read about how we use and misuse data and statistics.  Really a fun, fundamental book.