The behavior of pipeline metrics in the real world is nuanced, and understanding that behavior is key to understanding how and when pipeline metrics lie to you. Brian Skowron, president of Lullabot, explores in the following Bureau of Digital Guest blog post.
“As far as the laws of mathematics refer to reality, they are not certain; and as far as they are certain, they do not refer to reality” -- Albert Einstein
Sales pipeline metrics assign certainty to that which is uncertain. They can be simple or elaborate, but all pipeline metrics evolve from this basic rationale:
If I can reliably measure the amount I’ve proposed…
And I can reliably measure how often I win proposals…
And I can reliably measure when proposals will close…
Then I know how much I will sell in any given month.
On the surface, this seems reasonable. The first two measures are easy to track, objective, and irrefutable. The third measure is a bit more squishy, but if an opportunity is properly qualified, more often than not you have some idea when that opportunity will be won or lost.
Why pipeline metrics seem useful
Being able to forecast revenue is the holy grail for anyone running a business. If you can reliably predict what your revenue will be every month, everything else within your business can fall in line behind that knowledge—you can right-size your expenses, sharpen your operations, and plan your growth.
The allure of pipeline metrics is they seemingly give you that knowledge, they feel scientific, and they radiate a comforting sense of certainty. It’s not uncommon to use this type of language when discussing pipeline metrics:
Based on our 23% close rate, we’re pacing 14.3% ahead for October revenue, and should exceed November revenue by 8%.
Or…
Our pipeline is showing we’re 7.5% short of pace for next month. We need to figure out where to make up this shortfall.
These statements are often lies, and the decisions they justify can be catastrophic.
Why? Because these figures are uncertain prognostications of reality, yet their specificity conveys a certainty that lends them more authority and weight than they deserve. Whereas nearly every financial KPI in a business is backward-looking, measuring what has already happened, pipeline metrics are one of the lone voices speaking to the future. Pipeline KPIs sing an irresistible siren song that is difficult for management to ignore.
How pipeline metrics lie
Despite their simplicity, the behavior of pipeline metrics in the real world is nuanced, and understanding that behavior is key to understanding how and when pipeline metrics lie to you.
To illustrate things better, let’s use a metaphor. Imagine you are standing alone in an empty room. On the floor is painted a large target. In your hand, you carry a bunch of marbles.
Further, imagine you’ve tossed all of these marbles in the air at once, and — while they are airborne — are asked to give a measure of how many will land on the target.
If you borrowed the same forecasting rationale as most pipeline measures, you could look at the historical percentage of marbles that ended up on target on prior throws (close rate), and apply that percentage to the amount of marbles you’ve tossed. That would give you a simple forecast of marbles on target. Let’s say it’s 30%. This seems straightforward so far, and you’d probably feel pretty comfortable planning that around 30% of your marbles are going to end up in the target.
However, this forecast quickly starts to break down for a number of reasons.
Truth #1: Your marbles are all different sizes.
If marbles are analogous to proposals in this metaphor, there’s one major problem. Your marbles are all different sizes. Some are normal marble sized, some are the size of a baseball, and one or two are basketball sized. And really, forecasting the number of marbles in the target isn’t quite as important as forecasting the gross marble weight. How many ounces of marbles will you have on the target at the end of this toss?
Suddenly your forecasting methodology is starting to show some cracks. You might be confident that 30% of your marbles will end up on the target, but if two of your marbles are basketball sized, how different will your gross marble weight be depending on where they land? When it comes to forecasting revenue (gross marble weight in this metaphor), your close rate is far less meaningful than which deals close and which don’t.
Truth #2: Your marbles change size in the air.
You’re fortunate to know exactly how many marbles you’ve tossed in the air, and you have the added benefit of having weighed them ahead of time. There’s only one problem, they keep growing and shrinking after you’ve tossed them.
For anyone in sales, the phenomenon of proposal revisions and adjustments are nothing new. The product or service you think you’ve proposed could face the harsh realities of budgets, or perhaps needs change and scope increases. Bottom line, the marble weight you started with—or the dollar amount of the deal you proposed—will likely change before it hits the ground.
High performing sales teams will do their best to capture every revision and change in their CRM so that pipeline KPIs are reflected accurately, but rarely is this discipline upheld until the final contract is signed. What that means is you suddenly find the original marble weight you were forecasting on has changed by the time the marble lands, sometimes significantly so.
Truth #3: You don’t know when your marbles will land.
Our metaphor has been straightforward thus far, but it’s still got another major flaw. We’re presupposing that all of the marbles we’ve tossed will land when we expect them to.
The truth is once the marbles have left our hand, we don’t know when they’ll come back to earth. We have a loose idea and expectation of when they’ll land, but at the end of the day the forces of physics and gravity determine their ultimate hang time. If our marbles are proposals, our client’s sales cycles are the forces of physics and gravity—they decide when our marbles are going to hit the ground, and they decide if those marbles land in the target or not.
Every seasoned sales professional knows that a communicated decision deadline is aspirational at best. And yet, for the sake of pipeline metrics, those professionals must predict the unpredictable. Therefore they go with their best guidance, which is usually the communicated decision timeline, which is usually wrong. Some organizations abstract this, and try to apply an “average sales cycle” to their forecast calculations. I’m sure there are industries that have reliable and repeatable sales cycles – the vast majority do not.
Back to the metaphor, this means we’re tossing up marbles of varying sizes which are changing constantly, and we need to predict how many will land in the target at an exact snapshot moment; but we don’t know when they’ll fall to the ground. Needless to say, this is not a recipe for certainty.
Bonus Truth: The windows are open, and it’s windy outside.
The last lie inherent to pipeline metrics is the lie of close rate (or win ratio). In the marble metaphor, the close rate was based on the historical percentage of marbles landing on target. Most sales organizations also base their pipeline close rate on historical performance.
Unfortunately, the old adage of “past performance is no guarantee of future results” applies here. While it’s tempting to assume close rates remain constant, they are subject to a myriad of different factors, many of which exist outside of your business. Things like economic swings, seasonality, changes in the competitive landscape, and even regulatory changes can all affect your ability to win business, and in turn, affect your close rate.
In our metaphor, these external forces are like the room had windows with the wind blowing through. Depending on which direction the wind is blowing, and how hard, it can blow your marbles in different directions than you anticipated, and it can certainly push your marbles out of the target even if you’ve thrown them the exact same way you always have.
What to do when pipeline metrics lie
If the value of pipeline forecasting is so immense, yet pipeline metrics are so prone to error, what should you do? And what should you do when your Pipeline Metrics lie? Because trust me, they will lie.
What most companies do
Most companies run face-first into their pipeline metrics’ shortcomings. The certainty they thought they were measuring ends up proving to be very uncertain in a painful way. A big revenue miss happens when it wasn’t anticipated. Or a big revenue win happens when it wasn’t expected. Usually, the former scenario leads to some tough scrutiny on sales operations and pipeline measurement, whereas the latter leads to suspicions of sandbagging and padding numbers.
What results in either scenario is another vain attempt to make the uncertain pipeline metric more certain, which is a fool’s errand.
I’ve seen all kinds of pseudoscience to dial in the pipeline metric to greater precision. Opportunity stages are the most common attempt. The thinking goes something like, “if our win rate is X%, clearly some of our opportunities are further along and will have a higher probability of winning, some are less far along and have a lower win probability. We should define those stages and probabilities and weight our opportunities based on them.”
So for instance, a contract negotiation would have a higher win probability and more pipeline weight, whereas a qualification conversation would have a lower probability and lower weight. While most sales teams have a small handful of opportunity stages, one sales organization I was briefly involved with identified 15 different opportunity stages - all in a futile attempt to create certainty where there was none.
Other common approaches involve seasonally adjusting close rate, trying to apply different close rates across different industries and verticals, or putting some advanced analysis to approximate the length of sales cycles. In every instance, the goal is always to make the pipeline metric more and more certain—regardless of complexity, regardless of overhead, regardless of hassle. I’ve often wondered, as have many sales professionals I’ve talked to if their management would prefer they spend 9 out of every 10 sales hours creating a more certain forecast, even if it only left 1 hour for on-the-ground selling.
What you should do
With everything wrong about pipeline metrics, do they have value? I do believe a sales organization benefits from some measure of its performance beyond revenue, and there is a balance to strike in the way you measure your pipeline. For most organizations, the best approach is “minimum viable measures” – meaning, to hit peak performance, a sales team should try to measure the absolute bare minimum to be useful, and then put all other energy towards selling.
What qualifies as the absolute bare minimum will vary from company to company. Are you a large public company that trades in commodified widgets? Or are you a small to medium-sized professional services organization? Do you have a repeatable pricing structure or do you do a lot of custom work? Do your sales cycles occur in days or years? All of these things can determine the depth of pipeline measurements you can engineer, and how reliable and useful they might be.
At Lullabot, we’ve purposefully held the same rudimentary pipeline measurements, and same simple opportunity stages for the past 9+ years, and so far, our sales team has not outgrown them. We’ve also enjoyed 9+ years where pipeline measurements have not been a focus for our sales team, and instead, every conversation is about closing business and serving our clients. Our pipeline measures are far from perfect, far from precise, and far from certain; but they do the job and everyone understands them.
Rather than prescribe a specific one-size-fits-all set of minimum viable pipeline measurements, I believe there are some principles every organization should follow when deciding how to measure their pipeline.
Embrace uncertainty
Remember, no matter how much you refine your forward-looking pipeline metrics, they will still lie to you. Instead of treating this as a shortcoming to be fixed, learn to embrace it as a feature. Accept that your pipeline metrics will be imperfect, understand their idiosyncrasies, and look for the reality behind the numbers they report. Think of them more like a barometer giving you an imperfect indication of the weather, not a high-resolution doppler radar image.
Learn your deal flow
Are your marbles uniform or lots of different sizes? Do they tend to change size after you’ve thrown them in the air? Are they airborne a long time?
The more you know your deal flow, the more context you have for your pipeline metrics. Talk to your sales team, ask them how they expect the next few months to perform (and make it safe for them to answer honestly). Try to understand what they see as the big factors affecting their deals, and the risks to their opportunities. Can an opportunity occupy a high probability pipeline stage, but still have a high chance of loss? Yes. Your pipeline metric won’t tell you that, but your sales team will.
Stop chasing precision
Generally speaking, every step you take towards more precision in your pipeline metric requires taking more time out of your sales team’s hands. More precision requires more advanced measurements, which require more administration and tracking on the part of each seller. In the pursuit of precision, you will quickly reach a point of diminishing returns, and once you pass that threshold; you’ll start to undermine your team’s performance.
Time is the most important asset a seller has. Every minute spent tracking an opportunity or updating it is a minute not spent talking to a client, writing an email, or following up on a proposal. If your metrics aren’t perfect, but good enough, they are good enough left alone.
Examine what you really need to know
Perhaps your business is such that knowing a percentage point difference in anticipated revenue next month unlocks some strategic decision, but most businesses aren’t like that. Take a thorough accounting of what you really need your pipeline numbers to tell you. Be mindful of the difference between what you want to know, and what you need to know. Optimize for the latter and let go of the former. Your pipeline metrics will never tell you everything you want to know (they lie, after all).
Most of the time, businesses need a lower fidelity picture of their future revenue than they think they need. One way of framing the question is to ask, “If my future revenue each month was represented as a 1-5 rating with 1 being desperation and 5 being record sales, would that be enough to make the decisions I need to make? If not, why, and what more do I need? What decision can’t I make, and how important is that decision?”
Set it and forget it
The quickest way to render a pipeline metric useless is to change it.
Pipeline measurements are more valuable the more consistent they are. They’re still imperfect, but if they’re consistently imperfect, they allow you to compare months to other months, years to other years; and the longer they’re in use, the more you get a feel for how they behave. Like a clock that’s always five minutes fast, a flawed pipeline measure is still useful if you understand its imperfections.
Once you establish your pipeline metrics, stick with them. They are only to be iterated upon as the last resort.
Instill a proactive, accountable sales culture
The most important ingredient to any pipeline measurement is the culture of the sales team it’s measuring. You should always work to foster a sales culture built on trust, autonomy, teamwork and accountability. These attributes will make your pipeline metrics more accurate than any math will. A sales team that is proactive, and whose input is trusted and valued will have a natural incentive to report their opportunities accurately, and advise management in the best interest of the company. Conversely, a team that fears the consequences of missed numbers will feel incentivized to misreport and inflate their pipeline. Pipeline metrics are often weaponized by management in unhealthy ways, and pipeline metrics don’t motivate good sellers. The ideal sales team is one that generates intrinsic motivation and urgency, treating pipeline metrics as nothing more than one more data point among many.
Pipeline lies are part of the picture
To sum it up, if you’re responsible for reporting pipeline metrics, or are reconsidering your current pipeline measurement strategy, accept that your measurements will lie from time to time, and that’s okay! A lie is less harmful when everyone knows it’s a lie and everyone has a shared understanding of the truth behind it. The truth of a sales pipeline exists beyond the numbers, so always try to see the entire picture; as it comes into focus, your pipeline numbers will actually help you see more, even when they’re lying.
Interested in writing for the Bureau of Digital blog? We’re always looking for guest bloggers! Reach out to smith@bureauofdigital.com.