Roger Craver explains which metrics matter (& wch don't)
asked a dumb-ish question. Super-guru Roger C. answered ... wisely, kindly AND in SHOCKING! depth. I learned a ton. Save this one for the files.
"Roger Craver here...."
[he wrote] Good and extremely important questions you ask.
[Roger expanded on his theme, here in Part 2]
What we MEASURE vs. what we MANAGE
It’s fashionable these days for many nonprofit fundraisers and their consultants to claim they are “data-driven.”
The problem is [40 years of front-line experience talking here]: MOST focus on the “data” part of that slogan.
Few understand [he said] the requirements of the second word – – “driven.”
In reality, what we choose to measure will likely reflect on what we choose to do or choose to manage.
THE BOTTOM-LINE DIVIDE: Vanity vs. Value
It’s very important that we choose metrics that really matter in terms of being helpful -- or even fundamental -- to growing our donor database and our income.
I divide data between what I call “vanity metrics” and “value metrics.” The distinction between these types of metrics lies in the question:
“What would I do differently based on the information provided by that metric?”
If that question doesn’t arise around the data you’re collecting, you probably shouldn’t be spending time or money to collect and analyse them.
In essence, part of the problem with “vanity metrics” is that nobody does anything with them.
More importantly, and dangerously, "vanity metrics" are often used to drive absolutely bad decisions. For example: "If I spend more on online search advertising, can I drive up the number of website visitors?"
A largely meaningless strategy. A largely meaningless result!
Some great examples of vanity metrics
Benchmarks. My complaint with benchmarking is that few organizations use it as a questioning metric to guide or develop new actions. I applaud use of benchmarks and the organizations that do use it for that purpose.
Unfortunately, too many fundraisers use benchmark as a proud justification for doing nothing except boasting to their board or CEO. As in “we’re doing great! We’re above average in the benchmark for retention.”
If ever a worthy metric concept was misapplied for the sake of vanity, it’s the benchmark. In the social media age, here are some vanity metrics I’m sure you’ve heard bandied about day after day ad nauseum:
Number of hits. A fossil from the early days of the web. If you have a big site with many elements, there will be lots and lots of hits. So what? Better to count the actual number of people involved. I.e., number of page views.
Number of visits or number of unique visitors. Sheds no light on why they visited, why they stick around, why they left. Pretty much worthless.
Number of likes, friends, followers. Simply a popularity contest. Only if you know how many followers will take action when you make a request do you really have an actual metric. Most organizations don’t use it this way.
Another of my pet vanity metrics, which will surprise some, is acquisition response rate.
An amazing number of direct response fundraisers measure “success” or “failure” by response rates as in “our rival’s acquisition gets only a .65% rate and we get 1.2%!”
Pretty much worthless because it tells you only how a specific campaign went, and raises almost no "what should I do differently” questions, other than “what should I do differently to get the response rate up?”
["GUILTY!" I murmured into my upturned winter collar. A cold day getting colder.]
What all this reflects is a singular obsession [Go, Roger!] with nothing but campaign metrics and various manipulations that produce only variations on the same numbers. Columns and columns of redundant, non-useful metrics – – created by just switching around the numerator and denominator of response rate, average gift, average cost AND…that magic output: cost to raise a dollar.
Valueless because it leads to no insight, no action.
One set of useful metrics that could be done on the same report, would be forward-looking, and remind the report reader what the goal truly is falls under my rubric of “value metrics.”
A “value metric” is one which you can manage around. By that I mean what steps can you take improve this metric so the value to your program is increased.
Here are some examples:
Number of New Donors Making a Second Gift. A harbinger if not dead-on predictor of the retention rates and Lifetime Value an organization is likely to enjoy in the future.
Number of New Donors Retained In to the Second Year. If you ask and answer the question as to why so many donors leave the first year and what your organization is doing to lose them or hold them you’ll be on a true track to growth. Fail to answer these questions: it’s more of the same no-growth mode.
Multiple-Year Retention Rate. Same as above ... but by tracking these year-on-year (YOY) numbers you can spot trends, problems and opportunities. Why? Because year-over-year comparisons of this metric will trigger additional questions and answers for improving your program.
Lifetime Value of a Donor (LTV.) At the end of the day all the actions you take to improve retention, average gift and donor commitment/loyalty will be reflected in the LTV value of each donor, all donors collectively.
LTV is the key metric on which you can benchmark, guide and then track the success – – or failure – – of your intermediate and long-term strategies. That’s why I call LTV the GPS of fundraising.
Lagging, Leading and Key Performance Indicators. All of the value metrics set forth above are what statisticians call “lagging indicators” meaning they reflect what is already happening and are generally not predictive of the future.
RFM or RFV (Recency, Frequency and Monetary Value) seems to be the trade’s preferred choice when it comes to lagging indicators. The problem with RFM is that it’s limited to some snapshot of the past. Good for trend analysis. So-so for segmentation and projection. Not good for guiding you in figuring out how to create greater breakthroughs, create better donors more valuable donors.
This is why sooner of later organizations that want to grow have to get into the business of measuring/scoring loyalty or commitment. But, that’s beyond your question right now.
[Roger Craver is one of The Agitators. He's an unrepentent trouble-maker. He's carved out an extraordinary career as a hyper-effective fundraising strategist. As a direct mail early-adopter and developer, he helped Common Cause and the Sierra Club grow to national prominence. Countless other charities benefit every day from Roger Craver's discoveries ... not to mention all those come-later direct-mail agencies who cash in on his pioneering work. I was paid nothing for this endorsement other than a thick steak and an iced martini on Martha's Vineyard.]