Stronger Culture. Better Performance.

September 8, 2011 in Change, Learning 4 Comments

Rethinking Measurement

Thanks to David Patt, whose comment on an Acronym post pointed me to a really good article by Matthew Forti about measurement for nonprofits. The Acronym post was from Scott Briscoe, building off of Joe Rominiecki’s notion of “loveable losers,” those programs associations have that lose money but provide enough value to justify the negative hit on the bottom line. Scott pointed out that our focus on measuring financials first is what generates the term “loser,” but maybe that measurement isn’t as important as engagement or something else that is mission-related.

Forti’s article is focused more on service-oriented nonprofits and talks about mistakes we make in scientifically demonstrating a measurable impact with our programs, because in order to prove cause and effect, we end up picking fairly isolated (maybe even random) impacts to measure. We show that there was a 6% decrease in days absent from school compared to the year before (yay!), but are we really making a long-term difference in the lives of those children, or of their communities? In Forti’s words, are we changing the “trajectory” of the client?

More nonprofits need to explore what it means to hold their organizations accountable for meaningful change in the trajectory of a client, however they define it.

This means a new definition of what “results” are, which means some new ways of measuring it. This will be a challenge, because I doubt anyone feels like they have the time to develop a new set of metrics. We’re barely keeping up with our current reports. But rethinking your metrics is probably a GREAT way to start changing some trajectories. Measure different things, and at different intervals, with the intention of actually learning more about your system and how it operates. That can lead to some behavior that changes trajectories. You won’t know the answer when you start down the path (which can be scary for stakeholders), but this is the kind of courage we need right now to break free from what has been holding us back.

image credit

4 Comments

  2. David M. Patt, CAE
    13.09.2011 at 2:29 pm

    Thanks for digging a little deeper into the topic, Jamie. I’ve had to defend financially marginal programs by showing how they generated more members or otherwise made our association more attractive. Dollars were not always the right measurement tools.

  3. Links for September 18 2011 | Eric D. Brown
    18.09.2011 at 5:47 pm

    […] D. Brown Leave a CommentstLight.options({publisher:'caabf3e0-fd2f-4585-a293-1f82ac0368ed'});emailRethinking Measurement by Jamie Notter Quote: …rethinking your metrics is probably a GREAT way to start changing some trajectories. […]

  4. Liam Barrington-Bush
    20.09.2011 at 10:35 am

    Thanks for this Jamie!
    Working primarily in the non-profit world, metrics are too often the bane of my working life; the obsession (to put it lightly!) to quantify human stories and experiences into a top line ‘result’ figure, only tend to ensure that the real value of the service, or campaign is lost in presentation.
    David Boyle’s ‘The Tyranny of Numbers’ addresses a lot of the associated issues with our obsession with counting things, but the real gem on this front is in ‘Getting to Maybe: How the World is Changed’, where it identifies the serious problems with supposed ‘funder accountability’ mechanisms, and the range of alternatives (trust being a big part of them) that lead to better results.
    I wrote about the many ways monitoring and evaluation kills learning, accountability, trust, and our relationships with those our organisations support, here: http://www.concretesolutions.org.uk/?p=437
    Strongly suggest Michael Quinn Patton’s ‘Developmental Evaluation’ work as well, as one framework for getting past the issues w/ pre-determined outcomes and using numbers to describe complex situations.
    Cheers,
    Liam