Measuring Success Without Historical Benchmarks

Posted February 16, 2010 by Thomas Ryan

A question was raised during SageCircle’s recent CoffeeTalk on Performance Metrics: For programs just starting a measurement program with no historical benchmarks, how do you measure success?

This is a very good question. Of course, the easy answer is that whenever you start your measurement program, that will become your benchmark against which future improvements will be measured. And the follow-on from there, typically, is that you should start your measurement program…NOW!

But neither of these easy answers is necessarily true.

First, depending on what you intend to measure, historical context actually may be available. Let’s say you want to start assessing the impact your program is having on how your company is being represented in the analysts’ published research. For many organizations, this is the first area of performance metrics that they look at.

ASG has been monitoring analyst research for several years now, and we have a research database that has captured well over 60,000 analyst research publications. In many cases, we can mine our database to determine how a company has been represented over the last six months or the last year, whatever is desired. But even if a company is doing this work internally, back-dated analyst research is available to support review and analysis.

A caution, though, is that this can be moderately expensive in time and/or dollars; so this should be factored into the startup costs for initiating the measurement program. For this reason, many companies simply choose to monitor research on a going-forward basis.

So back to the original question: how do you measure success over the first few months as the historical context is established?

The answer depends on what you intend to measure. If all you are tracking is mentions (a typical Public Relations measure), then historical context is absolutely essential and your measurement program will produce little value and no actionable feedback for several months.

However, if you are tracking the three parameters that ASG recommends (Share-of-Voice, Average Tonality, and Net Market Impact), tremendous value can be captured from the very first period report.

For example, what is the Average Tonality of the coverage your company is receiving? Is your coverage mostly positive? Negative? Mixed? Knowing this can drive action…NOW!

What is the relationship between the Share-of-Voice you receive and the Net Market Impact being generated. Let’s say you are receiving 8% Share-of-Voice, but you are benefitting from 11% Net Market Impact. This tells you that you are better at turning raw coverage into positive impact than others within your market.

And is the Share-of-Voice you receive the result of multiple brief references, or is it driven by one or two feature reports? Again, this can provide insight that can guide the execution of your outreach and support. And it can provide this guidance NOW…without waiting for 6 or 12 months of historical context.

So, excluding the possibility of going back and examining historical research, whenever you initiate your measurement program, that is your benchmark for going forward. But it is CRITICAL to understand (and it is CRITICAL that your program sponsors understand) that there is tremendous IMMEDIATE value in a measurement program…as long as you are measuring the right things.

One final note regarding when to start a measurement program. While it’s easy to say…NOW! The truth is that it depends on what is going on inside your company and what is going on within the analyst community.

Attempting to initiate a measurement activity like an interview-based study between early September and mid-November simply won’t work. Many of the analysts you will want included are going to be focused on their fall conferences.

And what if your company has a major launch scheduled for November or maybe February. Is it better to do a study NOW and then perform a follow-up study at some time after the event? Or should you initiate your program following the event? And should the follow-up study be done immediately following the event, or maybe 4, 6, or even 8 weeks later? These are non-trivial questions.

For these reasons, we recommend that companies looking to start a measurement program talk with someone like SageCircle or ASG…talk with companies experienced in AR measurement to help you decide what to measure and when the most opportune times are to initiate your measurement program.

Post a Comment