Forrester Blog ROI

Early last year Charlene Li of Forrester fame wrote a report on blogging ROI. I saw this pop up in a recent presentation and wondered if you couldn't improve on some of these metrics.

Here is the list (you can click on the image to enlarge it):



















Not a bad framework for understanding how to value a corporate blogging effort. But I would change the following:

1. Blog Traffic - unique visitors to a blog is a pretty useless measure from both an effectiveness standpoint and as a stand-in for advertising cost. This is because blogs tend to have a very high bounce-rate - the number of people who come to the blog and immediately leave because it wasn't what they were looking for. This is just an artifact of 'search', not a blight on the blog itself. Advertising, in contrast, tends to get served up in places where it at least has some relevance to the page/site/task at hand. Not really an apples-to-apples comparison.

I would suggest equating non-bounced blog traffic with the value of click-through advertising traffic (not impressions). So how much you would have paid to get the equivalent number of people to click on your ad in said content channel?

2. Press Mentions - Having your blog talked about by the press is an obvious direct substitute for PR cost. If the talk is positive, all the better. Ironically, I think equating this to advertising cost in the publication probably undervalues it. People are far more likely to remember a mention in an article than an ad. Although I would value this on a case-by-case basis. Not all publicity is good publicity. Believe me, it's not.

3. Technorati and comments - Not every blog has a Technorati ranking so this may or may not be a useful measure depending on the industry you are in. Comments as equivalent to a 'buzz agent'? I've never hired a 'buzz agent' but I would imagine they might be a bit put-out by comparing their efforts to blog comment numbers. Buzz agents can move a lot of interested traffic to your site and that traffic might turn into comments, but it's probably qualitatively different from just general comment leavers - who are more likely to confine their WoM to your blog and your blog only.

Track-backs and Technoarti mentions are probably best valued by the cost of a 'buzz agent'.

4. Comments as customer insight - A complete farce. In the article Forrester equates $180,000 worth of qualitative research to 100 comments on a blog a month. That's effectively turning qualitative research into a sophisticated suggestion box. Which it isn't. Good qualitative research is commissioned to solve difficult issues - brand positioning; deep (very deep) consumer understanding (think about the difference between a blog comment and an ethnographic study); communication testing/evaluation; etc. If you are using qualitative research to get a few ideas every now and then from consumers, you are wasting a lot of money.

5. UGC and NPS (User Generated Content and Net Promoter Score) - I can see where they are going on this one, but it depends on establishing a robust relationship between sales and NPS. I had a look at the NPS's historic relationship to sales in this post - not a pretty picture for that industry at least. I haven't used NPS extensively, but can't help feel a little dubious about its link to sales (this is not going to be the case in all industries, there are bound to be some success stories. As part of the value equation for a blog though, it needs to be looked at on as case-by-case basis).

6. Leads - This one makes a lot of sense. Blogs can generate sales leads. Pretty easy to measure as well - "where did you hear about us?". Probably one of the most tangible value results.

So over all, while some of these metrics need to be tweaked and others (the consumer insights one in particular) ignored, it's not a bad framework.

It's the sort of thing I can see presented to the CEO or CFO as an honest attempt at valuing a social media effort. It might not be exact, but it's probably not going to be significantly distorted.

Of course, blogging and social media have flow-on effects that are harder to measure (honesty, trust, openness, closer connections to customers) but equally as important.

Sphere: Related Content

Metrics wish list

Scott over at Artificial Simplicity just posted about some metrics he wishes he could measure. I just got off a phone call with a very intelligent ex colleague of mine who was also dealing with metrics he wishes he could measure.

On the call, my esteemed ex colleague brought up the Mystery and Puzzle notion so eloquently explained in a Gladwell op-ed piece for the New Yorker. A great idea to apply to anything you want to try and understand - be it marketing metrics or conundrums in general.

To quote directly from the Galdwell piece:

The national-security expert Gregory Treverton has famously made a distinction between puzzles and mysteries. Osama bin Laden’s whereabouts are a puzzle. We can’t find him because we don’t have enough information. The key to the puzzle will probably come from someone close to bin Laden, and until we can find that source bin Laden will remain at large.

The problem of what would happen in Iraq after the toppling of Saddam Hussein was, by contrast, a mystery. It wasn’t a question that had a simple, factual answer. Mysteries require judgments and the assessment of uncertainty, and the hard part is not that we have too little information but that we have too much.

The distinction is a great one. Too often we think we're trying to solve a puzzle, when in fact there is no real solution or vital missing piece. What we have is a mystery.

I would venture that the vast, vast majority of metrics/measure we use to look at consumer behavior are mysteries or parts of mysteries. Yet, in most cases, we believe them to be solutions to puzzles. Why? Because puzzle solutions are far easier to understand. You end up with a final solution, a factual outcome, a 'result'.

I think Scott asks some great questions, but they are all mysteries - and hence tough to solve.

The Brand Utility Mystery
A mystery if their ever was one - but a great thing to try and understand. Scott describes utility as more than 'usability'. It encompasses a brand's ability to understand and anticipate. Amazon being the gold-standard.

There is no one score than can encompass this though. It's just not possible (I've seen it tried). The big problem is tacit knowledge. It's too hard to divorce what a person knows from their perceptions of how easy or useful something is. Give the same utility test to top users of Amazon and Barns and Nobel and you will get the same result. You find utility in what you use often, for the very reason that you use it often.

So what would you need to do to get to this concept? Probably multiple measures that segment out new, recent and older users (to try and control for tacit knowledge). Apply different questioning techniques to each segment. Look at adoption times for new functionality across these groups. The goal would be to build up a body of insight, not a single measure. Less sexy, less useful... but it's a mystery!

Shareability/Network/Viral Effects
Another pure mystery, and Scott treats it as such. I'd love to know this one, I'd make a million dollars. An interesting question is why is this a mystery? Why is it so hard to predict what will and won't go viral? I think one of the reasons is that network effects are significantly more important than content. But you can't control network effects - who sees it, at what time, who they send it to, who they in turn send it to, what they say about it, etc, etc. There are way more great pieces of content (or conversations) that never went viral than there are networks with no content.

Participation
Again, a great mystery. What does encourage participation in a community? A host of factors have to come into play here - networks, interest, time, branding, incentives, hype, content, functions, fun, etc. The list is long. I'd love to see someone even try and model it! If you broke it down into manageable chunks - importance of just incentives for instance - you might learn something. As a broader question though, it's unknowable.

Linking Sales and Branding
This was the conversation I had with my ex colleague. Again, this delves into the realm of mystery. It also has to be the single most common question you get from Marketers - why are my sales going south but my brand metrics are strong? (or vice-versa). The truth is that there is never a clear relationship, never a stable relationship, and it's never solvable - it's not a puzzle. I worked for one of the fastest growing phone companies in the US in recent years and saw sales skyrocketing and brand metrics tanking - it made perfect sense. But it wasn't a situation that was typical and nothing you learned from it was useful to anyone other than that specific company.

Mysteries aren't 'general'. You can't derive learnings and apply to situations that look similar. I've blogged about this before, but if you really want to understand uncertainty, read The Black Swan - possibly the best book I've ever read that deals with what is and isn't knowable, and what to do about it.

Most of Marketing deals with mysteries. Beware people that package mysteries as puzzles - a common technique used by Marketing Research companies to sell you research. Beware consultants and industry pundits that package their experience as laws.

Sphere: Related Content