Friday, December 19, 2008

The Quest Continues: Finding Ways to Measure the Effectiveness of Social Media

Once again in my search for concrete metrics for the measurement of participation in the social media sphere, I've come across some interesting material that touches on the subject just enough to whet my appetite, but not really give me any foundation for building a strong business case for a social media campaign. Perhaps I've spent too much time with CFO's and bottom-line focused CEO's; I still believe that if you can't figure out how much money your efforts are generating, why would you participate at all?

My most recent nugget of golden info comes from an interview with Lawrence Swiader, Director of New Media at the National Campaign to Prevent Teen and Unplanned Pregnancy as published by The Buzz Bin. (I'm not up for debating the moral basis of the organization by the way -- we'll leave that for someone else's blog... I want to discuss the metrics and only the metrics.)

Swiader notes that the success of their social media campaigns (they use all the usual culprits like Facebook, Twitter, blogs and the like), will be measured firstly by the number of eyeballs and ears they reach. Secondly, a "higher level" evaluation will be undertaken to somehow measure how much information has been imparted and then stored in some way (I assume this means via an online vehicle and not 'stored in the brain'). Swiader doesn't give any details as to what this second evaluation might entail, but perhaps that is because it hasn't been attempted yet. It still sounds promising.

What I like about this interview is that although no hard numbers or formulas are given, it has reminded me that campaigns, social or otherwise, shouldn't be about just "throwing it out there" and then seeing what the results are, but that there has to be an ultimate goal, and then progress may be measured in relation to that goal. For the National Campaign, that goal is to reduce unplanned and teen pregnancy in the US, and therefore their social media goal is to reach as many people in their target audience as possible. We may not all work in the not-for-profit sector (I don't), but this simple methodology should still apply.

So let's apply these same simple principles to a business model I am very familiar with -- the business conference/tradeshow industry. Let's say that my ultimate goal is to generate as much profit as possible. To do this, I know that I need to attract as many attendees and big dollar sponsors as possible. My social media goal, then, is to reach as many of the eyeballs and ears of my two target groups as possible, which will yield some quantitative results. Then it seems to me that the next logical step would be to substantiate these findings with some sort of qualitative report that describes the quality and depth of the conversations and not just how many times something was read -- afterall, activity is an indication of interest. The combination of qualitative and quantitative results should provide a relatively accurate picture of the effect of my social media campaign.

But we can't stop there. Because in the conference world, and many other businesses, there remains an important question -- what does this mean in terms of bucks in the bank? For the National Campaign, it's important that they decrease the percentages of unplanned pregnancies. In some ways this is not dissimilar to ROI or revenue. At the end it's all about how the metrics you choose translate into a business case that poitns to progress you've made towards a larger goal.

So back to the money... In order to justify spend (time and money) on my social media campaigns, it seems logical to me to then connect my qualitative and quantitative findings in the following ways:
1) Plot highest levels of social media activity (buzz) with sales levels to look for patterns
2) Find a way to identify the bloggers, Twitter-ers and others who participate with my company in the social media sphere and determine if they eventually buy (No, I don't know how to do this... I will keep looking for ways to accomplish this formidable task, though.)
3) Survey my market to determine impact - poll them to determine if social media may a difference in their buying decision, ask where they heard about us, etc.

With a combination of these three, perhaps I might inch closer to a better understanding of the social media/profit equation. I have yet to find a true answer to the question, "how do you measure the effectiveness of social media?", but I do think that I'm closer to reformulating the question. Stay tuned.


  1. The keys, and they are largely under- practiced, are to identify what you want a certain group of people to do, and then influence those people to do it. There are three critical success factors: 1) you must reach the right people, 2) your message must be persuasive, and 3) they must act.

    In the example you cite, the people reached could be an array of those who will organize, work, and/or sponsor in addition to reaching the teens.

    I am an event marketing ROI consultant so your blog post came up in my alerts.

    You can find more information about measurement under the "Solutions" section of our web page and in my blog which is accessible from the same site.

    Ed Jones, Constellation Communication Corp.

  2. P.S. you can determine those who eventually buy , what they buy and how much they spend through post- event random sample research with the marketing base or event attendance at an event. This technique is in practice on a regular basis for many companies.


  3. Ed,

    Thanks so much for your comment!

    Here is a question, though... How do you then link the conversational activity to the buyer? For example, if someone interacts with my brand, how do I figure out if they purchased from me or not?

    Again, thanks for your comments - very insightful.


  4. Hi Katrina,

    Thanks for the reference and for your post. For the record, I did mean that it is my hope that people will store information in their brains and use it later. I plan to test for it in a longitudinal evaluation. So, if we succeed in producing eyeballs, we then need to understand whether the information has really been transferred. If so, we start looking for evidence of the intended end result: in our case lower rates of unplanned pregnancies.

    I feel strongly that we in the social media sector need to hold ourselves to higher standards of evaluation; we need to go beyond simple Web metrics--which become less important with every passing day anyway.

    The logic model is the key: awareness to knowledge, to action, to change, to ultimate impact. That impact could be improving the nutrition of children in Africa, freer press in Turkey, or increased market share for your company. If your logic model can't "make the trip," social media in any form won't get your constituents there.


    Lawrence Swiader

  5. Hi Lawrence,

    Thank you for your stopping by and for your comments. Glad to get clarification regarding the "storing information" part of your interview. I think I've been in discussion with so many online marketers I've forgotten about the idea of actually understanding something and not just filing an idea away.

    You mentioned in your comment that the next stage is to do a longitudinal evaluation. I must admit that I do not know what you mean. May I trouble you to clarify the term for me?

    I do agree that we need to go beyond web metrics. At the same time, I'm finding it very difficult to prove that a campaign is successful without any kind of hard metrics like those applied to the web. I realize this is a bit close-minded; ideally I'd like to marry some kind of hard metrics with qualitative analysis that speaks to the depth of understanding or "buy-in". I just haven't found a way to reconcile these in a way that makes them meaningful.

    Thank you for the logic model - that's brilliantly simple. It's really the same model that's been applied to just about every type of marketing, isn't it? But it really helps to spell things out.

    You've given me a lot to think about - thank you so much!


  6. Hi Katrina,

    By "longitudinal," I just mean over some period of time. The time would allow us to understand whether the desired impact has occurred. This kind of evaluation, if over many years and a wide geographical spread, is very expensive.

    At the moment, I am thinking about an evaluation of a local implementation (pick two small cities) before going national.

    Regarding Web metrics, I totally agree. I love the hard metrics to which we have such easy access. It's just getting harder to understand those metrics and to gather them since so much activity happens in places that are not the "destination Web site" where the log files reside. This is true even for shopping sites where the final outcome is a purchase via a shopping cart because that's only part of the story. Sites that review products need to be taken into account among other things. That's the part that's hard.

    Good luck with your work!