Proof in the Pudding

A Vocabulary Lesson - Rigor and Diversity (or - Two ways to measure the strength of your presence on Wikipedia)

2010-03-28T21:28:00.000-07:00

My title's ridiculous - I know, I know. It is also ridiculous that my poor blog has been neglected for so long... but no longer! Here we are, delving back into the pudding to find the proof... in other words, trying to figure out how to measure the success of marketing tactics, campaigns, etc.

My recent web wanderings have brought me to a blog post by Katie Paine, who via her blog has introduced me to two new terms of which I've grown rather fond of, rather quickly... "rigor" and "diversity". Paine uses these terms in reference to measuring your brand's strength on Wikipedia. And since, as Paine attests to, Wikipedia dominates organic search listings, this is something that we, as marketers, need to be paying attention to.

"Rigor", again, according to Paine, refers to the number of times an article is edited. "Diversity" refers to the total numbers of unique users (note - not readers, but users). And apparently, especially if you're a big brand, then these stats can be indicators of brand strength - as can measuring the sentiments of what is written about your brand in Wikipedia.

While I value the importance of non-financial measurements, I cannot help but wonder how this information, once gathered, might be acted upon. If you have poor Wikipedia rigor and diversity, what do you do? Or are these just symptoms of a greater disease for which the cure is a brand strategy overhaul? And if this is the case, what are some of the other factors that should be considered? While I love measurement, sometimes it's difficult to seperate what is essential to measure as opposed to what is simply interesting. Food for thought.

Extreme Marketing: Live Video and Tweets from the Operating Room

2009-08-01T23:43:00.000-07:00

No, your eyes do not deceive you. This blog post really is a response to a New York Times article about hospitals that broadcast and tweet from operating rooms as a marketing tactic.

As with previous blog posts, my aim is not to make any sort of value judgement about the moral basis of this type of action. I'm interested in metrics (although the idea of the live broadcast of a surgery does fascinate me).

According to the article, although little data is published therein, webcasts of surgery and other such reports via social media of the ins and outs of surgery seem to be working. They're attracting viewers, followers and even new patients - all quantifiable points of analysis. And for hospitals in the US, attracting patients is key to maintaing a healthy bottom line. Health care is, afterall, a business.

It is so interesting that with all of the talk about how social media is difficult to measure, that the hospital mentioned in the above mentioned article uses such simple web-based metrics as well as "customer" (patient) feedback.

The Battle Over Metrics: Is ROI an appropriate measure for social media and brand?

2009-03-08T21:19:00.000-07:00

My new position in the not-for-profit world has had me thinking a lot about marketing measurement and not only how to prove "success", but also how to best allocate funds across different initiatives in order to get the maximum gain (in my case, for the organization). How do you determine that if you pulled some funding from one initiative in order to allocate it to another that you would end up with a stronger ROI? And what about the areas of marketing that are not as easily "measurable" in terms of ROI, like social media, for example? How do you decide to pull money out of the marketing budget fot these areas over "tried and true" direct mail campaigns, etc.?

Through the wonders of the internet, I found an article that discusses just this topic. It was written by someone by the name of Ron Shevlin , in his blog Ron Shevlin's Marketing Whims.

Shevlin refers to social media, as well as branding, as "infrastructure", and so when we allocate funds for these endeavours, we are in fact investing in infrastructure. This is an interesting approach. I suppose the same could be said of PR. Shevlin states that these areas are often measured in terms of KPIs (key performance indicators), but that these are not adequate performance measures. Shevlin claims that these are not good indicators of incremental changes in performance or of their effect on the whole marketing funnel. I agree. I think they are also rather arbitrary.

This puts online marketers, branding folks and the like on the back foot when they try to explain their efforts to marketers obsessed with ROI (like me). Afterall, what is the ROI of having a company Twitter account? You tell me. You see, I think it all boils down to the fact that consumers (be they companies, donors or individuals) are complex creatures. They (we) don't make buying decisions based on only one factor. So while logically most of us would say that, "yes", the brand name was important when we bought our MP3 device, we would be hard pressed to report how influential that brand was in our buying decision or whether if presented with another item of the same value, but weaker brand we would still make the same decision.

At the same time, I don't buy it that social media, PR, WOM and all the other forms of "alternative" marketing (I can hear my colleagues in these fields screaming at me now...), can't be measured. Sure, maybe these areas are harder to measure than say, direct mail, but surely not impossible. Okay, so what if we can't have perfect forms of measurement? We should still be able to show progress. We should collect what info we can. We should watch for trends. And with enough data, surely, surely, we should be able to show some correlation between these trends and ROI. Then we can take a crack at determining how much of our marketing budgets should go towards "infrastructure." Don't agree? Comments are open, folks.

Wine for Conversation: An Example of Calculating the Value of Online Influencers

2009-02-08T23:26:00.001-08:00

Those of you that know me personally will know that if offered a drink, I almost always opt for a glass of Australian red. For those of you that DO know me personally, it won't surprise you that in a short fit of random Google searching, I used the keywords "wine" "marketing" and "ROI." The best result? An intriguing blog post by an Aussie marketer by the name of Matt Granfield.

Granfield, who works for a digital marketing agency called e-CBD, agreed to a social media "exchange" as he calls it wherein Kirrihill Wines sent him twelve bottles of wine for instigating/creating a conversation about their products.

In his post, Granfield describes why he is deemed by Kirrihill to be an influencer. Essentially it comes down to a few simple numbers - how many people can he reach with his online network as well as a subjective judgement as to how much clout he has with this group and with the wine industry.

Granfield then uses some made-up, but realistic statistics to predict the actual value of sending him some bottles of vino - how many bottles he himself is likely to buy and how many bottles his friends and members of his network are likely to buy. This obviously makes sense, and is a nice simple, clear-cut example of how small investments in nurturing influencer marketing can really pay off.

There are other benefits to influencer marketing of this type that Granfield's blog post doesn't mention. Firstly, his post make me picked up by other bloggers... and well, the proof is in the pudding in this case, no? (You know I had to write that... I couldn't resist). So in fact, Kirrihill Wines is actually getting exposure to not only Granfield's network, but to mine as well (which, not to brag, is quite large - check out my LinkedIn profile for proof or follow on me on Twitter). Plus, links to the winery from Granfield's pages, as well as this blog will aid in organic search engine result positioning.

Important Side Note:
Australians living in various parts of Victoria are currently facing an enormous tragedy with bushfires burning out of control across the state. Victoria is a large wine-producing area, along with being the home to over 5 million Australians. Please join me in making a donation to the Australian Red Cross to help support our Aussie friends in crisis.

What's Next, Next Director?

2009-02-02T06:13:00.000-08:00

Recently, I interviewed marketing and strategy consultant Michael Field (@michaelfieldcom) regarding his new LinkedIn group Next Director, how he promoted it, and how the group has grown.

Katrina: What is Next Director and what are the benefits to becoming a member of the group?

Michael: Next Director is a global online community of more than 300 emerging and aspiring company directors who connect via the LinkedIn online network. The benefits to members include access to a global network of aspiring, emerging and practicing company directors, governance experts, regulators, recruiters and others interested in directorship.

Next Director provides practicing directors with the opportunity to contribute to the director community through sharing their experience and connecting with emerging directors.
Aspiring and emerging company directors can connect and engage with experienced directors, recruiters and regulators to obtain more practical insights into the role and requirements of a company Director.

Specialist executive recruiters and recruiters for board positions have also shown a strong interest. I expect that to be a positive relationship for all participants.

Katrina: How many members are a part of Next Director, and how did you go about attracting them?

Michael: Next Director was launched on 22 November 2008, it has already attracted more than 300 directors globally, including Australia, United States, Canada, Chile, United Kingdom, Ireland, Netherlands, Belgium, Switzerland, Russian Federation, United Arab Emirates, Japan, China, Hong Kong, India and Singapore. Experienced directors were hand-picked from my own network as well as those whose career path is moving beyond management and into director roles.

The marketing was conducted exclusively online through social media networks such as LinkedIn, Facebook and Twitter. The viral effect has been quite extraordinary and has produced a number of interesting results:

1) The quality of applicants has been extraordinarily high. I attribute this to the calibre of the founding members and the professional focus of networks such as LinkedIn.
2) The global reach was immediate and very experienced directors have joined, showing a great interest in contributing their expertise.

Katrina: Who did you expect your target audience to be, and what are the demographics of your current group?

Michael: To join Next Director, your career and experience must demonstrate that you are managing those aspects of your career that make for a suitable director. This can include formal education and skills gained through employment or other community involvement. It is a difficult criteria to define, but Next Director is clearly focused on Directorship, not management issues.

The demographics of the group are interesting:

Location
47% are from Australia
30% are from the United States
7% are from the United Kingdom
The balance are evenly distributed across the countries mentioned earlier.

Role
50% are in senior management positions exploring directorship as a next step
16% are executive directors
2% are Chairmen
Academics, regulators and governance advisors represent much smaller numbers.

Katrina: What are your future plans for Next Director?

Michael: When the group reaches ‘critical mass’ and additional services are required (such as a dedicated website) I will respond to that need. I have looked at a number of funding models and obviously in the fast-moving social media market there are many exciting options available.
I have met with a number of service providers including specialist executive search firms to consider developing additional services.

I see digital distribution of content as a key platform for growth of Next Director.

About Michael Field

Michael Field is a strategic marketing consultant based in Sydney Australia.He has worked in a number of senior marketing roles in listed technology companies. As the former General Manager of Member Services and Marketing for the Australian Institute of Company Directors he held wide ranging responsibilities including the ASX Top200 programme.
Michael has served on the board of Save the Children Victoria and is a former guest lecturer in marketing at ECU in Western Australia.
He now manages his own marketing consultancy at http://www.michaelfield.com/

If you are not already a Next Director member on LinkedIn you are invited to join at: http://www.linkedin.com/e/gis/1331907

Links

Follow Michael on Twitter

Follow Next Director on Twitter

Join Next Director

Michael Field's Consultancy Site

Exclusive Interview with Zappos' CEO: Measuring Success Through Customer Feedback

2009-01-26T11:50:00.000-08:00

According to Tony Hsieh, CEO of Zappos, the proof of the pudding really is in the eating - so to speak. When your focus is providing the absolute best customer service above all else, feedback from your customers is all the proof that you need to show that your marketing is working. Read on for Proof's exclusive interview with Tony.

Katrina: When did you launch Zappos Canada? What metrics did you look at before deciding to cater to this market? Do you have plans to launch any other geographically-specific sites?

Tony: We launched Zappos Canada several years ago in response to requests from Canadians that wanted to be able to shop from us. We actually did not look at any metrics. We do not currently have plans for othergeographically-specific sites, but we do ship internationally to some countries from our regular Zappos site.

Katrina: What types of marketing do you employ at Zappos, and how do you measure the success of your marketing programs?

Tony: Most of the money that we would have spent on paid advertising we instead put into the customer experience (such as free shipping both ways, surprise upgrades to overnight shipping, 24/7 customer service, etc.).We've grown from no sales in 1999 to over $1 billion in gross merchandise sales in 2008. The #1 driver of that growth has been from repeat customers and word of mouth. We really view anything we do to improve the customer experience as a form of marketing because it drives word of mouth.

Katrina: It's clear that customer service is key to everything Zappos does. How do you measure the success of your customer service initiatives?

Tony: We receive comments from our customers and we also randomly survey our customers.

Katrina: As your main sales vehicle, your website is crucial to your business. What types of improvements have you made to this site over the years, and how did you build up the business case for each of these?

Tony: We are actually in the process of launching a new site, which we are internally calling "zeta". You can find it here:

http://zeta.zappos.com/

Our original site was designed primarily to sell footwear. Today, we sell a lot more than footwear, including clothing, accessories, and even electronics. Zeta is designed to handle all the different product types.

Katrina: How is your marketing team structured and how large is your team?

Tony: We have a direct online marketing team that handles our keyword buys on search engines such as Google as well as our associate (affiliate) program. We also have a brand marketing team that handles some of our brand advertising, such as print or TV. Combined, our marketing team is about 15 people.

Katrina: What is your vision for Zappos ten years from now?

Tony: Hopefully 10 years from now people won't even realize we started selling shoes online. We just want the Zappos brand to be about the very best customer service and very best customer experience.

For the Zappos site, click here
For the Canadian Zappos site, click here
For Tony's corporate bio, click here
Follow Tony on Twitter

Are Email Open Rates Still Relevant?

2009-01-11T23:11:00.000-08:00

I launched my first-ever email marketing campaign in 2004 (just call me a late bloomer - the industry I was working in wasn't an early adopter to say the least). At the time, the most common metrics, and the ones I used to measure the success of my email campaigns, were click-throughs and ... yes, it's true... open rates.

Today open rates are viewed by many of my esteemed colleagues as passe and irrelevant. Afterall, it's not practical to assume they are anywhere near accurate due to factors such as the "preview" option that many email clients allow and the option to open an email without downloading images. (For the uninitiated, an email "open" is recorded when a server requests the download of a hidden image.) Open rates, though, used to be all the rage. Well, really, they were pretty much the only metric marketers had in their arsenal at the time.

Another major issue with using an open rate as a key metric is that it does not show any sort of "engagement", and therefore many pros dismiss the idea that open rates can tell us anything about the success of a campaign, since engagement is really what we're after.

There may be hope for open rates, yet, however. Although they may not be the be-all and end-all of email marketing metrics, they are still good for something. In a guide on email open rates written by Mark Bronlow in August 2006, Bronlow suggests that although benchmarking is perhaps not very useful because of a lack of standardization in the way open rates are calculated, that open rates may still be used effectively for comparison of different email campaigns launched for the same product at the same time. For example, it may be used to determine the best subject line for an email if the subject line is the only variable between two otherwise indistinct email creatives. Falling open rates of a regular newsletter, are also a good metric to watch, as this can indicate reader fatigue, and suggests that you may need to refresh your content.

Bronlow also suggests that open rates may be examined by market segment, and that this can yield some useful information. For example, Bronlow's advice is to monitor the open rates of groups from different sign-up sources. If they are low from a particular group, then this should be investigated.

Taking this one step further, I suggest that open rates might be looked at in relation to the different segments of your list divided by seniority, industry, geography, etc. If any of these groups show a low or high open rate, it may be time to gear a piece specifically to serve this segment.

Back to Basics: Calculating ROI

2008-12-25T22:52:00.000-08:00

Most of the marketing positions I've held have required me to demonstrate how my department's activities have produced a significant return on investment (ROI). I've learned that this seemingly simple concept is not as cut and dry as it might appear - even if we leave social media marketing and perhaps even traditional advertising out of the equation. Yet demonstrating the value of marketing activities has always been how I've proved that what I am doing is worthwhile.

Jim Novo, author of Drilling Down, makes the point that because there are so many ways of calculating ROI, the best way to calculate ROI is to use an equation that your company (ie your Board or CEO or CFO) finds relevant. How true. (You can read Novo's full article here.) Perhaps this should be step one in any quest for demonstrating accountability in the marketing department - determine what metrics the big bosses want to see.

In the earlier parts of my career, I did just that. When I calculated ROI, I took into account only the cost of acquisition and the resulting profit of a campaign (this was all that my "big bosses" wanted to see from me for a long time). As I progressed and became the manager of a department, however, I had to account for additional factors including the cost of staff salaries (my "big bosses" became even bigger bosses - CEOs and the Board and the like). My understanding of ROI changed as a result, and has continued to change with increasing responsibility and "higher" perspectives. The "higher up" I go, and the more I reflect, the more and more factors I believe I need to take into account. The more I learn about measurement, the more I believe that the results of the calculations I have relied upon only give me a glipse into the effectiveness of my marketing activities, and therefore how limited I have been in my ability to substantiate the value of marketing activities that relate to longer-term profitability goals.

The type of ROI calculation I am most familiar with - ie profit/cost of acquisition, is what Novo refers to as "simple ROI". As Novo points out, although this type of calculation is important, the results demonstrate only "front-end profit." These results ignore other important factors like the effect on the brand, customer loyalty and repeat purchases. According to Novo, these are all factors that can be measured, too, but the calculations are somewhat more complex. I'll delve into these areas in due course in this blog. I have always known that these factors were missing, but to date I have been unable to really find adequate methods for quantifying marketing's contribution to these areas.

In regards to web metrics, Novo suggests a very interesting model. It seems rather straightforward. In order to calculate the ROI of push campaigns where information is gathered through a company's website, especially where the campaigns are complex, Novo suggests that we might examine the revenue over any given period divided by the number of, let's say, unique visitors to the company website to determine how much each unique visitor is "worth". For example, if my revenues over one week are $1000, and I have had 100 unique visitors to my site, then by this calculation, each unique visitor is worth $100 to me.

I think this is a useful calculation, but I'm not sure I'm satisfied with assuming that my revenue is directly related to the number of uniques to my site. I think there are too many factors at play for this to be an accurate measurement. Still, I think that demonstrating the relationship between increases in uniques and increases in revenue are definitely worth looking into. If these were plotted in a graph over several months, or preferably, over several years, this would yield a benchmark for the ideal number of unique visitors to help stimulate revenue. I find this intriguing. I think this might be the case for a number of different methods of measurement - they are incomplete on their own, but taken with other relevant metrics they become more useful.

In fact, the more I come to understand about measuring the effectiveness of marketing activity, the more I believe that any marketer worth her salt needs to create a system of interconnected metrics to attempt to show progress towards company goals. Granted, this picture will always be incomplete because we cannot measure absolutely every effect for want of time, however if we choose a variety of the most important metrics, we will surely arrive at a more complete picture... and that complete picture may just save our marketing budgets (or our jobs), and I am convinced will absolutely allow us to speak from a stronger position when the big bosses want to see measurable results.

The Quest Continues: Finding Ways to Measure the Effectiveness of Social Media

2008-12-19T19:13:00.000-08:00

Once again in my search for concrete metrics for the measurement of participation in the social media sphere, I've come across some interesting material that touches on the subject just enough to whet my appetite, but not really give me any foundation for building a strong business case for a social media campaign. Perhaps I've spent too much time with CFO's and bottom-line focused CEO's; I still believe that if you can't figure out how much money your efforts are generating, why would you participate at all?

My most recent nugget of golden info comes from an interview with Lawrence Swiader, Director of New Media at the National Campaign to Prevent Teen and Unplanned Pregnancy as published by The Buzz Bin. (I'm not up for debating the moral basis of the organization by the way -- we'll leave that for someone else's blog... I want to discuss the metrics and only the metrics.)

Swiader notes that the success of their social media campaigns (they use all the usual culprits like Facebook, Twitter, blogs and the like), will be measured firstly by the number of eyeballs and ears they reach. Secondly, a "higher level" evaluation will be undertaken to somehow measure how much information has been imparted and then stored in some way (I assume this means via an online vehicle and not 'stored in the brain'). Swiader doesn't give any details as to what this second evaluation might entail, but perhaps that is because it hasn't been attempted yet. It still sounds promising.

What I like about this interview is that although no hard numbers or formulas are given, it has reminded me that campaigns, social or otherwise, shouldn't be about just "throwing it out there" and then seeing what the results are, but that there has to be an ultimate goal, and then progress may be measured in relation to that goal. For the National Campaign, that goal is to reduce unplanned and teen pregnancy in the US, and therefore their social media goal is to reach as many people in their target audience as possible. We may not all work in the not-for-profit sector (I don't), but this simple methodology should still apply.

So let's apply these same simple principles to a business model I am very familiar with -- the business conference/tradeshow industry. Let's say that my ultimate goal is to generate as much profit as possible. To do this, I know that I need to attract as many attendees and big dollar sponsors as possible. My social media goal, then, is to reach as many of the eyeballs and ears of my two target groups as possible, which will yield some quantitative results. Then it seems to me that the next logical step would be to substantiate these findings with some sort of qualitative report that describes the quality and depth of the conversations and not just how many times something was read -- afterall, activity is an indication of interest. The combination of qualitative and quantitative results should provide a relatively accurate picture of the effect of my social media campaign.

But we can't stop there. Because in the conference world, and many other businesses, there remains an important question -- what does this mean in terms of bucks in the bank? For the National Campaign, it's important that they decrease the percentages of unplanned pregnancies. In some ways this is not dissimilar to ROI or revenue. At the end it's all about how the metrics you choose translate into a business case that poitns to progress you've made towards a larger goal.

So back to the money... In order to justify spend (time and money) on my social media campaigns, it seems logical to me to then connect my qualitative and quantitative findings in the following ways:
1) Plot highest levels of social media activity (buzz) with sales levels to look for patterns
2) Find a way to identify the bloggers, Twitter-ers and others who participate with my company in the social media sphere and determine if they eventually buy (No, I don't know how to do this... I will keep looking for ways to accomplish this formidable task, though.)
3) Survey my market to determine impact - poll them to determine if social media may a difference in their buying decision, ask where they heard about us, etc.

With a combination of these three, perhaps I might inch closer to a better understanding of the social media/profit equation. I have yet to find a true answer to the question, "how do you measure the effectiveness of social media?", but I do think that I'm closer to reformulating the question. Stay tuned.

Can trust be measured? Daryl Mather thinks so.

2008-12-17T20:25:00.000-08:00

Related to loyalty, but not to a loyalty program, per se, Daryl Mather, Senior Reliability Consultant at Meridium wrote an interesting post about measuring trust. The article seems to target professional consultants, however the concepts apply to all types of, well, "reputation marketing" for lack of a better term.

What I love about the article is that it actually talks about REAL NUMBERS. And for me, REAL NUMBERS equates to accountability.

Mather suggests that trust can be measured in the following ways:
- % of business from repeat clients (Mather says 30% of your work should come from repeat customers)
- % of business from both passive referrals and auto-referrals, where passive referrals come from clients who refer you to others, and auto-referrals are the result of your blog, press about you, etc.
- quote strike rate - ie, how many quotes are accepted vs the number sent out

These are really simple metrics that could be used for many purposes. Substitute "quotes" for "personalized letter and brochure" and you have yourself a recipe for direct mail metrics, for example. Mather's post focuses on a client-consultant trust relationship, but this could easily be translated into a consumer-brand relationship. Essentially using Mather's metrics as a foundation, you could build a scorecard for any type of business trust relationship.

Measuring Social Media Marketing

2008-12-13T19:46:00.001-08:00

In doing some research for this blog, I came across a great little article, written over a year ago by Michael Brito, Senior Marketing Manager, Yahoo! for the Search Engine Journal.

It's worth a read.

Brito suggests that measuring the effectiveness of social media isn't all that complicated. If you're using social media to drive traffic to your website, then you can use traditional web marketing measurement metrics (say that three times fast) such as "unique visitors", "time on site", etc.

In terms of measuring social media "at the source", so to speak, the metrics are quite similar and essentially boil down to how many interactions are had between target audience and interface.

At the end of the article, Brito points to the idea that these metrics still have to be related back to revenue, but doesn't give any suggestions as to how this might be done. I see this as key - afterall, if we don't know how much money ouractivities are bringing in, then is there a point to participating in the whole social media sphere at all? Looks like I'll be doing some more research on this topic.

How do you measure the value of a loyalty program?

2008-12-11T19:58:00.001-08:00

Today I attended a roundtable presented by the Toronto Chapter of the American Marketing Association entitled "Cracking the Code: The Future of Loyalty Marketing". The panel included Steven Allmen, GM, Business Development; David Soberman, Professor of Marketing, Rotman School of Business; Pat McGoey, Vice President, Marketing Solutions, Loyalty Program Consultant; Rubina Havlin, Managing Director, Credit Cards, Scotiabank and Chris Whitaker, Senior Vice President, Partner, OSL Marketing. The panel was moderated by Jim Warrington from Fantail Communications (who is also a Past President of the AMA).

Loyalty programs have always been of interest to me, but I have only really dabbled in them in my professional life. So today's session was particularly interesting to me. I came to the session wanting to know how they work, and of course, how the success of loyalty programs may be measured.

So how do you measure a loyalty program? Is it by the sheer number of customers that sign up for a particular program? This hardly seems sufficient since as many of the panelists pointed out, many of us carry around wallets full of plastic cards from programs we've abandoned long ago. If not that, then what other variables may be measured? Purchase value? Frequency? All of these are surely worth measuring, but at the end of the day, do they mean anything? Even in combination I am not convinced that this even scratches the surface.

One of the panelists (if this is you, please feel free to let me know and I will amend my post), suggested that it comes back to "lift, shift and retention" (among other factors). I see how this is the case, but I am still baffled as to how this information could be coupled with other data as well as qualitative feedback in order to distill any clear sense of "value". What do you say to a CFO who asks what the dollar value is at the end of the day?

Sorry, folks, but I don't have any answers. If you do, I invite you to share them with me.