Searching for the real value of Facebook marketing

A case study showing how to research the impact of Facebook on sales

The journey of measuring Facebook success has been fast paced. It started with how big your fan page was (counting likes), then to visits from social profiles, to getting very excited by the concept of Engagement Rate. Some opinion believes social media is best used to drive brand awareness, others for customer service. And there are brands that use Facebook as a successful sales channel.

There have been frameworks and methodologies as well as plenty of "proof" (largely published by service providers in the social media industry), yet the fact still remains that demonstrating the value of a Facebook page or Twitter account is still hard in 2013, especially if you're not a retail brand.

The reason? For travel brands like us it's hard to imagine that anyone is going to see a post of ours on Facebook, click through to our website and buy a £2000 holiday. It's just not going to happen. Yet, the way we've traditionally measured success online has been using this exact method i.e. last click attribution.

But for some time many social media marketers have believed (and hoped) that there's value earlier in the customer journey that last click attribution can't track. The general idea being that a customer sees something on Facebook or Twitter which sparks or encourages a journey along the path to conversion. For a travel brand this might include a brochure request, an email sign up, repeated web site visits and, just perhaps, a future holiday purchase. Of course, for existing customers there's the promise of staying 'front of mind' by appearing in their news feeds each day.

We recently carried out some research to seek an answer to some of these questions, using a ski brand's Facebook page, which at the time had around 30,000 likes. First, we posted a 20 question survey to the Facebook community and offered an incentive for completed responses. We received ~400, ~200 more than the previous survey we'd run without an incentive.

The focus of the survey was to better understand our community and what the benefits might be for the brand. Some of the questions were taken from this study by Forrester which focuses on current and future purchase intent as well as the propensity to recommend a product or service.

I've shared some of the results of the survey below.

The second approach used attribution modelling (Crystal has used Tagman since 2011) in an attempt to understand how Facebook was impacting the path to conversion. Over 12 months of data was used to analyse the paths which had Facebook as a touch point.

Here's what we found out.

The survey

Q. The first question we asked was whether they were a customer and if they were, how many holidays they'd been on?

82% of responses were from customers (not an unsurprising number). We then used the results from this question to segment the responses from the following questions, so we could get a more accurate idea of how our customers (the main focus) differed from the non-customer responses.

Q. We asked why they'd liked the Facebook page?

There's lots of research out there that will tell you people follow brands to access discounts and deals. That may be so, but it's clearly not the case here for Crystal. It was also great news for our social media team as there's a lot of people out there who clearly see the brand as a publisher.

Understanding the answer to why your community exists is clearly a fundamental step in improving things moving forwards. The results to this question also prove that each community is different and relying on secondary research isn't always reliable.

Q. And we asked whether they had ever done any of the following actions after seeing our brand's content in their news feed?

The results here were really encouraging, suggesting that there are plenty of valuable actions that result from the brand publishing content to its community. What's also interesting is the range of outcomes that the channel seems to be impacting upon.

Q. Here we asked how likely they would be to recommend us to friends and family (1 = they wouldn't, 5 = they definitely would)

This was one of the Facebook Factor questions from the Forrester study. The results suggest Crystal has a high percentage of customers who would recommend the brand to friends and family. A simple improvement to this would have been to ask the same question to the brand's email database to see whether there were noticeable differences in the propensity to recommend based on the channel. What we could take from this is that we have a lot of highly engaged brand advocates in our community.

Using the same scale, we also asked how likely it would be that they'd book their next holiday with us. Over 87% gave us a score of 3, 4 or 5.

Overall, these results were very positive. We'd heard back from a significant number of customers, with a large number expressing both a high interest in purchasing and recommending the brand in future.

Attribution results

First of all we were keen to understand where in the path to conversion Facebook actually played a role. We used what's becoming a standard classification in attribution modelling: namely, an introducing role, an influencing role or a converting role, (depending on where the channel was found in the path to conversion).

Of the paths studied with Facebook in the path to conversion, 67% of the time Facebook was found in the middle of the path, therefore playing an influencing role. 30% of the time it introduced customers to the brand and, perhaps most crucially, only 3% of the time did it play the role of converting channel. Click to enlarge the image below for a summary.

Using different attribution models

As I mentioned at the start, using last click attribution isn't going to necessarily reflect the value a brand is getting from a channel like Facebook. When we compared the percentage revenue attributed to the channel using different attribution models, the difference was stark (a variation of 18%).

We also found the average number of touches in the path rose if Facebook was one of those touches (see below), perhaps suggesting customers on Facebook are more engaged with the brand (being connected to the brand via more channels). And it wasn't all positive news. Interestingly, average revenue dropped by 18% if the path to conversion included Facebook. Lots of reasons could explain this, the obvious one being that the occasional flash sales the brand runs, reduces the average price of a holiday bought by customers using this channel (put another way, the brand's tactics play a part here).

Non-customer results

When we looked at the non-customer data, we were surprised to find that there was a lot of positive sentiment towards the brand, even though they had, in many cases, no other connection with the brand and no purchase history. For example, an overwhelming majority (~87%) answered that they would recommend the brand to friends and family.

We also asked what action, if any, non-customers had taken after seeing the brand's content on Facebook. More than 50% said they'd visited the brand's website, 40% that it had made them excited about an upcoming holiday and more than 30% said they had researched a destination they'd not previously thought of visiting.

These results were despite them never having booked a holiday with the brand.

Of course there are lots of 'what ifs' within these results but the general trend and attitude the non-customer responses showed the Facebook community as a good way of beginning a conversation with people who had an interest in what Crystal do, but who've yet to travel with the brand.

A Final caveat

There are holes and problems with all research that's carried out, including mine. Each brand's social media will vary dependant on a whole host of things. However, we learnt a lot about our community simply by asking them a few questions. It also won't be long before analytics programs are providing accurate multi-model attribution modelling at the touch of a button. Google Analytics Premium is already half way down that path: