Facebook Audience Testing Case Study: Elijah McCoy Campaign

The goal of this project was to see if we could effectively market our content to an audience outside our core demographic in a data-driven manner. We picked this video as the test subject. The success of this experiment validated our strategy for the YEAR project.

1. Identify goals

Success metrics for this campaign included:

  • Engage non-core audience
  • Identify which of the target personas our list resonates with the most.

2. Brainstorm audience personas

We reviewed the video and discussed what kinds of audiences would engage with it.
We identified four audience candidates:

  1. FEE Donors
  2. Black Entrepreneurship Fans
  3. Inventors and Makers
  4. Mike Rowe Fans who like Entrepreneurship

We tested four additional placements to display the content:

  1. YouTube link from the FEE page
  2. FEE.org article on the FEE page
  3. Facebook add (content specified within ad)
  4. Facebook Video Upload

We excluded Donors from the additional placement test, this left us with creating 12 ad sets (4 placements x 3 audiences, each containing 1 ad).

3. Create saved audiences

Each audience was iterated, so the below represents the final combination of criteria. We identified a number of people, hobbies, and shows which our personas might have in their Facebook profiles, but only the below were available as targeting options:


4. Measure test results:

Placement
In terms of placement, we saw that ads with the YouTube link, FEE.org article, and in-ad content were not responding at all. The per-engagement maximum was set at 15 cents, and there were only 1-4 engagements out of 40+ reach. Cost per engagement was 3-14 cents. Once we changed format to Facebook-hosted video, reach and results immediately took off, at 1 cent per engagement.

Audience (Persona)
Out of the three audience we tested, the “Black Entrepreneurship” audience responded the strongest. Here are the video engagement stats for the first day:


Across all metrics, “Black Entrepreneurship” outperformed other audiences. Donors also performed well, as expected.
5. Boost winning ad set

Based on the first day, we decided to boost “Black Entrepreneurship” to $100 per day and “FEE Donors” to $30 per day. All other ad sets (and audiences) were stopped.

6: Measure results

Delivery Summary
  • $440 was spent boosting this video, with $38 used for the first day of calibration, and the rest directly on the selected audiences.  
  • A total of 322,000 people were reached. 178,000 of those were from direct paid reach, while the rest were organic. However, once advertising stopped, video views dropped for 30,000+ per day to well under 1,000. The majority of organic views originated from people who saw the ad.
  • After the ad campaign ended, organic views dropped rapidly from over 10,000 to a few hundred (see below). This video is not effective with our existing audience, and did not have a lot of organic momentum.
Return on Investment
  • There were a total of 111,000 video views, and 8,000 reactions. Each dollar thus generated 252 views and 18 likes.  
  • Cost per 1000 impressions was $1.50 for the “Black Entrepreneurship” audience and $8.03 for the donor audience.
  • The FEE Store link was clicked 301 times.  There were 28 Real Heroes sold during the ad period vs 26 for the several months prior, so the ad likely drove virtually all of those sales. Facebook matched 7 purchases via Offline Action tracking (each purchase may involve more than 1 book.) 



Open Questions
  • How many of the users who engaged with the ad will become frequent readers?  We’re not sure how to track this. (Google Analytics? Facebook Insights?)
  • How to measure the value of paid ad exposure? Book sales? Shares? Organic uplift? Are paid views as such worthwhile?
  • Given that our existing audience was not the most responsive to this video, and given that we were attempting to target groups outside of our core user base, is it possible that we need to reach a larger number of people with paid views in order to reach some unknown saturation point that will spark more organic growth? In other words, do we need to spend a lot more money before network effects will take over?
  • Once we’ve effectively identified an audience target through testing, would different means to reach those audiences be more effective than paid social media ads? For example, should we have planned an advertising strategy that included direct outreach to major audience influencers around Black Entrepreneurship, such as Daymond John?

Lessons Learned
  • Targeting segments effectively requires testing different audiences.
  • We can effectively boost content to new demographics and even get them to buy products, but converting them to returning visitors will require a new strategy.
  • We need a clear call to action for ads to measure effectiveness and extract value.


Leave a Reply