An Experiment: Testing Facebook’s News Feed Filtering Algorithm

Testing Facebook News Feed Filtering

Facebook recently reported in a sales PDF that organic Reach is going to drop due to competition in the News Feed. Many marketers were upset by this, claiming they were once again being forced to pay for ads — led by an inflammatory post by AdAge.

I have plenty to say on that topic. But I want to be as scientific in my response as possible. And it can’t all be said in a single blog post.

As a result, I’m going to start slowly. Today’s goal is to dig deeper than the typical reactionary blogger and marketer when it comes to the quality and composition of the News Feed.

Back in August, Facebook shed light on how they determine what you see in your News Feed via their filtering algorithm (also known as EdgeRank). Facebook’s goal, they said, was to show you more of what you care most about and less of what you don’t.

Of course, brands in particular balk at this. We hate that we don’t reach 100% — or 20, 30, 40 or 50%. So instead of testing this, many brands straight up assume they are being screwed.

Now that reports indicate that organic Reach will drop further due to increased competition in the News Feed, I figured this was a good time to take a closer look at what is shown to me.

As a result, I am putting Facebook’s claim that they show me the content I care most about to the test. In particular regarding content shared by brand Pages.

[Tweet “Does Facebook show you what you want to see from brands in the News Feed? Here’s an experiment…”]

The Experiment

Facebook said that the typical user would receive approximately 1,500 stories per day from friends and Pages if the News Feed were unfiltered. Instead, users are shown only about 300.

[NOTE: Keep in mind this includes a TON of light-weight stories. Friend commented on this post, liked that post. Friend checked in here, played an app there. Trust me, you don’t miss the majority of those 1200 stories!]

So the first thing I did was go to my News Feed and open a spreadsheet. I then went story by story and documented the source of those stories covering the past 24 hours.

If the source of that story was a friend or a public figure I follow, I simply classified that as “Friend.”

If the source of that story was an organic post from a brand Page that I like, I wrote down the specific brand that it came from.

And if the source of that story was an ad — regardless of whether I otherwise Like the Page associated with that ad — I classified it as “Ad.” Note that this would include any Sponsored Stories that mentioned my friends.

A few things I was looking for…

  • Are organic posts from Pages being squeezed out?
  • Are ads taking over the News Feed?
  • Are the posts I’m seeing from brands what I actually want to see?
  • Is the number of stories being displayed consistent with Facebook’s claims?

Let’s find out…

The Breakdown of Stories

Distribution of Filtered Facebook News Feed Stories by Source

I actually received quite a few more stories than the 300 Facebook says to expect — 373 in all. Here is how those posts are broken down:

  • 239 Stories from Friends (64.1%)
  • 106 Organic Stories from Pages (28.4%)
  • 25 Ads (6.7%)
  • 3 Stories from Lists (0.8%)

I don’t know about you, but this seems like a fair breakdown. Not too many ads. Organic stories from brand Pages aren’t being drowned out. Lots of stories from friends.

The Breakdown of Page Stories

Facebook Organic Page Post Distribution News Feed

Now, here is where it gets interesting…

Those 106 Page stories came from a grand total of 38 unique brand Pages, for an average of 2.8 per Page. Most marketers seem to assume that only some of our posts are shown. But really, based on what I’m seeing, it appears I see almost everything that I care about most.

When I looked through that list of 38 Pages, these were definitely all brands I care about most (or have interacted with recently). And what’s funny is that I rarely — or almost never — comment on brand posts. But Facebook “somehow” knows what I like.

Take a look at what I saw from the brands represented the most:

  • Mashable – 16
  • TechCrunch – 15
  • The Onion – 10
  • Green Bay Packers – 7
  • Post Planner – 5
  • – 4
  • NFL – 4

I almost never comment on content shared by these brands (Post Planner being the one exception). But I do read comments. And I do click on links. This was enough to tell Facebook that I would like to see a combined 31 link shares from Mashable and TechCrunch covering a 24 hour period.

Answering the Questions

Okay, so now let’s get back to some of the questions I was hoping to answer…

Q: Are organic posts from Pages being squeezed out?

It sure doesn’t seem that way. More than a quarter of the stories I saw in my News Feed covering a 24-hour period were organically from brands. I certainly don’t want 50% or more of the stories in my News Feeds to be commercial.

Q: Are ads taking over the News Feed?

Not at all. Only 6.7% of the stories I saw were ads.

Q: Are the posts I’m seeing from brands what I actually want to see?

Absolutely. I saw 31 posts total from Mashable and TechCrunch. As I look through the list of brands represented, there is a good reason for all of them to be there.

Q: Is the number of stories being displayed consistent with Facebook’s claims?

Yes. In fact, I saw 24% more stories than Facebook said I should expect.

Closing Thoughts

First of all, it’s quite clear to me that Facebook is showing me the content I want to see, particularly from brands. Or to put it more accurately, what I end up seeing is engaging content — I don’t know what I couldn’t see. Looking at that list, I have no complaints or arguments.

Second, what I see here conflicts with the biggest complaints from brands in a couple of ways…

One claim is that brands can’t enter the News Feed without paying for ads. This is clearly not the case since 28.4% of the stories I documented in this study were organic posts from brands.

Another claim is that Facebook is preventing brands from reaching people who actually want to see their content. Well, that’s certainly not the case for me when it comes to Mashable, TechCrunch and others. I’ve shown Facebook through my actions that I like their content, and as a result I see a ton of it.

This doesn’t mean that brands don’t have an argument. I understand the feeling of being shortchanged when you pay for ads to increase Likes but then can’t reach those users in the News Feed.

But you know what? That falls on us. The 38 brands that reached me during this 24 hour period have provided compelling content that inspires me to interact with them in some way. As a result, I see their content regularly.

Sure, filtering content certainly does fatten Facebook’s pocketbooks by making it more necessary to advertise. But — separating my roles as a user from that of a brand for a moment — I fully believe that it also improves my experience consuming content on Facebook.

The filtering isn’t perfect. Maybe Facebook should expand the number of stories highlighted in News Feed per day to 500. Maybe users should be given an unfiltered option.

But what I’m not seeing here is filtering done for the sole purpose of screwing brands, at the expense of user experience. If you want to reach more Fans organically, you need to compete with the mounds of content that users could see every day. To be “preferred,” you need to do one of two things: 1) Be awesome or 2) Pay to reach them.

Or you could do both and get even greater results!

What do you think? Are you seeing what you want to see in your News Feed? Let me know your thoughts in the comments below!