2 Ways to Create A/B Split Tests of Organic Content on Facebook

A/B split tests are helpful for uncovering the most effective posts or ads in a scientific way. Did you know that you can do this with organic content?

You’re probably familiar with A/B split tests for Facebook ads. This process allows you to test variables like targeting, optimization, and creative in a scientific way, showing variations to random groups without overlap.

What’s great is a similar process is available for organic posts that you publish to your page. Actually, there are two different available ways to do this, one through Meta Business Suite and one through Creator Studio.

It’s entirely possible that this isn’t fully rolled out yet. In fact, I’ve seen some bugs in one version, and it also seems that what you see may be different from person to person.

With all of that said, I want to walk through the two ways that I can create A/B split tests of organic content…

Meta Business Suite

From the “Posts & Stories” section of Meta Business Suite, create a post.

A/B Test

Click the “A/B Test” icon to start a test. It will look like this…

A/B Test

You can include up to four versions of a post. All you do is create each version by providing the copy and either an image or video. For links, either provide the URL in the copy or paste it into the link preview section.

When you’re done, you can preview all versions of your post before publishing…

A/B Test

You can even schedule the post if you’d like.

A/B Test

The test will run for 30 minutes. A different segment of your page followers will see each version. Once the test is complete, the winning version will be published to your Facebook page. The other version(s) may still exist in news feeds as people engage, comment, and share.

Creator Studio

The approach using Meta Business Suite is extremely simple. But, maybe you want to add a little bit of complexity? You can do that with Creator Studio.

For whatever reason, some people have easy access to Creator Studio while others don’t. From my page and Business Suite, I no longer get links to the tool.

You should be able to access Creator Studio by clicking this link.

From there, go to Post Testing under “Tools.”

A/B Test

Click “Start a Test” and it will look like this…

A/B Test

This is a lot different from the Business Suite version, right?

The first step will be to select a Content Type.

A/B Test

Just know that video posts can only be compared against other video posts. You can mix and match between image, link, and text posts.

If you do choose to test video posts, you’ll need to click the “Edit” button at the far right after uploading the file.

A/B Test

This will give you access to all of the various customizations that are unique to videos.

A/B Test

When you’re done, it will look something like this…

A/B Test

As you can see, you can publish or schedule the test. Once you click to schedule or publish, you’ll get some more settings…

A/B Test

This is one of the big differences between A/B tests from Meta Business Suite and Creator Studio (at least for me — see the upcoming section about bugs). You can select from various “key metrics.”

A/B Test

This way, you determine the metric Meta uses to determine the winner. Choose from Comments, Shares, Reactions, People Reached, and Link Clicks.

You can also customize how long you want the test to run.

A/B Test

Choose from 10 minutes, 30 minutes, 1 hour, 3 hours, and 24 hours. Once the test is complete, the winning post will be published to your page.

A Note on Bugs

It’s entirely possible there’s something I’m not seeing when creating a test from Meta Business Suite. When I choose to publish or schedule a test, I get the following…

A/B Test

That “moment” takes many moments. Actually, I’ve never found an end to it. Originally, I thought it just took a while to publish each individual version. Then I thought maybe it wouldn’t “complete” until the test was over. But I waited that out, and this message kept showing.

I’m not sure if another important screen is supposed to appear next. It’s possible that additional customizations that happen in Creator Studio are intended to appear here. No idea. I do know that the test still ran even though this page didn’t finish loading.

It still does appear that there are significant differences between Business Suite and Creator Studio A/B split tests. But, I can’t say with 100% certainty that all differences that I’m seeing are intentional.

Viewing Results

Once again, I believe there’s a situation of what I see and what others see as being different.

If I create an A/B test in Meta Business Suite, there is nothing (in that section, at least) that shows me the test results. I can view the individual post versions and metrics associated with them, but that’s not unique. It’s the same as any other post.

Creator Studio, though, shows clear test results. These are found within the same Post Testing section under Tools.

A/B Test

And guess what? These results include all A/B tests of organic posts, including those that originated from Meta Business Suite.

Click through to see a side-by-side comparison.

A/B Test

First, I can’t ignore that the “Reached” stats are clearly wrong for any image post. There’s no way that a post that reached one person has seven reactions.

Second, this is part of the potential problem of these short tests. This shows the results for a very short period of time when the test ran. Is that enough time to choose a winner? It’s one reason to consider going with the 3-hour or 24-hour options in Creator Studio.

You can also choose to track the results of each variant after the test is complete. Of course, the winning post will see the biggest benefit of viewing results this way.

A/B Test

If you don’t like the post that Facebook chose, you can override the selection and publish a different one to your page. That is done at the top.

A/B Test

Is This Useful?

I have to admit that I avoided experimenting with A/B split tests of organic posts for quite some time. It sounded great in theory, but I was also negatively influenced by how A/B tests work for ads and assumed this would be pretty much the same.

A/B tests for ads take anywhere from a few days to four weeks. I have no patience for doing this with every organic post. There’s also a matter of the overall usefulness of A/B tests for ads since they are best suited for learning something over a few weeks that you can then apply in the future.

But, this is way different in the best ways. The test is super easy. The test will be completed in anywhere from 10 minutes to 24 hours. The winning post is published to your page.

Also, this is really the only way you can have Facebook optimize for an action when publishing an organic post. If your goal is link clicks, you can have Facebook choose the version that results in the most link clicks. That’s great!

I’ve only started using this feature so far, but early returns are solid. I’ve run two tests, and one resulted in a winning post that has reached over 17,000 people with close to 2,000 engagements.

Of course, don’t expect this to be miraculous either. If you submit four post versions that all stink, expect to get bad results. This at least allows you more opportunities to find something that hits — and also learn from what works and what doesn’t.

An Ads Tie-In

While this is for organic posts, A/B testing can absolutely benefit our advertising.

One of the negatives of A/B testing of ad creative is that you can spend a lot of money simply running a test to see which ad is most effective. But, you could move that effort to organic testing. Find the post that gets the most link clicks (or whatever you want), then put money behind that winning post.

It would seem that this could make some of the guessing, experimentation, and budget a bit more efficient, at least in theory.

Your Turn

Have you experimented with A/B tests of organic posts? What results are you seeing?

Let me know in the comments below!