To A/B test social media across many brands, pool brands that share an audience type, pair them, run the same variable in the same window, and judge each post against its own account's median. That turns six small tests that prove nothing into one test that does. The rest of this post is the operating detail.
TLDR: One account rarely posts enough to read a test. Group similar brands into a cell, split variants evenly across them, normalize results to each account's own median, and publish one verdict per test.
Why single-brand tests stall
Most brand accounts publish three to five times a week. A fair test wants 8 to 10 posts per variant, so one account needs a month or more per test, and by then the season has changed.
Noise is the other problem. In Mydrop's data, the middle half of Instagram posts ran from 1.1% to 6.7% engagement. A single account's two-post comparison will almost never clear that.
Pooling fixes both: five brands posting three times a week give you 15 posts a week to split.
Design a test cell
A cell is a group of brands that can fairly be compared.
| Decision | Rule |
|---|---|
| Who joins a cell | Brands with similar audience type and platform mix |
| Variants | Two, maybe three, never more |
| Assignment | Alternate variants per brand, so each brand runs both |
| Window | Same dates for every brand in the cell |
| Held constant | Format, slot, hashtag rule and link rule |
| Measure | Post result divided by that account's median |
Dividing by each account's own median is what makes pooling honest. A small account at 7% and a large one at 1% can sit in the same test, because both are measured against their own normal. Mydrop's own cross-account numbers use the same idea.
Run regions and languages as separate cells
Do not mix a Madrid account and a Chicago account in one cell just to hit a sample size. Language, culture and time zone change what works. Run a cell per region, then compare verdicts, not raw numbers. For tests across 20 or more markets, see automating localized A/B tests.
Put every test on one shared calendar
At scale, the main failure is two teams testing different things on the same account at once. Keep a single test calendar that lists each cell, variable and window, and give each test an owner who can say no to a collision.
In Mydrop, a multi-brand calendar lets you see every brand's scheduled posts in one view and label variant posts, and Workspaces keep each brand's data and permissions separate while the test owner sees across them. Per-post results sit in analytics, and a multi-profile benchmark view helps you compare each account to its own baseline.
What not to test at scale
Do not spend a cell on levers that barely move results. In Mydrop's September data, hashtag count moved median engagement by at most 5%, and weekday sat within 2% of the median. Cells are expensive, so test format, opening line, offer and call to action.
Report one verdict per test
Brand leads do not want twelve charts. Give each test a one-line verdict:
- Winner and the size of the lift against median.
- How many posts and brands it rests on.
- What changes in the playbook as a result.
Store the verdict in a shared library so a new brand can start from what already worked. See a reusable experiment library.
Three problems you will hit in the first cell
- One brand posts less than the others. Weight by posts, not by brand, or cap each brand's contribution so the busiest account does not decide the result.
- A brand has a special event mid-test. Mark those posts and exclude them from the verdict.
- Leads push to stop early. Agree the end date up front and let the window finish.
Small, boring rules like these keep a pooled test honest.
Set up your first cell this month
Choose three brands with similar audiences, one variable, and a five-week window. Schedule the posts in one calendar and write the verdict rule before the first post goes out. To set up several brands in one workspace, start free on Mydrop.
Google review
Trustpilot review
Product Hunt comment
