I saw your last DM 👀

GM! Paperboy here 🗞️

Last week I audited an 8-figure Shopify brand with 0% incrementality on their top 3 Direct Mail flows.

Their SaaS rep kept telling them they were seeing a 45x ROI. All I saw was $10k a month getting lit on fire.

Most brands I audit have the same problem. So let’s fix it.

As a primer, there are a few types of direct mail:

• Campaigns targeted to your list - like a standard email campaign, just in the mail

• Campaigns to a prospecting list - like a cold email campaign, just in the mail

• Flows targeted to your list - like an email flow… just in the mail

You get the gist.

How I uncovered the ROI mismatch

I did a very simple holdout test analysis using order data.

Here’s a step-by-step:

Step 1: Setup a proper holdout audience on EVERY campaign/flow.

Some apps have it out of the box. If not, set up a random sample in your flow, or extract one manually from the list you upload.

Step 2: Let it run until you have at least 10k sends or ~3 months have passed. If you're not getting enough data in 3 months, you might be too early to the channel.

Step 3: Collect the data using real store data. Do not, I repeat, DO NOT trust the SaaS platform's holdout data. Your incentives are not aligned.

Export the following:

• A spreadsheet of all orders starting ~2 weeks out from the start of your DM campaign

• A spreadsheet of your audience with emails (and/or phone), cost per mailer, and a column flagging "Holdout" or "Delivered"

Step 4: Run an analysis

If you know pivot tables: pull a sumif revenue column into your audience spreadsheet from the orders sheet, add a cleaned-up date column and a count column (makes calculated fields easier). Run a pivot filtered by holdout vs delivered, with a calculated formula: (Revenue - Cost) / Count.

If you don't know pivot tables: paste what I wrote above into Claude and pray it does it properly. Or learn pivot tables lol.

You'll end up with a clear $ per send and can run ROI calcs on whether it was worth it.

Caveats: You might notice a few weird things in your data:

• Recipients getting multiple mailers

• Mailers failing to send at a high rate

• Address verification failing often

I strongly suggest doing this pull and analysis manually the first few times (or every so often) before outsourcing to AI. You might catch structural issues holding you back.

My thoughts on different types of DM

Flows

I ran quite a few flow holdout tests over the last few months for various agency clients.

Many of our flows were not driving incremental revenue, and either needed a rethink or a complete shut off.

DM SaaS account managers will typically tell you to set up abandon cart, browse abandon, and abandon checkout flows because they're low-hanging fruit. I think this is mostly wrong and here's why:

• A mailer takes 3-7 business days to arrive from production

• In a flow, the mailer is usually the "last resort" so it might not fire until day 5-10

• Your customer gets the mailer 8-20+ days after they were originally shopping

8-20+ days isn't very low-hanging fruit. To me, that’s a dead lead (unless you're in a long purchase consideration biz).

And for every dead lead, you're sending a mailer. That's a ton of inefficient spend you could be optimizing.

Campaigns to List

Mixed experience here. It's very brand dependent, so I'll share the highest ROI campaigns I've seen in order:

• Seasonal sales tied only to DM that run for a full month (less timing issues)

• Seasonal catalogs with 10 to 30 page mailers (sign up to this brand and you’ll see what I mean)

• Major drop or sale early access. Note: requires very dialled in ops

Anything not tied to a release or sale is hard to drive ROI on, since the medium is 100x more expensive than firing off an email or SMS.

Campaigns to cold prospects

I don't have enough data here to share what works and what doesn't.

I see it as independent from your flows and campaigns, and a unique top-of-funnel channel. For most brands, this should be the paid media and growth team's job to oversee, not the retention marketers.

If your boss is asking you to manage it, I'd try to replicate the best offers from your ads and landing pages.

Important testing considerations

Let's say you run that holdout test and find it isn't incremental. Do you scrap the program?

One reason I don't tell new brands or brands under $10m/year to play with this channel is that it requires far more work (and data) than people realize.

The holdout proves the existing mailer isn't working, not that the strategy doesn't work.

Like email, SMS, or popups, you want to test lots of variables:

• Offers

• Timing of offers

• Design

• Copy

• Segmentation

It's a separate channel with its own nuances.

So does direct mail work?

My take: most brands overspend on flows, don't run holdout tests, and send under-optimized mailers. They also send too few promos to high-value winback audiences and too many to dead leads.

Does it work? Of course. Are most brands doing it wrong? Yes.

Hope this email helps change that.

How to design standout mailers

Most mail people get is meal kits, real estate listings, or grocery flyers. People auto-pilot and default discard 99% of it. But mail still gets a 100% open rate, because people sift through to find anything important.

Most companies send uninspiring creative that gets skipped right by. The ones that stand out get a few extra seconds, and usually still get tossed for lack of relevance.

As a brand mailing your owned audience, you have a leg up. They know you, and you know why they shopped your site.

Use what you know about the customer to make relevant creative. Use what you know about copy and design to stand out next to the slop. Use what you know about offers to make it irresistible.

Do this right and you unlock better ROI on the channel and more credibility with your customer.

Meme of the Week

That’s all for now,
Paperboy 🗞

p.s. Did you like the newsletter? Hate it? Reply and tell me why! Or just tell me what you’d like to hear about next. I read and reply to every message.

Need help with retention?

30 or 60-minute consulting session: mentorpass.co/filippejic

Work with my Agency: bedfordmarketing.co