- An agency that reports opens is reporting the wrong number. Clicks and revenue per person are what track sales.
- If nobody at your agency can explain why each flow is built the way it is, the flows probably have no intention behind them.
- Klaviyo's default attribution counts opens as well as clicks, so "email revenue" in a deck can look better than what actually happened in your store.
- Before you fire anyone, check the account yourself. You'll either find the problem or find out the agency's doing fine.
How do you know if your email agency is not working?
You feel it before you can prove it. Revenue's flat, the reports look busy, and you can't say what changed last month.
Here are the seven signs I'd check. For each one: what it looks like, why it matters, and what to do about it.
Sign 1: They report open rates
What it looks like: The monthly report leads with open rate. Wins are "opens up 8%."
Why it matters: Apple Mail inflates opens. In pooled data from 4 consumable brands, click rate tracked revenue at 0.65 and open rate at 0.32. For one brand, open rate tracked revenue negatively: more opens, less money. Apple Mail auto-opens pushed open rates past 85% on some segments.
What to do: Ask for clicks and revenue per person on every send and every flow email. If they push back, that's an answer. More in open rate vs click rate.
Sign 2: Every campaign goes to everyone
What it looks like: Open your last ten campaigns. If every one went to your whole list, there are no segments.
Why it matters: You earn less per person, and nobody can tell what worked. A fitness tech brand I work with earned $0.19 per person on an engaged-segment send to 7,145 people, and $0.04 per person on a full-list blast to 49,475.
What to do: Ask what segments they'd build and why. A good agency should have an answer in one sentence. See email segmentation for ecommerce.
Sign 3: The reports have no analysis
What it looks like: A list of what was sent and what it made. No "here's why" and no "here's what we're changing."
Why it matters: A report that only describes the past isn't doing the job. You're paying for judgment: what worked, what didn't, and what happens next.
What to do: Ask for three things each month. What worked, what didn't, and what they're changing because of it. If the answers are vague two months in a row, that's the sign.
Sign 4: You're chasing them for replies
What it looks like: You send a question and wait days. You end up following up on your own requests.
Why it matters: Email is time-sensitive. A restock, a launch or a broken link can't wait a week. When something sells out, the restock email is your biggest send of the month. If your agency is slow, that's the send that slips.
What to do: Agree on a response time in writing. Then see if they keep it.
Sign 5: You've been handed off
What it looks like: The person who sold you is gone after the kickoff call. Now it's someone you've never met, and maybe they've changed twice.
Why it matters: Every handoff loses context. The new person doesn't know why your flows were built the way they were, so they either leave them alone or start over.
What to do: Ask who is actually in your account each week, by name. For what it's worth, my clients work with me directly. No hand-offs to someone they've never met.
Sign 6: Nobody can explain your flows
What it looks like: Ask "why does the welcome flow have five emails?" or "why does the winback stop after one?" and you get a shrug or a template answer.
Why it matters: When I take over from cheap setups, the most common problem is flows with no intention behind them. They exist because a template said so. The data says intention matters. In my 4-brand data, welcome email #1 earned about $4.50 per person, while emails #2 to #5 earned about $0.10 each. Winback email #3 earned about 2x email #1. If nobody knows that, the flows aren't being built around what earns.
What to do: Pick two flows and ask what each email is for. Every email should have a job you can say in a sentence.
Sign 7: The numbers only look good in their deck
What it looks like: Email revenue in the report keeps climbing. Your Shopify revenue doesn't.
Why it matters: Klaviyo's default attribution is last touch, with a 5-day window for email clicks and a 5-day window for email opens. Since Apple Mail inflates opens, some of that "email revenue" may be sales that would've happened anyway. Klaviyo does let you remove Apple Privacy opens and bot clicks from attribution, but someone has to turn that on.
Broken tracking cuts the other way. Across the brands I work with, about 20 sends showed opens and revenue but zero clicks. That's broken tracking, not a great email.
What to do: Ask which attribution settings their numbers use. Compare email-attributed revenue to your total store revenue month over month. Across the brands I work with, email averages about 32% of total revenue. If yours is far off in either direction, find out why. Here's what percentage of revenue should come from email, and more on how Klaviyo attribution works.
Should you fix it or switch agencies?
Not every sign means fire them. Some are fixable with one honest conversation.
Here's how I'd think about it:
- One or two signs: Raise them directly. Give it a month with clear asks.
- Three or more: Get an outside look before you decide. You want to know what's actually broken, not just how you feel.
- Sign 6 or 7 on its own: Take it seriously. If they can't explain the flows or the numbers don't hold up, the problem is deeper than reporting.
If you've decided to move, read how to switch email marketing agencies first. There are things to grab from your account before anyone leaves.
And if you're not sure what you should be paying for what you get, what a Klaviyo agency costs lays out the options. If you're under about $25K a month in revenue, you may not need an agency at all. A specialist is usually the better fit.
How do you check your agency's work yourself?
You can do most of this in an afternoon:
- Open your last ten campaigns and check who they went to.
- Open every flow and check every email is live, not stuck in draft.
- Look for sends with opens and revenue but zero clicks.
- Check your attribution settings.
- Compare email revenue to total store revenue for the last six months.
My Klaviyo audit checklist has the full list. If you'd rather have someone go through it with you, that's what a free Klaviyo audit should be for, as long as it's a real one and not a pitch.
Where these numbers come from
The open versus click correlations and per-person flow numbers are pooled from 4 consumable brands I work with: 2,897 campaigns and $17.7M in email revenue. It's a small sample, so treat it as directional. The 32% figure is the average across the brands I work with.
FAQ
How long should I give a new email agency before judging results?
I work in 90-day blocks. A lot of brands notice a lift in the first 30 days, and most see it within 90. That's a pattern, not a promise. But you should see clear work, clear reasoning and fast replies from week one.
What should an email agency report every month?
Clicks and revenue per person for campaigns and flows, what share of store revenue came from email, what worked, what didn't, and what they're changing because of it. Opens can be in there, but not as the headline.
Is it normal for an agency to hand me off to someone else?
It's common. Whether it's fine depends on whether the new person knows your account. Ask who does the work before you sign, by name.
Can a bad agency make my email revenue worse?
It can. Broken flows, a missing reorder reminder, or campaigns sent to everyone all cost you money every day they run. That's why it's worth checking instead of guessing.
What should I ask my agency before I leave?
Ask for access to everything they built in your account and confirm you own it. Then read switching email marketing agencies for the full list.
Founder of Bardown Digital, a Chicago-based Klaviyo email and retention agency for Shopify brands. He's worked with brands doing $500K to $100M a year, with $100M+ in email-attributed revenue across them.