I was on a monthly reporting call with a client's agency last month, watching them click through a 34-slide deck of beautiful charts. Keyword rankings up. Backlinks acquired. Blog posts published. Traffic trending green. The CMO was nodding along, reassured by the color palette.
I waited until they reached the "Questions?" slide. Then I asked: "What's our citation share in Perplexity for our top ten buyer prompts?"
Silence. The account director smiled the way you smile when you don't know what language someone is speaking. "Could you... clarify what you mean by citation share?"
I clarified. Then I asked four more questions. By the end, the account director was taking notes with the frantic energy of a student who just realized the exam was on a different textbook.
That call cost them the retainer. Not because they were bad people. Because they were optimizing for 2022 in 2026, and the gap between those years is now measured in lost revenue.
Stop Asking the Wrong Questions
If you're a Head of Marketing still running the same monthly script, you're not auditing your agency. You're subsidizing their nostalgia.
Here are the four questions that need to die:
- "What did our keyword rankings do this month?"
- "How many backlinks did we earn?"
- "How many blog posts did we publish?"
- "Where is our organic traffic trending?"
In 2022, these were the right questions. In 2026, they're mathematically uncoupled from pipeline. Keyword rankings can hold steady while AI Overviews steal every click. Backlinks can accumulate while LLMs ignore them because they're from low-trust directories. Blog posts can pile up like cordwood while your core pages remain structurally invisible to the models that actually matter.
If your agency confidently answers these four questions—if they lead with them—it proves they haven't adapted. They're defending legacy metrics they know how to manipulate, hoping you don't notice the pipeline is still flat.
The Five Questions That Actually Matter
I now run this exact interrogation every time I evaluate a marketing partner. If they can't answer all five with specific data, we're not working with them. Period.
Question One: "What's our Citation Share?"
The exact phrasing: "What is our Citation Share across ChatGPT, Perplexity, Gemini, and AI Overviews for our top 20 buyer prompts, and what's the month-over-month trend?"
If they can't give you a hard percentage—we're cited in 23% of prompts, up from 15% last quarter—they aren't measuring AI visibility. They're measuring 2010s visibility. The work might look pretty on a spreadsheet, but it's disconnected from the interface where modern buyers actually research.
I asked this to one agency last year. They showed me a "brand awareness" chart instead. I showed them the door.
Question Two: "Which Pages Are Actually Being Cited?"
The exact phrasing: "Which of our existing pages are being cited by LLMs, and what's our Extraction Rate on the top 10 highest-leverage pages?"
This tests whether they've audited your historical content for AI extractability. Most agencies are still writing new top-of-funnel blog posts while your comparison pages, pricing docs, and case studies sit there—structurally invisible because they weren't formatted for machine parsing.
If they haven't mapped which pages the models actually pull from, they're spending your money on noise while your signal rusts.
Question Three: "Where Is Our Entity Broken?"
The exact phrasing: "Where is our brand's entity described inconsistently across third-party sources—G2, Reddit, Wikipedia, PR—and what's the engineering plan to fix it?"
LLMs don't trust what you say about yourself. They trust consensus. If your website says you're an "AI infrastructure consultancy," your LinkedIn says you're a "digital transformation agency," and your G2 profile says you're a "web development firm," the models get confused. Confusion equals invisibility.
An agency that hasn't mapped your third-party footprint is ignoring the foundational layer of entity trust. They're building a house without checking the ground.
Question Four: "What's Our Cannibalization Rate?"
The exact phrasing: "What is our AI Overview cannibalization rate on our top 50 revenue-driving keywords, and which pages are we restructuring to survive it?"
This is the ultimate diagnostic. If your organic traffic has plummeted but your keyword rankings look fine, AI Overviews are eating your lunch. The model is answering the question directly in the search interface, and users never click through.
An agency without a precise cannibalization metric is reporting on the wrong layer of reality. They're telling you the weather while you're drowning.
Question Five: "Who's Beating Us and How?"
The exact phrasing: "Which specific competitor is gaining AI citation share fastest in our category, and what is the mechanical playbook they're using that we're not?"
Citation share is zero-sum. If your agency is only reporting your internal numbers—"we published 12 posts!"—without mapping competitive velocity, you're flying blind. I want to know that Competitor X just got cited in 40% of prompts because they deployed FAQ schema on their comparison pages, or because they landed a Tier-1 mention that we're missing.
If they don't know who's winning and why, they don't know the game they're playing.
How to Read Their Reaction
Watch closely when you drop these five questions. The response falls into three categories:
The Elite Response: They answer all five with specific data, extraction rates, and a clear architectural plan. They might even push back on your assumptions with better ones. If you find this agency, pay them more and get out of their way.
The Danger Zone: They can answer one or two confidently, then deflect the rest by pivoting back to "traffic growth" or "brand awareness." They're stuck mid-transition—halfway between old SEO and new Citation Engineering. Have a very serious conversation about their next 90 days. Give them one quarter to catch up, then cut them if they don't.
The Dead Weight: They can't answer any of the five. They'll try to reassure you with jargon, or they'll promise to "look into it." The size of the gap between what you're paying and what they're delivering is the exact amount of money you'll save by firing them today.
The Hard Truth
The questions you ask shape the work you get. If you ask about keywords, you'll get keyword reports. If you ask about citations, you'll get Citation Engineering.
Most agencies aren't evil. They're just optimized for the metrics you incentivize. Change the questions, and you force them to either evolve or expose themselves.
That account director from the 34-slide deck? I heard he got promoted three months later—at a different agency, still running the same playbook. The CMO who asked the hard questions is now running marketing at a company that's actually growing.
The difference wasn't talent. It was the interrogation.
— James, Mercury Technology Solutions, Hong Kong, May 2026
