Why We Don't Do Phishing Simulations
A 19,500-person trial found phishing simulations cut click rates by about 2%. Here's what that means, and why we built a training platform without them.
It's the first question we get on almost every sales call. Do you do phishing simulations?
No. Deliberately. Here's the reasoning, including the parts that are inconvenient for us.
What the largest study on this actually found
The study is Understanding the Efficacy of Phishing Training in Practice, by Grant Ho, Ariana Mirian and colleagues at UC San Diego and the University of Chicago, published at the 2025 IEEE Symposium on Security and Privacy. It's an eight-month randomized controlled trial run at UC San Diego Health, covering more than 19,500 employees across ten simulated phishing campaigns — the largest published trial of anti-phishing training inside a real working organization.
Two findings.
Embedded phishing simulations — the click-and-get-trained model — reduced the likelihood of clicking a phishing link by roughly two percent. Across twenty test groups, the best result was a 1.7% lower failure rate for trained users versus untrained.
Annual mandatory awareness training showed no measurable relationship to phishing failure at all. Employees who had completed training within the last thirty days performed no better than employees more than a year overdue.
The researchers' own conclusion was blunt: organizations should not expect training as commonly deployed today to substantially protect against phishing, because the magnitude of protection is simply too small.
That second finding is not comfortable for a company that sells training. We're including it because leaving it out would make this post dishonest, and because it points directly at what we think training is actually for.
The honest read
A two percent improvement isn't zero. On a large enough population it prevents real incidents, and the study did find one meaningful bright spot: employees who fully completed training were 19% less susceptible. The problem was that hardly anyone completed it.
So the finding isn't quite "training does nothing." It's closer to: passive training does nothing, completed training does something, and getting people to actually complete it is the hard part.
That reframes the design problem. If completion is the variable that carries the benefit, then the useful thing to optimize is completion — not click rate on a test.
What simulations are and aren't good for
Fair is fair. Simulations do two things well.
They give security teams a population-level signal about which departments and which lure types are landing. And they generate a teachable moment that lands harder than a scheduled module, because the person just made the mistake.
That's real value if you have a security team to act on it. Most of the companies we work with have an office manager, a controller, or an MSP tech who owns security among six other jobs. For them, a monthly click-rate dashboard is a number nobody has time to act on.
And there are costs that don't show up in the vendor demo.
Simulations damage internal trust. The mechanism of a phishing test is that your employer lies to you, convincingly, to see if you fall for it. Do it with a fake bonus announcement or a fake layoff notice and the fallout is a real HR problem — companies have made the news for exactly this. The employees most rattled by it tend to be the ones already worried about job security, which is a rough way to spend goodwill.
Simulations train people to distrust internal email specifically. Reporting rates go up. So does the volume of legitimate internal messages flagged as suspicious, which lands on the same overloaded person.
Simulations produce a compliance artifact nobody actually asked for. A click-rate report is not what an assessor wants. More on that in a minute.
None of this means simulations are worthless. It means the value is concentrated in organizations with a security function to consume the output — and that's not who we serve.
What we build instead
ClickCerts is built around a different claim: training's dependable output is documented completion, not behavior change. Don't get us wrong, our training is great, but **what makes it awesome is that it's designed to build repeatable habits that make secure choices easier, especially under pressure. **
Quarterly courses on a 90-day window. Real content, real quiz, and at the end a dated certificate with a verifiable serial for each employee. An exportable completion record showing who finished, when, and what the course covered.
That's the product. It's narrow on purpose.
We're not going to tell you it will drop your click rate, because the best available evidence says we couldn't back that up. What we will tell you is that when your carrier asks whether all employees receive security awareness training, or an assessor asks you to evidence it for the assessment period, or a prime contractor's flow-down requires it — you can answer in about thirty seconds with a document, instead of two weeks of reconstruction.
If your problem is proving training happened, that's what we're for. If your problem is stopping the click, there's plenty that helps — and some of it you may already be paying for.
Worth checking your Microsoft bill before you shop, in particular. Attack simulation training is built into Microsoft 365. It runs from the Defender portal, it uses real payload templates, and it assigns follow-up training automatically to anyone who clicks. The licensing detail matters: it requires Defender for Office 365 Plan 2, which is included in Microsoft 365 E5 and attaches to Business Premium as an add-on — Business Premium on its own ships with Plan 1, which doesn't include it. There's also a 90-day Plan 2 trial if you just want to see what your click rate looks like before committing to anything.
For a lot of small companies that's the better path. Configuration time instead of another subscription, and the results land in the same portal your admin is already in. We'd rather send you there than sell you a second tool that does a job Microsoft already does.
Where the leverage actually is
The same research that found training barely moves click rates also pointed at the alternative: layered technical controls that keep the decision away from the employee in the first place.
Phishing-resistant MFA is the big one. If credentials alone can't produce a session, a harvested password is a much smaller event. Passkeys and FIDO2 keys beat app-based codes, which beat SMS.
Then: mail filtering with external-sender banners and link rewriting. Conditional access that blocks legacy authentication. A one-click report button that routes to someone who will actually look. And for the highest-value action — money movement — a hard out-of-band verification rule, because that's the attack that costs six figures.
Those controls don't require anyone to be perceptive on a Tuesday afternoon.
So why train at all?
Two reasons, and we'd rather state them plainly than dress them up.
Because it's required. NIST 800-171 and CMMC require awareness and role-based training. HIPAA requires it. Cyber insurance applications ask about it. Prime contractors flow it down. The requirement exists whether or not any given study likes it, and unmet requirements have consequences that are easier to predict than breaches. [Just because CMMC is paused, the training requirement has not] (https://www.clickcerts.com/blog/cmmc-phase-2-suspension-what-still-applies).
Because reasoning transfers better than pattern-matching. Teaching someone to spot a suspicious email has a short shelf life — AI-written lures ended the typo-and-bad-grammar era. Teaching someone that any change to where money goes gets verified by phone, on a number you already had is a rule that survives contact with a message that looks perfectly legitimate. That's what our content aims at. Building repeatable habits that make secure choices easier, especially under pressure.
The short version
We don't do phishing simulations because the evidence for their effect on click rates is thin, the internal-trust cost is real, and the artifact they produce isn't the artifact anyone asks you for.
We do documented, certificate-backed quarterly training because that's the part of this that reliably works — and because when someone asks you to prove your people were trained, "we run simulations" isn't an answer.
If a vendor tells you their simulation program will make your people un-phishable, ask what study they're citing. Then read the one above.
You may be interested in this related post: Watch for Typos and Bad Grammar. (And Other Phishing Advice That's Now Wrong.)
