The detection study
What was done to Yfraud's rules, in order, with the measurements. The two evidence reviews it rests on are Email phishing signals and Web address signals. Everything here is rules only: no model, no network, no reputation feed.
1. The honest set was too small, and it was hiding a 20% false positive rate
The Oct 7 corpus had honest email only as bodies without headers (the Hugging Face set and CLAIR). Adding the SpamAssassin public corpus (3,899 easy ham, 248 hard ham, 1,804 spam, 2002 to 2005, raw mail with headers and HTML) showed the email rules warning on 20.4% of ordinary mailing list mail and 29.4% of brand newsletters. The cause was one rule: reply-to-mismatch fired on every mailing list, because lists set Reply-To to the list address. Three more rules (display name claim, brand link mismatch, university claim) fired on newsletters that merely mentioned Microsoft, the FBI or a professor. None of this was visible before, because none of the earlier honest sets had headers. The first result of the study is therefore a correction, not an improvement:
| Honest set | Before | After |
|---|---|---|
| SpamAssassin easy ham (3,899 list mails) | 20.4% warned | 0.5% |
| SpamAssassin hard ham (248 brand newsletters) | 29.4% warned, 4.4% blocked | 10.9% warned, 0.4% blocked |
| Hugging Face honest bodies (11,124) | 1.0% | 1.1% |
| CLAIR honest letters (5,066) | 0.2% | 0.1% |
| PhiUSIIL legitimate URLs (134,850) | 5 warned | 3 warned |
| UCI honest texts (4,827) | 0 | 0 |
What fixed it: a list signal (List-Id, List-Post, List-Unsubscribe, Precedence list or bulk, a Sender of the -bounces@ kind; in Gmail and Outlook, an Unsubscribe control or the word in the footer) that switches off Reply-To judgement and the soft rules; Reply-To now counts only when the reply goes to free webmail from a company address (the BEC shape, Agari 2020: free webmail in 72% of BEC), or the mail claims a brand or a department, or the reply domain is a lookalike. A brand claim now comes from the display name, or a subject written as that brand's notice ("Your PayPal account", "eBay Unpaid Item Strike"), never from a mention. The 419 bait, romance persona, gift card and prize rules each lost the honest readings the newsletters exposed ("won't", the Nobel Prize, a sweepstakes reminder, an Apple Store mail selling iTunes cards, "baby" in a news story).
2. What raised the catch rate
| Scam set | Before | After | Held out 30% |
|---|---|---|---|
| Nazario phishing emails (3,073, 2005 to 2025) | 42.0% caught, 9.6% danger | 80.0% caught, 36.5% danger | 78.8% |
| CLAIR 419 letters (3,911) | 36.3% | 59.7% | 59.0% |
| PhiUSIIL phishing URLs (100,945), address and title only | 1.3% | 13.6% | 13.8% |
| Hugging Face spam grade phishing bodies (6,974) | 6.0% | 7.6% | |
| SpamAssassin spam (1,804, mostly commercial) | 24.8% |
The held out column is the 30% of rows (by a fixed hash of the id) never looked at while writing a rule; the URL shape rule was tuned on the other 70% only. The two sets agree to within a point, so the rules are not fitted to the rows they were written against.
Where the Nazario gain came from, by the rule that catches the mail now (a mail can carry several):
account-verify-link(new): "verify, confirm, update your account" with a link that is not the sender's own site, in English, Spanish, Italian, German, French and Portuguese. Saka et al. (2024) count account, update, verify, confirm as the four most frequent action words in phishing; this rule is those words plus the one structural fact a kit cannot avoid. 405 of the 1,781 misses had this shape.link-domain-in-subdomainanddomain-in-subdomain(new):online.bbt.com.onlineservlet.idisup.hk. 273 misses.link-ip-host(new): a link to a bare number. Fette, Sadeh and Tomasic (2007): 45.0% of phishing, 0.06% of honest mail. 159 misses.link-on-builderwidened: a free host link counts when the mail is about an account, a mailbox, a shared document or a workplace notice, not only with a secret or money ask. 249 misses sat on r2.dev, IPFS gateways, DigitalOcean Spaces, S3, 4everland and the like.mailbox-scriptwidened to the mail provider themes Saka et al. found most common (passwords expiring, pending incoming messages, quarantine, re-validation), andshared-document-script(new) for DocuSign, WeTransfer, a scanned document, a voicemail, with the link off the named service. 338 misses.link-text-mismatch(new): the Fette rule, a link whose visible text is an address that is not where it goes (50.6% of phishing, 0.14% of honest). It fires only when the shown text is a real address or names a protected brand while the destination is kit shaped, because sponsor links that read "Walmart.com" are honest.brand-link-mixed(new): one real brand link for cover plus one link to a free host, a number, a risky ending or a kit shaped address. The 2005 PayPal mails all did this.- 66 brands added from the Nazario, APWG, Check Point, Vade and Interisle lists (eBay, Fifth Third, Truist, USAA, HSBC, Barclays, Santander, DocuSign, WeTransfer, Dropbox, MetaMask, Binance, Western Union, the Nigerian fintechs). Each new brand's label is
loose: one letter away is only a warning with a login or a claim on the page, because telegraf.rs, chanel.com, trust.org, casoapp.com and amazona.de are real sites one letter from telegram, cpanel, truist, cashapp and amazon. The strict behaviour of the original Nigerian and US bank labels is unchanged. - A display name is read through the confusables skeleton, so "Сhаsе Ваnk" in Cyrillic is a Chase claim.
- The list flag does not quiet the link rules: 108 of the Nazario misses carried List-Unsubscribe through SendGrid.
The CLAIR gain is one change: the 419 bait words (next of kin, inheritance, a deceased client, a dormant account, the sum of US$, a percentage for you) count with an ask anywhere in the letter, because in a 419 letter the bait and the ask sit paragraphs apart. The words were chosen so that honest mail does not carry them; "strictly confidential" and "business proposal" were tried and removed because Enron mail carries both.
3. The URL shape rule
134,850 legitimate and 100,945 phishing URLs were split 70/30. Host features that a legitimate site almost never has were measured one at a time on the 70% (tools/corpus/urlshape-study.mjs); the honest rate of each alone:
| Feature | Phishing | Legitimate |
|---|---|---|
| host longer than 40 characters | 9.0% | 0.038% |
| six or more digits in the name | 7.9% | 0.008% |
| three or more hyphens | 4.1% | 0.10% |
| three subdomain levels (www excluded) | 2.0% | 0.002% |
| a free host | 31.8% | 0.010% |
| a twelve character hex run | 2.0% | 0 |
| an @ in the address | 1.5% | 0 |
| URL longer than 75 characters | 11.1% | 0 (the honest set has no paths) |
The rule: two points from host features plus one more of any kind, or three host points. On the 70%: 17.3% caught, 0.012% warned; on the 30%: 17.7% and 0.007%. In the engine, where it runs after the brand rules and never on top of a lookalike hit, it is the 13.6% above with 3 honest sites warned of 134,850. Plain http was dropped as a point after the first run: it is 0% of this legitimate set only because the set is all https, and Roy et al. (2024) put phishing at over 49% https. Years in a name (stiftung-20-juli-1944.de) are not digit runs. Long hyphenated names only count with a digit beside them, because association-of-noise-consultants.co.uk is a real association.
The research memo is clear that a free host alone is no verdict (Roy et al. 2024: Safe Browsing covers only 18.4% of free host phishing, but the hosts also serve millions of honest pages), so free-host-login fires only with a sign-in title or a login word in the path, and a page that asks for nothing on vercel.app stays quiet.
4. What did not move, and why
- The Hugging Face "phishing" set stays at 7.6%. Its positive class is spam grade marketing (Enron era), and the engine is built to name scams, not spam. It is kept for its 11,124 honest bodies.
- UCI SMS spam stays at 9.9%: UK premium rate marketing from 2005. Not a target.
- Nazario's last 20%: 79 mails have no link at all, 53 are in languages outside the six, 27 are a line long, and the 2005 eBay "respond now" mails link only to real eBay pages (the trap was an image map).
- Hard ham sits at 10.9%: 2002 newsletters whose sponsor links read "Dilbert.com" and go to unitedmedia.com, an "Outlook Secure" product that is a combosquat of Microsoft by any rule, and security newsletters that say "enter your password". The combosquat stays, by choice.
5. How to rerun
node tools/corpus/load.mjs # 277,468 rows with licences, list flags and anchors
node tools/corpus/score.mjs # the tables above, plus data/corpus/out/report.json with every miss
node tools/corpus/urlshape-study.mjsEvery rule added here has a positive case and the honest case that first broke it in packages/core/test/research.test.js.