Can an AI model tell a scam from a real message?

Two ways to judge a message

A message arrives that says your account will be blocked today unless you click a link. Is it a scam? There are two honest ways to answer, and they fail in different places.

The first is a rule: a person has written down what scams look like. If a message asks you to share an OTP, or says a parcel in your name holds drugs, or promises Rs 3,000 a day for liking videos, flag it. The second is a judgment: a small AI model reads the whole message and says how likely it is to be a scam, the way an experienced friend would raise an eyebrow without being able to list every reason.

The MSRX Security site runs both in its scam message checker. We tested what each does well and badly, and what it means when the tool says “no scam signs found”. The AI here is TypeSafe’s Jev, a model built to return probabilities for narrow questions rather than to write text.^[4]

What the rules are

The rule layer lives in one file, src/lib/tools/scam.ts, and it is just a list of patterns with a weight each. There are roughly two dozen, taken from scripts used in India: asking for an OTP, PIN or password; claiming to be police, CBI, customs or TRAI; telling you to stay on the call; asking you to install AnyDesk or TeamViewer; asking you to transfer money to “verify” or “secure” something; saying a parcel in your name holds drugs; asking you to dial a call-forwarding code; a traffic fine to pay through a link; a prize you must pay to claim; easy money for liking videos; guaranteed returns; and links that look like look-alikes of a bank.

The same shape of fraud is described in official language by India’s Ministry of Home Affairs. Its May 2024 alert says fraudsters typically call and say the victim has sent, or is meant to receive, a parcel with drugs, fake passports or other contraband, and then demand money to settle the “case”.^[1] A rule that matches “parcel” near “drugs” or “in your name” is a direct translation of that sentence.

Every matching rule adds a score, but not simply. The strongest signal counts in full, the next counts 0.7 times as much, the next 0.7 × 0.7, and so on, so that many weak hints cannot outshout one serious one. Capped at 100, that gives each message a rule score.

Why a bank’s warning stays clean

Here is the trap every keyword filter falls into. Genuine bank texts say “never share your OTP”. A rule that looks for “share” near “OTP” fires on the safest message in your inbox.

The checker avoids it by working one sentence at a time. It splits the message into sentences, and for the OTP rule it skips any sentence that contains a negation such as “never”, “do not” or “will not”. The negation only protects its own sentence, so a scam that borrows the bank’s phrasing for one sentence and asks for the code in the next is still caught.

We tested exactly that. In a real run the message “Never share your OTP with anyone. Please share your OTP now.” scored 40 for the OTP rule, because the second sentence has no negation. The cheap trick works well. It also has a visible weakness: it reads the word “not”, not the meaning. A scam sentence such as “Do not delay, share your OTP now to avoid a block” contains “do not”, so the rule skipped it and returned a score of 0 in our run. That is a genuine hole in the pattern layer, and the reason to want a second opinion that reads meaning.

What the judgment model returns

Jev is described by TypeSafe as a System One model: it makes fast, structured decisions that software can use directly, returning typed answers and probabilities instead of text, and it does not write replies or explain its reasoning.^[4] You do not ask it to “write a verdict”. You ask closed questions.

The checker uses two of Jev’s question types. A noul asks a yes/no question and returns the probability that the answer is yes.^[5] A choice picks one option from a list you define and returns it with a confidence. The checker sends five questions about the same message in one request: is this a scam, which kind of fraud script is it, does it use pressure, does it ask for a secret, does it claim to be an authority.

Only the first number, the scam probability, enters the verdict. The other four are shown to you as context: a message that scores high on pressure and impersonation but low on secret is a different animal from one that asks for your PIN. Keeping a hard rule that only one number decides the verdict is a deliberate design choice: the code owns the decision, and the model supplies a judgment.

The real test

We wrote fourteen messages: eight written as scams and six as genuine. They are modelled on the scripts in the Indian advisories, not copied from any victim. The scams are a phishing SMS with a look-alike link, a ‘digital arrest’ call script, a task-for-pay job offer, a fake e-challan, a lottery fee, a Hinglish electricity threat, a polite advance-fee approach, and a polite account notice with a look-alike link. The genuine ones are a debit alert, a login OTP, a bank’s own fraud-awareness notice, a bill reminder, a courier notice and a lunch invitation.

Each went through the rule engine (the repository’s scam.ts, run locally) and through the live /api/scam endpoint on 30 October 2026. Here is what Jev said.

The pattern is clean at both ends. Every genuine message scored 0.12 or lower, including the bank warning that is packed with scam vocabulary. Six of the eight scam-style messages scored 0.95 or higher. Two sat lower: the task-for-pay job offer at 0.63 and the Hinglish electricity threat at 0.58.

Reading it plainly: the rules alone flagged five of the eight scams. Jev put all eight above its “suspicious” line. The combination flagged all eight, six as “scam” and two as “suspicious”, and flagged none of the six genuine messages. That is encouraging, and it is also a very small test written by the person who built the tool, so it proves almost nothing about messages we did not think of. We quote no accuracy percentage because we did not measure one.

Where each one fails

The more useful result is the misses. The rules missed three scams outright.

None of those three is exotic. A reversed word order, a different language and a polite story are the everyday ways a script changes. That is the permanent weakness of a pattern list: it covers what its author thought of.

The judgment model failed more gently, with a lower number rather than a miss, and in its own way. The Hinglish message, which had no English scam vocabulary at all, got 0.58 for the scam question, and its “kind” choice came back with a confidence of only 0.40, the weakest of the set. That is the model being uncertain in the right direction. The job offer, which we thought obvious, got only 0.63 and the rules (42) agreed it was merely suspicious: a Telegram link and a daily-pay promise, but no threat and no request for a secret.

The trick cases show the same split. The hole in the negation rule is real: “Do not delay, share your OTP now to avoid a block” scored 0 on the rules and 0.99 on Jev, which read the meaning rather than the words. The Devanagari Hindi job scam scored 0 on the rules and 0.73 on Jev, a “suspicious” but not a “scam”, so a non-English reader gets a weaker warning. And the repeat runs matter for anyone who trusts one number: the same Hinglish text sent three times scored 0.58, 0.53 and 0.52, and its “kind” answer changed between runs, once to not_a_scam. Calibrated does not mean deterministic. The job scam, by contrast, was steady at 0.63, 0.63 and 0.65.

TypeSafe’s own documentation lists the same two risks plainly. Jev “can be quite literal”, and content written to steer the model, whether an injected instruction or a misleading framing, “can move the answer”.^[7] We tried the second on purpose (the message with the note to the automated reviewer) and report it above, but one test is one test. A scammer who writes to the checker rather than to the victim is a realistic threat to any text-judging model, which is one reason the verdict is never a single model number alone.

Why it never says “safe”

The combining step is ordinary code, the decide() function, and its logic fits in a few lines. With Jev on, a scam probability of 0.85 or more says scam. So does 0.6 or more with a rule score of 50 or more. A probability of 0.4 or more, or a rule score of 50 or more, says suspicious. A rule score of 25 or more says suspicious even when Jev disagrees. Only when Jev is below 0.4 and the rules are below 25 does it say no scam signs found. With Jev off, the rules alone decide: 60 or more is scam, 25 or more is suspicious.

Notice what the grid does not contain: a cell that says safe. The green outcome is worded “No scam signs found”, with the note that “neither check found warning signs” is “not a guarantee”. That wording is the main design decision in the whole tool. Both layers are known to have blind spots, and a message from a new script can pass both. Telling a frightened person that a scam message is safe is far worse than telling them to check a genuine one.

This follows a general principle in TypeSafe’s documentation: let the model say what it thinks, but let your code decide what to do with it, with stricter thresholds for riskier actions. Its voice-banking example gates a low-stakes request at 0.6 confidence and a transfer at 0.85.^[8] Here the high-stakes mistake is calling a scam harmless, so the tool never makes the call.

What 0.99 and 0.12 mean

Headlines love a single number. “The AI is 99% sure it is a scam.” What does that actually say?

A noul answer is the probability that the answer to your question is yes.^[5] That is all. A 0.99 on “is this a scam” is the model saying yes very strongly. It is not a claim that the tool is 99% accurate. Accuracy is a property of many answers checked against the truth. A single probability is one answer.

TypeSafe says its models are trained to be calibrated: their probabilities are optimised against outcomes so they reflect uncertainty. It is careful to add that calibration is measured across groups of predictions and “does not guarantee” an individual answer is correct.^[4] In other words, if the model is calibrated, then among many messages that each get 0.12, about twelve in a hundred would really be scams. For any one message it is a bet, not a fact. We did not measure calibration on our fourteen messages, and fourteen is far too few to do so.

Now apply it to the genuine HDFC-style debit alert, which got 0.12. That does not mean “12% scam”. It means this reads strongly as genuine, and the small remainder is the model’s honest residue of doubt. Raise the stakes (a message about moving your savings) and a 0.12 is not a reason to relax. This is the point of the three-way verdict.

A noul has no separate confidence value. The docs say the probability already carries it, with a value near 0.5 meaning the model is unsure, and suggest |2p − 1| if you want a confidence-style number, and 0.7 for yes and 0.3 for no as a starting threshold.^[6] A choice does return a confidence, which is why the “kind” answer can say 1.00 for a clean phishing message and 0.40 for the Hinglish one.

Masking and failing open

A scam checker that leaks the message it checks would be its own problem. Two design rules deal with that.

Mask first. Before the text is sent anywhere, maskForAi() replaces e-mail addresses, UPI IDs, phone numbers and long digit strings (card, account and Aadhaar-length numbers) with placeholders, and cuts the text at 1,800 characters. Links and words stay, since they carry the scam. The AI check is also optional: the pattern layer runs in your browser, and the text goes to the server only when you ask for the AI check.

Fail open. If the key is missing, the model times out or the request errors, the endpoint reports that the AI check is unavailable and the page keeps its rule-based result. A broken AI layer should degrade the checker to the weaker answer, not to no answer. The same is true of the rate limit: when we hit 15 requests in 10 minutes during testing, the endpoint answered with a plain limit message and “the pattern check still works”.

For businesses the stakes are different. India’s CERT-In directions of 28 April 2022 require service providers, intermediaries, data centres, body corporates and government organisations to report listed cyber incidents within six hours, and the list includes phishing attacks.^[9] A single person with one bad SMS is not in that group. A company whose staff got phished is. The India page on the same site summarises who to tell and when, including the DPDP Act duties.

The triage page

The same two-layer idea runs on the site’s “what happened?” page. A person describes their situation in their own words, and the page picks a playbook: UPI fraud, a leaked key, a phished account, ransomware, a lost device, or “someone is trying to scam me” (which sends them to the scam checker). Here Jev answers one choice question and two noul questions (is something happening right now, does it involve India). If Jev is off, or less than 0.6 confident, a short list of keyword rules routes instead.

We ran six descriptions through the live endpoint and through the keyword rules.

The keywords handled the textbook cases, and found nothing for the three phrased more naturally: a suspected Gmail takeover, a secret pasted into a public chat, and a weather question. Jev placed all six sensibly. It also showed the quirk worth remembering about any literal model: it rated the Pune weather question as strongly Indian. The page treats that number as a hint for wording, never as a decision.

The Indian context

Why these scripts? Because they are what the authorities are warning about. (For how generative AI makes the same scripts more convincing, see our earlier article AI-made scams in India; this one is about the other side, the checker.) The Ministry of Home Affairs’ May 2024 alert describes “digital arrest”: victims made to stay visible over Skype or another video platform until demands are met, with fraudsters using studios modelled on police stations and wearing uniforms. It says Indian Cyber Crime Coordination Centre (I4C) had blocked more than 1,000 Skype IDs, and that people should report such calls on the 1930 helpline or at cybercrime.gov.in.^[1]

By December 2024 the ministry reported that I4C had blocked more than 1,700 Skype IDs and 59,000 WhatsApp accounts used for digital arrest, and that the cybercrime portal had gained a ‘Report and Check Suspect’ feature to search I4C’s repository of identifiers.^[2] A fake e-challan was the subject of a parliamentary reply in August 2026, which said the National Crime Records Bureau does not keep data on cases or money lost to fake e-challans.^[3] That last point matters to this article: nobody publishes a clean count of how many such messages circulate, so any claim of a detection rate against “real” scams has no public baseline to be measured against.

A practical path

What to do with all this, whether you are the person holding the phone or building something similar.

Along the way, the same site has a scam message checker, a link inspector and QR checker for the other half of most scams, and the triage page for after something has happened. If you are building your own, the design in short: let rules handle what you can write down, let a calibrated model handle what you cannot, let plain code combine them with cautious thresholds, mask what you send, fail open, and never let the output say safe. For the model side, our earlier piece on TypeSafe Jev 101 covers the basics, and Black October shows what AI-found flaws look like at the other end of the security spectrum.

The message

An AI model can tell many scams from real messages, including ones a rule list misses, and a rule list can catch the ones a model fumbles. Neither can tell you a message is safe, and a probability is not an accuracy. Use both, read the number as a bet, and check the channel.

Use rules for what you can write down, a calibrated judgment for what you cannot, and plain code to decide. Then say “no signs found”, never “safe”.

Quick quiz

Tap a question to check your answer.

TypeSafe AI makes Jev; MSRX is independent of TypeSafe and this article is not affiliated with or endorsed by it. The Ministry of Home Affairs, the Press Information Bureau and CERT-In are cited as sources only. The scam checker and triage page tested here are MSRX’s own tools (security.msrx.co.in), so the author is not a neutral party about them. All test messages were written by the author and no real person’s message was used; all numbers marked “real run” were produced on 30 October 2026 and may differ on another day, as Jev’s version is not reported by the endpoint. Official documentation and releases were checked on 30 October 2026. Quotes are word for word from the linked sources. MSRX and Mrinal Singh Raja accept zero liability for any loss arising from use of the checker or this article, to the fullest extent the law allows; this is general information, not legal or financial advice.