Classify

Labels and scores

Read the Botpoison Classify docs page "Labels and scores": https://botpoison.com/documentation/classify/labels-and-scores.md
The Botpoison agent skill is at https://botpoison.com/skills/botpoison/SKILL.md.
Use them to make this change in my project, on the stack it already uses. Ask me for keys or IDs rather than inventing them.

Every verdict has a label to act on and the score behind it. It also names the model that answered.

JSON

{
  "label": "clean",
  "score": 0.04,
  "model": "spam-standard"
}

Labels

SpamScore 0.8 and above
The message reads like spam. Drop it, or file it away.
CleanScore below 0.8
Deliver it as usual, or hold it for a person if its score is high enough for your own review queue.

Act on the label by default: if (label === "spam"). The label is Botpoison's recommended cut on the score, spam from 0.8 and clean below it, so the two never disagree. The dashboard counts and charts labels, so your form and your dashboard agree.

Scores

The score is a number from 0 to 1 that ranks how spam-like the submission is. It is not a probability. Each value stands for a false positive rate: the share of real form submissions in Botpoison's reference set, counted together, that score at or above it.

How often a real message scores this high

The share of messages from real people at or above each score.

Score	Real messages
Below 0.2	about 90%
0.2 and up	10%
0.3 and up	4.2%
0.4 and up	2%
0.5 and up	1%
0.6 and up	0.5%
0.65 and up	0.36%
0.7 and up	0.25%
0.75 and up	0.17%
0.8 and up	0.1%
Above 0.8	under 0.1%

Every score from 0.8 up is spam, so the table does not split those further. A higher score still ranks one message above another.

Every model release is fitted to this table, and a release ships only after a check on a separate set of real messages keeps its rates within twice the table at 0.3, 0.5, 0.65 and 0.8. So if (score >= 0.7) means about the same rate after a model moves to a newer release. The models page lists the release each model answers with today.

The rates count every real message in the reference set together, so sites with a lot of traffic weigh the most. Taken site by site, the share of real messages that reach a score is usually several times the table's rate, and higher still on a new site. The rates are measured on whole forms. A plain string sent as content is scored on the same scale, but its rates vary more around the table.

Choosing a threshold

Start with the label. Move to a number of your own only for a reason, and log the score beside what you did with the message while you tune it.

Default: 0.8, which is the spam label. 0.1% of real messages score that high. Drop these, or file them away.
Review queue: from 0.5 (1% of real messages) or 0.65 (0.36%). Hold these for a person rather than discarding them.

JavaScript

const { label, score } = await botpoison.classify(req.body, {
  model: "spam-standard",
});
// Keep the score beside your own outcome while you tune a threshold
log.info({ label, score });
if (label === "spam") {
  // 0.8 and above: 0.1% of real messages score this high
  return res.status(202).end();
}
if (score >= 0.65) {
  // Your own review queue: 0.36% of real messages score this high
  return holdForReview(req.body);
}
// Deliver it as usual

Forms with nothing to read

Some forms carry no free text at all, such as an email-only signup. The model has nothing to read in them, so Classify answers clean with a score of 0, unless the model is very confident: a score of 0.8 or more stays spam with its score. A review queue at 0.5 never holds a signup for having nothing in it.

A form has free text when it holds any of these:

A message or subject field, or a field whose name says nothing and whose value runs to five words or more.
Any word in a name, company, address or other field.
A link in any field other than email and website.
An email field that holds anything but one address, or a phone field that holds anything but a phone number.

A string sent as content is a message, so this never applies to it.

To see each label without real traffic, use test mode.