The answer should not grade itself

Part 4 of our AI 101 series, When AI makes things up, ends on one idea: the model cannot see that it is off. It sounds exactly as sure when it is wrong as when it is right, and nothing inside it connects the two. Someone else has to look.

We built that someone else into the product. This post is about how it works, what it promises, and, at more length than most product posts would bother with, what it cannot promise.

What you see

Under a reply in Ethyx you may find a small card labeled Fact-check. It opens with one overall reading: True, Mostly true, Missing context, Mixed, Misleading, Mostly false, False, or Not enough to say. Below that, each specific claim the checker looked at is marked Supported, Disputed, or Unverified, with a one-line reason and, where one exists, a link to the source that backed or contradicted it. The reply itself is asked to carry those links too, so a source is never only in the card.

If the card cannot settle anything, it says so plainly: We could not confirm or challenge this. Check these sources yourself.

On the newest reply in a text chat there is also a Fact-check button. Press it and the same check runs on that answer at your request.

The rule underneath it

Here is the part that matters. The model that wrote the answer never checks the answer. Neither does any other model from the same company. The checker is always a model from a different provider, with its own training data, its own curation choices, and its own critics.

The idea is borrowed from the basic principle behind peer review. A researcher does not referee their own paper. The journal sends it to someone who was not in the room when the argument was made, and who has no stake in it being right. That reviewer misses things too, but they miss different things, and that difference is the whole point.

Self-grading fails for a mechanical reason. Ask a model to check its own work and you are asking the same statistical guesser, trained on the same text, with the same gaps and the same eagerness to agree with the person in front of it, to notice a hole it already fell into. Sometimes it does. Often it politely restates the error with more confidence.

A model from another lab has read a different slice of the world and was shaped by different people. Take a concrete example of the kind of thing this is meant to catch. Some readers believe xAI's Grok carries a conservative tilt. Anthropic's Claude models draw the mirror-image criticism from conservatives. I am not going to adjudicate either reputation here. The point is narrower: whatever slant a model does have, a check by a lab that gets criticized from the other direction is more likely to flag it than a check by the model itself.

Be precise about what Ethyx does and does not do with that. We do not pair models by ideology. We do not have a table that says "send this one's answers to that one." What we guarantee is independence: the checker is a different provider than the author, every time. If no independent provider is available for a turn, the check does not run, and the model is told so rather than being allowed to grade itself.

And peer review has its own well-known limit, which applies here too. Reviewers share a field's blind spots. Two labs trained on the same public internet can be wrong together. A second opinion narrows the odds; it does not zero them.

Where it runs whether or not anyone asked

Most of the time the model decides to call the checker, the same way it decides to search the web. For a handful of topics we do not leave that to the model's judgment.

If the conversation is about voting (registration, deadlines, where to go, what is on the ballot), health (doses, symptoms, whether something is safe to take), law (whether something is legal, what the penalty is, what your rights are), or a blanket claim about a race, sex, gender identity, sexual orientation, or religion, the check runs. If the answering model skipped it, Ethyx runs it anyway on the reply before that turn is finalized, and attaches the result. The same is true when you explicitly ask, in words, whether something is true, or paste a link and ask if it is real.

Those are the topics where a confident wrong answer costs the most, and where a stereotype dressed up as a fact does the most damage. On the group claims in particular, the checker is told that a generalization with no real support is disputed or unverified, and that the gap is not to be filled with an explanation.

What Unverified means

Unverified means the check could not confirm it. It does not mean false. This is the single most important line in the card and the one people most often misread.

A claim comes back Unverified when the sources the checker found did not settle it either way. That covers a lot of true things: niche facts, very recent events, anything the open web has not written about much. It also covers made-up things. The card cannot tell you which, and it does not pretend to. It tells you to look.

The rest of the honest fine print: the checker can fail outright, in which case the reply is told to say what it could not verify and to invent nothing, no studies, no statutes, no polling places. The check is web search plus Wikipedia, not a newsroom. It is not legal advice and not a diagnosis. And it costs something, a small model call plus the searches, which shows up in your credits like any other work Ethyx does on your behalf.

It remembers being wrong

One more piece, because a correction that evaporates is not much of a correction.

When a reply is rated False, Mostly false, or Misleading, or any single claim in it comes back Disputed, Ethyx writes that down as a note on the conversation. Later turns are told, ahead of any other notes on the conversation, not to state those claims as true and not to cite the earlier reply as a source. The reply you already read is not rewritten; what changes is that the conversation stops treating it as ground truth. Supported and Unverified results do not write notes, only rejections do.

The check is only as good as its sources

Now the part I most want you to sit with.

Ethyx is not an authority on whether a source is right. The checker asks a much smaller question: do the sources it found agree with the claim? It is judging agreement, not truth. Those usually line up. Not always.

Repetition is not truth. If hundreds of websites publish the same false fact, a web search returns that fact with complete confidence, the checker reads several pages that all say it, and the claim can come back Supported with a real link attached. Nothing in the pipeline knows the pages copied each other or were all fed by the same press release. Part 4 called this rigging the crowd, and a fact-check built on search inherits the problem rather than solving it.

Wikipedia deserves its own sentence. It is a strong source for settled history and a weak one for anything recent, contested, or thinly covered, and the checker leans on it because it is free, structured, and usually right. Usually.

Sources vary in strength, and the card does not rank them for you. A link to a state election office and a link to a blog are both links. The citation is an invitation to click, not a seal. We put the URL in front of you precisely so that clicking is possible.

A tool, not a verdict

Put all of that together and the honest description is this: the fact-check is a second model reading search results. It can be wrong in the same ways the first model can be wrong. It is just less likely to be wrong in the same place at the same time, which is what a second opinion is for.

Supported means a source agreed. It does not mean the matter is settled. Disputed means a source disagreed. It does not mean the claim is dead. None of this replaces your judgment. It hands you the sources and a second reading so you can reach your own conclusion faster and with more to go on than the first reply's tone.

For anything that matters, a filing deadline, a dose, a legal question, where to vote, the last step is still you and an official source. Use the AI to understand. Use the source to decide. The card is there to shorten the distance between the two, not to close it.

Why tell you

We could have shipped this as a badge and let it speak for itself. I would rather write down what it does, because the description is the product.

"The checker is never the author's company" is a promise you can hold us to. "Unverified is not false" is a promise you can hold us to. "It runs on voting, health, law, and group claims whether the model wants it or not" is a promise you can hold us to. Stating them in public, alongside the limits, is the same posture as our earlier posts on what we can and cannot protect and who owns the work you give an AI. If we ever fall short of one of them, this page is the standard.

Signups are open. Try Ethyx below and put the card to work on something you already know the answer to; that is the fastest way to learn what it is good for. If something in this post does not match what you see, contact us. I read those.