Every answer is checked. Here is how we measure it.

OmniClew would rather say nothing than give a wrong answer. This page explains how answers are checked, how accuracy is measured, and where OmniClew is weaker.

Quoted from your content

Every OmniClew answer includes an exact quote from your content, and OmniClew rejects any answer whose quote is not a perfect match for the source. Visitors can open the source and see the passage highlighted in the original document.

Reviewed by you

You review the questions and answers before anything goes live, and you decide between statements that contradict each other. Answers you approve, edit or reject stay that way when OmniClew rebuilds.

Confidence that is tested, not guessed

OmniClew decides when to answer by testing itself. It asks its own answer key many rephrased questions and many questions that your content does not answer, and it sets its confidence level from the results. A build must pass quality checks before it can go live, and by default those checks require three things:

• Answer precision of at least 0.95. At least 95 of every 100 answers shown are correct.
• Answer recall of at least 0.80. At least 80 of every 100 questions that your content answers receive an answer.
• At most 5% of off-topic questions answered. Questions your content does not cover should almost always get no answer.

A build that fails the quality checks is not published.

Tested on real questions

Where possible, we also test OmniClew against real questions that your visitors have sent you, such as questions from your inbox. Real questions are a harder and more honest test than the ones OmniClew writes for itself.

What counts as accurate

The number that matters is how often OmniClew gives a correct answer to real questions, and how rarely it gives a wrong one. A wrong answer counts as worse than no answer. OmniClew is built so that when it does not know, it shows documents that may help, or nothing.

Published results

OmniClew is early, and we are still measuring it on real websites. We will publish results on this page when we have them, and we will not publish numbers we have not measured.

Where OmniClew is weaker

OmniClew is weaker on questions that combine facts from several documents, because it answers from one place in your content at a time. It is also weaker on fee tables in PDFs, because PDF tables currently lose their header row. We are working on both, and the roadmap lists them.