What happened when we ran our classification engine on itself?

We ran Complipath's own engine on Complipath itself. The classification engine returned minimal risk for both systems in the run, answered one sentence that is false about itself, and could not ask the question that matters most. We publish all three outcomes verbatim, because a self-assessment you can check is worth more than a clean one you cannot.

Runs of 11 August 2026 through the real classify(), app repository commit 8bebc71. Inputs and outputs recorded in full in this page's claims file.

What exactly did we run?

The engine behind the product's guided classification is deterministic: fixed rules mapping confirmed answers to a classification with citations. We answered its six questions twice (the engine as it stood that day; today it asks a short set of questions, more if your answers open follow-ups) — once describing the engine itself, once describing the draft generator — and ran the same classify() the workspace runs. No demo mode, no edited output; the claims file carries every answer we gave and every field that came back.

What did it say about the engine?

Risk level minimal. The reasoning began: "You answered that the system does not make, support or inform a decision about a person" — which is right: the engine classifies systems, not people.

Then it said something false: "because the model is trained in-house, also as provider". There is no model. The engine is hand-written rules over encoded legal knowledge, and its own vocabulary for "built in-house" assumes a trained model where none exists. We sent that finding into the product as a defect report instead of editing it out of this page — a self-run that only surfaced flattering output would not be evidence of anything.

What did it say about the draft generator?

Risk level minimal, with the role factor "Using a third-party model via API leaves you a deployer under Article 26." The role question for API-wrapped systems — whether that leaves you a provider or a deployer — is one the engine's own coverage page lists as something it cannot decide for you where you have not said so — what this check can and cannot decide keeps that list, written once by a person, never generated per visitor.

What could it not decide?

Whether its subject is an AI system at all. Question zero is not among the engine's questions — the decision tree says the same thing to every reader: every branch presumes the Article 3(1) definition is already met. Run on itself, the engine presupposes the answer to the one question this exercise was meant to illuminate. That is not a flaw discovered in embarrassment; it is the recorded boundary of the tool, and the run is evidence of the boundary, not an answer to the question. The definitional analysis — against Article 3, point (1), recital 12 and the Commission's guidelines — is its own page, whether Complipath is itself an AI system, and it reaches its conclusion by reading, not by running.

What this means for you

If you are a provider: this is what an honest self-assessment looks like at minimum — the real tool, the real inputs, the outputs kept even where they are wrong, and the boundary of the method stated. The same discipline applies to the assessment you record for your own systems: the reasoning is the evidence. The run and what followed it are dated in what changed, and when. The short form of this run, and whether the check needs an account, are among the questions, answered.

If you are a deployer: when a vendor hands you a clean self-assessment, ask for the inputs and for what the method cannot decide — for a high-risk system, the inputs are what the Annex IV technical documentation must record anyway. Ours are published; the boundary is named; and the classifications carry citations you can check against the articles they rest on. How far a team can carry that check by hand is measured on its own page: what in-house compliance work does well, and where it stops.

FAQ

What did the engine conclude about itself? Minimal risk, on both runs — the engine itself and the draft generator. The classifications, reasoning strings and citations are published verbatim in the claims file, together with every answer we gave, so the runs can be checked rather than taken on trust.

What did the engine get wrong? Its reasoning said "because the model is trained in-house" about a system that has no model — hand-written rules only. The vocabulary behind its build-type answer assumes a trained model where none exists. The finding went into the product as a defect report; the output stays published unedited.

What can a self-run not tell us? Whether the engine is an AI system at all. The engine's questions never ask it — every run presupposes the Article 3(1) definition is met. That threshold question is answered by reading the Regulation and the Commission's guidelines, on a separate page, not by running the tool.

Why publish an unflattering result? Because a self-assessment you can verify beats a claim you cannot. A company selling classification that edits its own engine's errors out of its self-run has published marketing, not evidence. The uncomfortable output is the proof that the rest is real.

Sources: engine runs of 11 August 2026 against the app repository at commit 8bebc71 — full inputs and outputs in this page's claims file and in the claims file of is Complipath itself an AI system; Regulation (EU) 2024/1689 (EUR-Lex), Article 3, point (1), for the definition the runs presuppose.

Where this question meets the product: what Complipath is. Answering it in a customer's questionnaire: how to answer AI questions in a supplier questionnaire.