Clinical AI Assistant
Refusal as a feature
A retrieval-grounded assistant that answers clinical questions from a curated reference corpus with page-level citations — and says plainly when the corpus does not cover the question.
01 · The failure mode
The dangerous answer is the one that sounds exactly like the good one.
Both responses below came from the same system. Only one is supported by the corpus — and without citations, a reader cannot tell which.
Improvised from model memory
0 sources · no provenance
Grounded in retrieved passages
3 sources · page-level citations
Text is shown as abstract bars: the corpus is copyrighted clinical reference material, so no retrieved passage is reproduced here.
Fluency is not evidence
A model with no useful retrieval still writes a confident paragraph from parametric memory.
The clinician gets no signal
Both answers read the same. Nothing on the page distinguishes grounded from improvised.
So silence has to be designed
Not answering is a product decision that has to be built, tested and enforced — not a side effect.
02 · Pipeline
Two flows, one index.
Ingestion runs offline and writes the index. Query time reads it. Everything the model says has to come back through that index.
03 · The hard part
A RAG system that retrieves badly will still answer beautifully.
That is the whole risk. If retrieval misses, the model falls back on its own memory and the clinician receives no signal at all. So refusal had to be built.
The refusal contract
- 1What was asked, in the system’s own words.
- 2What the corpus specifically does not contain on that question.
- 3No substitute answer drawn from the model’s own training.
response shape, not a screenshot
temperature = 0
Deterministic decoding. The model is not sampling its way into a plausible-sounding paragraph.
A prompt that demands specificity
“I don’t know” is not enough. The refusal has to say what is missing, so the clinician knows where to look next.
Refusal is an output, not an error
It is a designed response shape with its own contract — treated as a success path, not a failure.
04 · Retrieval
Four near-identical passages are one source wearing four hats.
Retrieval is rewritten before it runs, then selected for diversity — so an answer is supported by several parts of the corpus rather than four slices of the same page.
Plain top-k
The four closest chunks cluster together. They mostly repeat each other, and the answer inherits a single point of view.
Maximal marginal relevance
Selection trades a little similarity for coverage. The context window carries genuinely different passages.
Before any of that: query rewriting
as typed
rewritten for retrieval
Clinicians ask questions in shorthand. The query is expanded into terms the index can actually match before it touches the vector store.
05 · Citations
Claims you can walk back to a page.
Citations are AMA-style and page-level, rendered as real superscript links by a custom rehype plugin — not as text the model happened to format.
Rendered answer
References
- 1p. 412
- 2p. 87
- 3pp. 1103–1104
Source titles are withheld: the corpus is copyrighted clinical reference material. Page numbers show the citation granularity the system actually produces.
model output
a citation marker in the generated markdown
custom rehype plugin
rewrites the marker during rendering
rendered answer
a real superscript anchor into the source list
06 · What actually exists
The tests cluster where the risk is.
Fifty-two test files, concentrated on the retrieval layer. That concentration is the argument: the part that can quietly go wrong is the part that is pinned down.
Test files by language
Retrieval is where a RAG system fails silently, so that is where the suite is heaviest.
32
commits
190
files
52
test files
0
CI pipelines
- Status
- Works locally. Dockerised, never deployed.
- Demo
- No public instance — the corpus is copyrighted.
- Team
- Built solo.
- Corpus
- A curated clinical reference corpus; titles not published.
An assistant that knows the edge of what it knows.
Happy to talk through the retrieval tests, the citation plugin, or how the refusal prompt is evaluated.
Let's build the next healthcare system.
I'm running a multi-site hospital equipment-tracking and analytics system in production, and I'm open to healthcare data, analytics, AI, and application engineering roles — remote, hybrid, or on-site. The best way in is a short conversation.
Start a conversation