Text-to-Code Search
Find medical codes from free text using BM25 (lexical) and fine-tuned SapBERT embeddings (semantic).
This is medterm4ds's key differentiator. Traditional terminology servers require you to already know the code. medterm4ds lets you search by natural language and get ranked code candidates with confidence scores.
Quick example
import medterm4ds as mt
# Lexical: BM25 token matching (~1ms)
results = mt.search("diabetes", mode="lexical", count=5)
for r in results:
print(f" {r.score:.2f} {r.match_grade:8s} {r.source:15s} {r.code:12s} {r.display}")
# → 0.87 certain SNOMEDCT_US 73211009 Diabetes
# Semantic: catches novel phrasings where BM25 fails
results = mt.search("high blood sugar", mode="semantic")
# → 0.80 probable ICD10CM R73 Elevated Blood Glucose Level
# Hybrid: best of both worlds
results = mt.search("metformin pill", mode="hybrid")
# → 1.00 certain RXNORM 1161611 Metformin Pill
Three search modes
| Mode | Engine | Latency | Best for |
|---|---|---|---|
lexical (default) | BM25 inverted index | ~1ms | Known medical terms ("diabetes", "metformin") |
semantic | SapBERT embeddings + FAISS | ~70ms warm | Novel phrasings ("high blood sugar" → Hyperglycemia) |
hybrid | Reciprocal-rank fusion of lexical + semantic pools | ~130-170ms warm | Best accuracy (default for extraction) |
First semantic call in a process pays a one-time model load (~1 min cold, including the split-layout fetch). Numbers are v0.0.5 steady-state medians (see the releases page benchmark section).
Why three modes?
Lexical (BM25) is fast and deterministic — it matches tokens. But it fails on:
- Acronyms: "T2DM" → no BM25 match for "Type 2 Diabetes Mellitus"
- Synonyms: "high blood sugar" → no token overlap with "Hyperglycemia"
- Paraphrasing: "water pill" → no match for "Diuretic"
Semantic (SapBERT) catches these by embedding the query and finding nearby medical concepts in vector space. But it's 100x slower per query.
Hybrid fuses the lexical and semantic candidate pools by reciprocal rank — either system can carry a row, so a lexical miss no longer hides the semantic best answer (the old cascade failure).
Match grades
Each result has a confidence grade (modeled after FHIR Patient $match):
| Grade | Score range | Meaning |
|---|---|---|
certain | ≥ 0.8 | High confidence match |
probable | ≥ 0.4 | Likely match, review recommended |
possible | < 0.4 | Possible match, needs human review |
Search indexes
The search service lazy-loads pre-built indexes on first use:
| Index | Size | Source |
|---|---|---|
| BM25 JSONs (6 categories) | ~159 MB | HF model repo fhir4ds/medterm4ds (split layout, v0.0.5+; overridable via MEDTERM4DS_HF_REPO_ID) |
| SapBERT model | ~440 MB | Same repo (content-addressed per embedding space) |
| FAISS indexes (6 categories) | ~2 GB | Same repo (data units float to the latest published revision) |
No GPU required. All search runs on CPU.
Filtering by source
# Search only medications
results = mt.search("insulin", sources=["RXNORM"])
# Search conditions + medications
results = mt.search("diabetes medication", sources=["SNOMEDCT_US", "RXNORM"])
Available across all interfaces
# CLI
medterm4ds search "diabetes" --mode hybrid
# MCP
search(query="diabetes", mode="hybrid", count=10)
# FHIR API
GET /fhir/CodeSystem/$search?query=diabetes&searchMode=hybrid