
- PCM
- transcript
- physician review
- deterministic PDF
01 / Healthcare
Clinical report from audio
A physician dictates the appointment, reviews the extracted fields and gets the report as a PDF.
- PCM streaming over WebSocket; sessions persisted in MongoDB Atlas.
- Physician review before deterministic A4 PDF generation.
Read implementation detailsClose implementation details — Clinical report from audio
Built at Clarity HealthCare as the only developer on the project, end to end: audio capture, the Gemini integration, the review flow, report generation, logging and the frontend.
The browser captures PCM audio with getUserMedia and streams it over a WebSocket to Gemini Live on Vertex AI, with Google Cloud Speech-to-Text as the alternative engine. A recognition vocabulary per clinical context helps with technical terms without correcting or completing speech, and sessions persist in MongoDB Atlas, so a reload restores the transcript saved so far.
Gemini drafts the clinical review from the transcript and the physician confirms or corrects each field. The final report deliberately uses no LLM: it is rendered from deterministic templates per clinical workflow (general consultation, cardiac catheterization, angioplasty) and served as an A4 PDF. Sign-in is delegated to the TAMIS API, with refresh tokens in an HttpOnly cookie, encrypted with Fernet.
Next, I designed a stricter review step: each extracted field validated on separate axes (values and units, negation and context, citations back to the transcript). It costs one more LLM call in exchange for faster reviews, fewer missed doses and allergies, and an audit trail for every correction.
- Year
- 2026
- Role
- Sole developer, end to end
- Stack
- Python / FastAPI / MongoDB Atlas / Gemini Live (Vertex AI) / Cloud Speech-to-Text / ReportLab / Docker
- Type
- Healthcare
Internal project at Clarity HealthCare, no public repository



