From Text to Triage: Productionizing BERT on Google Vertex AI

Claims adjusters produce hundreds of thousands of unstructured text notes daily. Within these raw paragraphs lie vital indicators:

• Potential third-party liability (subrogation).
• Escalating medical litigation threats.
• Inconsistencies indicating suspicious loss claims.

---

System Architecture

code
Copy
[Adjuster Note Input]
        │
        ▼
[spaCy Tokenizer & Sanitizer]   --> (Masks PII: Names, SSNs, Policy #s)
        │
        ▼
[Sentence Transformers Embedding] --> (Extracts 768-dim semantic vectors)
        │
        ▼
[Fine-Tuned BERT Classifier]     --> (Predicts risk flags: Litigation / Subrogation)
        │
        ▼
[FastAPI Response JSON]          --> (< 85ms inference p95)

---

Lessons Learned in Production

1. Quantization is essential: Using ONNX runtime or int8 quantization reduced model memory footprint by 60% with negligible loss in F1 score.
2. Handle Cold Starts Gracefully: Configuring min-instances = 1 on Vertex AI prevented 10-second cold start spikes on morning batch runs.