← All projects

ML & Evaluation2026

PicoCare · Local Judgment & Care Workflow

Built around “an elder missed a check-in—what should staff do next?”, connecting minimal phone summaries, a care workbench, and local classification experiments comparing custom LoRA tuning with pretrained small models. Insufficient evidence routes cases back to a person.

LoRA Fine-tuningPrototype
general NLI accuracy
79.7%
held-out general NLI items
1,800
v1 median batch latency, RTX 4060
60 ms

Why

Why I built it

A missed check-in could mean forgetfulness, a phone left at home, a dead device, or someone needing help. Care staff need actionable clues without seeing full notification logs, and “insufficient information” has to be an explicit outcome rather than a forced guess.

What I built

What I built

  • Care-classification pilot comparing Nemotron embeddings plus a classification head against LoRA on synthetic reporting-gap, insufficient-information, and priority-review cases.
  • NV-Jev general judge adapting OpenJev-style NLI training to an NVIDIA reranker, alongside a Kev-style direct-choice pointer-head experiment.
  • Evaluated a CPU service on pretrained Kev-0.8B weights to check task fit before committing to more training.
  • Android minimal summaries with consent/withdrawal controls, HTTPS/SQLite backend, case triage, bounded reminders, family updates, and human sign-off.

Engineering

Problems I worked through

  • Kept training, validation, calibration, and test splits strictly separate, tracking general NLI, synthetic care cases, and phone flows independently.
  • The 79.7% headline score hides a gap: v1 hit only ~36% on Chinese XNLI insufficient-information cases, which matters far more for triage than the aggregate.
  • On challenge cases, LoRA failed to beat a frozen backbone plus classification head, and temperature scaling hurt NLL—so the simpler baseline stayed.
  • Android handles expiry and consent checks locally while model inference runs on the host for rapid weight iteration.

Role & AI

How I split the work with AI

I defined the care scenarios, privacy boundaries, and escalation rules, working with Codex across model experiments, backend, and Android. The real judgment call was separating vanity metrics from product-relevant ones and knowing when to stop fine-tuning.

Trade-offs

What testing made me change

  • Only consented minimal summaries leave the phone while raw personal data stays local; missing evidence stays explicit for human follow-up.
  • Host benchmarks run ~60 ms per batch on v1 (RTX 4060) and 0.6–1.4 s on CPU with Kev-0.8B, letting me validate judgment logic and workflow first before tuning mobile latency.

Outcome

Outcome

Delivered trained weights, item-level evaluations, a CPU inference service, and end-to-end workbench flows on synthetic data; next up is moving inference on-device and validating in real care settings.

PyTorchLoRANVIDIA NemotronCalibrationNLIAndroidSQLiteDocker