souprem.github.io/Relay
—| relayCasesEvalsGatesExperiments
Synthetic prior-authorization cases · evaluation prototype
Relay turns a model's probabilities into actions software is allowed to take, and records why.
24.4%
of held-out cases auto-processed
244 of 1000 cases
0 of 244
unsafe automations observed
95% upper bound 1.5%
92.1%
chose the correct action
921 of 1000 cases
How a case moves
2 · narrow judgments
Diagnosis supported
0.980
Step therapy met
0.980
Documentation complete
0.920
3 · policy engine, in order
providerpassed
agepassed
contradictionpassed
documentationpassed
auto_processfired
4 · one action
AUTO_PROCESS
Every gate passed; the first that fires decides.
Prior authorization, decided safely.