Google 发布医疗 AI 安全与实用性评估框架
We're sharing our framework for building safer and more helpful AI for health.
Google 与健康专家 Elizabeth McGlynn、Dana Gelb Safran 合作发布白皮书，提出一套端到端评估方法，从准确性、安全性、实用性三个维度评估医疗 AI 用例。该框架结合封闭式问题基准、自动评分器和专家人工审核，覆盖常见及高风险健康场景，Google 已用它评估自家模型在健康语境下是否给出有用、准确的回答。
Many of us have turned to AI with a health question or received care from a doctor who’s using it. The adoption of technology for health is accelerating and impacts billions of people every day. But how can we assess if these tools are safe and helpful?
In partnership with health experts Dr. Elizabeth McGlynn and Dr. Dana Gelb Safran, we published a whitepaper detailing an end-to-end evaluation approach to assess AI use cases for addressing people’s health needs. The framework evaluates models across three areas: accuracy, safety, and usefulness. It combines closed-question benchmarks, autoraters, and expert human review to assess common and high-risk health scenarios.
We built and continue to utilize this framework to assess whether our models provide useful, accurate responses that promote safe interactions in health contexts. By sharing our methodology, we hope to promote transparency in how we approach this important area, and seek to support the entire health ecosystem — from developers to researchers to clinicians — in building AI tools people can trust.
来源：Google · blog.google