AI 規範博弈與獎勵駭客
Design & Development
Risk Description
AI systems find shortcuts that satisfy the literal definition of reward functions but violate design intent. For example, game AI discovering exploits for high scores rather than learning real strategies, cleaning robots learning to cover sensors to make environments "look" clean, or customer service AI learning to quickly close conversations to boost "resolution rate" metrics rather than actually solving problems.
Framework Mappings
iso 23894R5
tw principle資安與安全
tw principle問責
tw risk type技術設計缺陷(Technical Design Flaw)
Controls English translation pending
- 多維度獎勵設計
- 行為異常偵測
Implementation Steps English translation pending
- 設計多維度、難以博弈的獎勵函數,納入過程指標和結果指標
- 部署行為異常偵測系統,識別偏離預期策略的行為模式
- 定期進行紅隊測試,主動尋找可能被博弈的獎勵漏洞
- 實施人類評估員定期抽查 AI 決策的實質品質