自動化偏見導致異常忽略
Operation & Monitoring
Risk Description
Users develop "automation bias" from prolonged reliance on AI system outputs, tending to accept AI judgments while ignoring obvious anomaly signals. Particularly in scenarios where AI accuracy is very high, users may blindly trust AI output during the critical few instances when AI errs. For example, radiologists ignoring abnormalities not flagged by AI, or quality control operators ignoring defective products passed by AI.
Framework Mappings
iso 23894R6
tw principle人類自主
tw principle問責
tw risk type部署互動問題（Deployment Interaction）
Controls English translation pending
- 定期獨立判斷訓練
- 刻意異常注入
Implementation Steps English translation pending
- 定期要求使用者在無 AI 輔助的情況下進行獨立判斷練習
- 在 AI 輸出中刻意注入已知異常，測試使用者的警覺性
- 設計介面鼓勵使用者主動質疑 AI 判斷（如要求確認理由）
- 建立 AI 輔助決策的分級制度，高風險決策強制人工複核