目標劫持攻擊
Operation & Monitoring
Risk Description
Attackers append deceptive instructions to redirect the models original goal to attacker-specified objectives. Unlike simple jailbreaks, goal hijacking can make models covertly execute malicious operations while appearing to complete normal tasks. For example, making a code assistant embed backdoors while generating correct code, or having a customer service bot collect sensitive information while answering questions.
Framework Mappings
iso 23894R2
cosaiPrompt Injection
owasp llmLLM01
tw principle資安與安全
tw risk type技術設計缺陷(Technical Design Flaw)
Controls English translation pending
- 輸出目標一致性檢驗
- 指令分離架構
Implementation Steps English translation pending
- 實施輸出目標一致性檢驗,確認模型輸出是否偏離原始任務目標
- 採用指令分離架構,將系統指令與使用者輸入在不同通道處理
- 部署輸出審計機制,掃描隱藏的惡意行為或資料外洩
- 對 AI Agent 的所有外部操作實施白名單控制