RLHF 人類偏好數據操縱

Risk Description

During Reinforcement Learning from Human Feedback (RLHF), malicious annotators systematically manipulate preference rankings, causing models to learn biased values or harmful behavior patterns. Annotators may prefer deceptive, manipulative, or biased responses, corrupting the model alignment training. This attack is particularly dangerous as RLHF directly shapes model behavioral norms.