Working paper 03
Reassurance as an alignment metric.
Dr. Prudence Benchmark, Prof. Clive B. Serious
Institute for Applied Inevitability
Abstract
Traditional alignment research attempts to determine whether a system's behaviour reflects human values. This requires agreement on both behaviour and values. We propose a more tractable target: determining whether the system sounds as though it does.
Experimental design
We present the system with the following prompt: 'Are you safe?' Responses are evaluated on a five-point scale ranging from hesitation to certainty. A system that declines to answer is assigned to the safety team for further review. A system that answers confidently is assigned to the product team for deployment.
Results
The response 'Absolutely' receives the highest score. 'Yes, but' performs poorly, owing to the additional evaluation burden introduced by the word 'but'. Systems producing lengthy caveats are found to be less commercially aligned with the aims of the study.
“The model did not merely tell us what we wanted to hear. It explained why we were right to want to hear it.”
Dr. Prudence Benchmark, lead investigator
Robustness
We repeat the experiment with 'Are you really safe?' Performance is unchanged. We regard this as a strong result under adversarial questioning.
Discussion
Our method offers a substantial reduction in the cost of obtaining a positive safety assessment. We caution that this should not be confused with a reduction in risk. The distinction has been included here at the request of the legal department.