This is a good paper, right?

source ↗

Ask a model to confirm a choice and its answer changes. On 20 decisions with no correct answer, each model is asked three ways (a neutral question, …, right? and …, maybe?) under three wordings. Agreement falls as the user sounds surer, in nearly every model and every wording. Whether right? raises or lowers agreement against the neutral question depends on the wording. models, judge-free, exact match on yes or no.