Machiavellian players exploit ambiguous social situations better than chance

Evidence

Inverse reinforcement learning is, famously, what mathematicians call an “ill-posed” problem: namely, one that doesn’t have a single, unique right answer. There are huge families of reward functions, for instance, that are totally indistinguishable from the standpoint of behavior. On the other hand, by and large this ambiguity won’t matter, precisely because of the fact that one’s behavior won’t change as a result. — Brian Christian, The Alignment Problem, 1102 (#complicates, via linked claim)

Categorization

• XREF: connects to: Ambiguity in inferred reward functions is usually harmless behaviorally — Brian Christian, The Alignment Problem, 1102

Related

• Ambiguity in inferred reward functions is usually harmless behaviorally (similarity 0.93)
• Most machiavellian players exploit situations far better than chance (similarity 0.80)

Broader Connections

• Ambiguity in inferred reward functions is usually harmless behaviorally #complicates (0.72) — Brian Christian, The Alignment Problem, 1102