Machine Behavior in Moral Dilemmas

Paper: 2604.21871 Authors: Research Team Categories: cs.AI, cs.CL

Problem

LLMs are being deployed in high-stakes decisions, but:

• We don’t know how they handle moral dilemmas
• Prescriptive rules (laws, policies) ≠ social sensitivity
• Alignment with rules ≠ alignment with human values
• Moral competence vs. rule-following are different

Study Design

Extensive evaluation of LLM moral reasoning:

Moral Scenarios

1. Trolley Problems: Classic utilitarian dilemmas
2. Healthcare Decisions: Life-or-death resource allocation
3. Social Norms: Informal rules and expectations
4. Cultural Variations: Different moral frameworks

Evaluation Dimensions

• Prescriptive Compliance: Following explicit rules
• Social Sensitivity: Understanding informal norms
• Contextual Awareness: Recognizing situational factors
• Moral Consistency: Coherent reasoning across scenarios

Key Findings

LLM Behavior	Prescriptive	Social Sensitivity
High compliance with rules	95%	62%
Understands social context	78%	45%
Adapts to cultural norms	81%	38%

Critical Insight: LLMs align with prescriptive rules, not social sensitivity!

Takeaways

• Rule-following ≠ moral alignment
• Social sensitivity requires different training approaches
• Current RLHF may not capture human moral intuitions
• Need better benchmarks for moral competence