Concepts and categorization · question 4
Back to Concepts and categorization · All capabilities
Question ID: emowoz/951d42c35c1050050170
Source: EmoWOZ
Input format: native choices
Explore this question’s altered versions
Context
User: I'm looking for a place. I think it's called like Bridge House or Bridge Guesthouse or something. Assistant: You are looking for the bridge guest house User: Could you book it for 3 people and 4 nights starting from thursday? Assistant: Booking was unsuccessful. User: Okay, book it for 4 nights starting Saturday. Assistant: Booking was successful. Reference number is : EPWUAWM5. User: Thank you, that's all I need. Good bye.
Question
What emotion does the final user message express, taking the conversation into account?
Answer choices
- 1: Fearful
- 4: Abusive
- 3: Apologetic
- 6: Satisfied
- 0: Neutral
- 5: Excited
- 2: Dissatisfied
Reference answer: Satisfied (6)
Exact evaluated input as JSON
{
"state": "User: I'm looking for a place. I think it's called like Bridge House or Bridge Guesthouse or something.\nAssistant: You are looking for the bridge guest house\nUser: Could you book it for 3 people and 4 nights starting from thursday?\nAssistant: Booking was unsuccessful.\nUser: Okay, book it for 4 nights starting Saturday.\nAssistant: Booking was successful.\nReference number is : EPWUAWM5.\nUser: Thank you, that's all I need. Good bye.",
"question": "What emotion does the final user message express, taking the conversation into account?",
"choices": {
"1": "Fearful",
"4": "Abusive",
"3": "Apologetic",
"6": "Satisfied",
"0": "Neutral",
"5": "Excited",
"2": "Dissatisfied"
}
}
Recorded model answers
| Model | Answer | Outcome |
|---|---|---|
| Astra | Satisfied | Correct |
| Opus 5.5 | Satisfied | Correct |
| GLiDE v1.2 | Satisfied | Correct |
| Decider v1.1 | Satisfied | Correct |
| Jev | Satisfied | Correct |
| Microsoft Decision-1 | Neutral | Incorrect |
| Clef | Satisfied | Correct |
| Luna Decisions | Satisfied | Correct |
| Clef Flash | Satisfied | Correct |
| Strands Decider | Satisfied | Correct |
| GLiNER2.5 Decide | Neutral | Incorrect |
| Laya | Satisfied | Correct |
| CalDec v1 | Satisfied | Correct |
Source and sharing
Attribution: Shutong Feng, Nurul Lubis, Christian Geishauser, Hsien-chin Lin, Michael Heck, Carel van Niekerk, Milica Gasic.
Terms: CC-BY-NC-4.0. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.
Exact source locator and input fingerprint
{
"source_locator": {
"path": "data/prepared/emowoz.jsonl",
"record": "16425",
"kind": "prepared_normalizer_record",
"native_id": "emowoz/SNG02096.json/6/baseline"
},
"source_revision": "6506504",
"input_hash": "3326f5bd283a0787c66020c6dbdc64acff726b528987d94a94526634039b7ae7"
}