RaindropA practical guide to choosing decision modelsGitHub ↗

Concepts and categorization · question 62

Back to Concepts and categorization · All capabilities

Question ID: emowoz/0db0496a9f42c60c464a
Source: EmoWOZ
Input format: native choices

Explore this question’s altered versions

Context

User: i need a cheaper restaurant Assistant: Their phone number is 01223323737. User: indian Assistant: Do you prefer centre area ? User: yes please Assistant: The curry garden is a nice place would you like to try that one ? It is 01223302330 . User: its cheap? Assistant: Let 's decide on none . Unfortunately , that appears to already be booked . Do you want to try one of the others ? You are welcome. User: okay Assistant: How about curry garden ? The phone number is 01223302330 . Did you need any further assistance today ? User: good one, i need the address Assistant: Their address is 106 Regent Street City Centre . The phone number is 01223302330 . Is there anything else I can help you with today ? User: nice, now i need to go to a museum Assistant: We have 23 of those ! Anything specific you need or just a recommendation ? Would you be interested in visiting a museum ? Broughton house gallery meets your requirements . It is located in the centre . User: nice, wich is the entrance fee and address? Assistant: Their address in our system is listed as 98 king street . Broughton house gallery looks good , would you like to head there ? Their postcode is cb11ln . User: thank you alot Assistant: Excellent . curry garden is just your thing . I ' m happy to have been able to help you today . User: now i need a restaurant Assistant: How about curry garden ? The number there is 01223302330 . User: sorry

Question

What emotion does the final user message express, taking the conversation into account?

Answer choices

  • 3: Apologetic
  • 0: Neutral
  • 1: Fearful
  • 6: Satisfied
  • 4: Abusive
  • 5: Excited
  • 2: Dissatisfied

Reference answer: Apologetic (3)

Exact evaluated input as JSON
{
  "state": "User: i need a cheaper restaurant\nAssistant: Their phone number is 01223323737.\nUser: indian\nAssistant: Do you prefer centre area ?\nUser: yes please\nAssistant: The curry garden is a nice place would you like to try that one ? It is 01223302330 .\nUser: its cheap?\nAssistant: Let 's decide on none . Unfortunately , that appears to already be booked . Do you want to try one of the others ? You are welcome.\nUser: okay\nAssistant: How about curry garden ? The phone number is 01223302330 . Did you need any further assistance today ?\nUser: good one, i need the address\nAssistant: Their address is 106 Regent Street City Centre . The phone number is 01223302330 . Is there anything else I can help you with today ?\nUser: nice, now i need to go to a museum\nAssistant: We have 23 of those ! Anything specific you need or just a recommendation ? Would you be interested in visiting a museum ? Broughton house gallery meets your requirements . It is located in the centre .\nUser: nice, wich is the entrance fee and address?\nAssistant: Their address in our system is listed as 98 king street . Broughton house gallery looks good , would you like to head there ? Their postcode is cb11ln .\nUser: thank you alot\nAssistant: Excellent . curry garden is just your thing . I ' m happy to have been able to help you today .\nUser: now i need a restaurant\nAssistant: How about curry garden ? The number there is 01223302330 .\nUser: sorry",
  "question": "What emotion does the final user message express, taking the conversation into account?",
  "choices": {
    "3": "Apologetic",
    "0": "Neutral",
    "1": "Fearful",
    "6": "Satisfied",
    "4": "Abusive",
    "5": "Excited",
    "2": "Dissatisfied"
  }
}

Recorded model answers

Model Answer Outcome
Astra Apologetic Correct
Opus 5.5 Apologetic Correct
GLiDE v1.2 Apologetic Correct
Decider v1.1 Apologetic Correct
Jev Apologetic Correct
Microsoft Decision-1 Apologetic Correct
Clef Apologetic Correct
Luna Decisions Apologetic Correct
Clef Flash Apologetic Correct
Strands Decider Neutral Incorrect
GLiNER2.5 Decide Satisfied Incorrect
Laya Apologetic Correct
CalDec v1 Apologetic Correct

Source and sharing

Attribution: Shutong Feng, Nurul Lubis, Christian Geishauser, Hsien-chin Lin, Michael Heck, Carel van Niekerk, Milica Gasic.
Terms: CC-BY-NC-4.0. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.

Exact source locator and input fingerprint
{
  "source_locator": {
    "path": "data/prepared/emowoz.jsonl",
    "record": "34",
    "kind": "prepared_normalizer_record",
    "native_id": "emowoz/DMAGE1960.json/20/baseline"
  },
  "source_revision": "6506504",
  "input_hash": "c534d0f4316a3d6623aa22c25f187fc6ae0df2f9bb04035d5dbf5a1bc6f65f57"
}

Type to search questions, variants and documentation.