RaindropA practical guide to choosing decision modelsGitHub ↗

Concepts and categorization · question 49

Back to Concepts and categorization · All capabilities

Question ID: emowoz/3470f6607c3130371dc8
Source: EmoWOZ
Input format: native choices

Explore this question’s altered versions

Context

User: i am excited about seeing local tourist attractions. The attraction should be in the type of college.

Question

What emotion does the final user message express, taking the conversation into account?

Answer choices

  • 3: Apologetic
  • 6: Satisfied
  • 4: Abusive
  • 1: Fearful
  • 0: Neutral
  • 2: Dissatisfied
  • 5: Excited

Reference answer: Excited (5)

Exact evaluated input as JSON
{
  "state": "User: i am excited about seeing local tourist attractions. The attraction should be in the type of college.",
  "question": "What emotion does the final user message express, taking the conversation into account?",
  "choices": {
    "3": "Apologetic",
    "6": "Satisfied",
    "4": "Abusive",
    "1": "Fearful",
    "0": "Neutral",
    "2": "Dissatisfied",
    "5": "Excited"
  }
}

Recorded model answers

Model Answer Outcome
Astra Excited Correct
Opus 5.5 Excited Correct
GLiDE v1.2 Excited Correct
Decider v1.1 Excited Correct
Jev Excited Correct
Microsoft Decision-1 Excited Correct
Clef Excited Correct
Luna Decisions Excited Correct
Clef Flash Excited Correct
Strands Decider Excited Correct
GLiNER2.5 Decide Excited Correct
Laya Excited Correct
CalDec v1 Excited Correct

Source and sharing

Attribution: Shutong Feng, Nurul Lubis, Christian Geishauser, Hsien-chin Lin, Michael Heck, Carel van Niekerk, Milica Gasic.
Terms: CC-BY-NC-4.0. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.

Exact source locator and input fingerprint
{
  "source_locator": {
    "path": "data/prepared/emowoz.jsonl",
    "record": "9314",
    "kind": "prepared_normalizer_record",
    "native_id": "emowoz/DMAGE3808.json/0/baseline"
  },
  "source_revision": "6506504",
  "input_hash": "33251a0160ca7c79402fb80288b185b336df4758c13f88d8500fd94afd346f07"
}

Type to search questions, variants and documentation.