lnet_tutor/docs/multiple-choices-and-idk.md

# Multiple Choices and "I Don't Know" Option

## New Question Type: multiple_choices

### Purpose
Allows questions with multiple correct answers (e.g., "Select all that apply").

### Format
```json
{
  "id": "q1",
  "type": "multiple_choices",
  "prompt": "Which of the following are mutable data types in Python? (Select all that apply)",
  "choices": [
    { "key": "A", "text": "list" },
    { "key": "B", "text": "tuple" },
    { "key": "C", "text": "dict" },
    { "key": "D", "text": "str" },
    { "key": "E", "text": "set" }
  ],
  "answer": ["A", "C", "E"],
  "partialCredit": true,
  "allowIDK": true,
  "points": 10
}
```

### Scoring

**Partial Credit (default):**
```
Score = (Correct Selections / Total Correct Answers) × Points
```

Example:
- Correct answers: A, C, E (3 total)
- Student selects: A, C (2 correct)
- Score = (2/3) × 10 = 6.67 points

**All or Nothing:**
```json
"partialCredit": false
```
- Gets full points only if ALL correct answers selected
- Any mistake = 0 points

**With Wrong Selections:**
- Penalty for selecting incorrect options
- Score = max(0, (Correct - Wrong) / Total Correct × Points)

Example:
- Student selects: A, C, D (2 correct, 1 wrong)
- Score = (2 - 1) / 3 × 10 = 3.33 points

## "I Don't Know" Option

### Purpose
- Encourages honest assessment
- Prevents random guessing
- Better measures actual knowledge
- Can be scored differently (0 points vs penalty)

### Availability

**Automatically added to:**
- `single_choice` questions
- `multiple_choices` questions
- `true_false` questions

**Not added to:**
- `essay` questions (can leave blank)
- `code_simple` / `code_exercise` (can leave blank)

### Format

**Enable (default):**
```json
{
  "type": "single_choice",
  "allowIDK": true,
  "prompt": "What is...?",
  ...
}
```

**Disable:**
```json
{
  "type": "single_choice",
  "allowIDK": false,
  "prompt": "What is...?",
  ...
}
```

### UI Behavior

**Single Choice:**
```
○ A. Option A
○ B. Option B
○ C. Option C
○ ? I don't know
```

**Multiple Choices:**
```
☐ A. Option A
☐ B. Option B
☐ C. Option C
☐ ? I don't know (if checked, clears other selections)
```

**True/False:**
```
○ True
○ False
○ I don't know
```

### Scoring Rules

**"I don't know" selected:**
- Treated as incorrect (0 points)
- NOT penalized (better than guessing wrong)
- Honest indicator of knowledge gaps

**Comparison:**
- Wrong guess: 0 points + false confidence
- "I don't know": 0 points + honest gap identification

## Complete Examples

### Example 1: Single Choice with IDK

```json
{
  "id": "q1",
  "type": "single_choice",
  "prompt": "Which decorator is used for static methods?",
  "choices": [
    { "key": "A", "text": "@staticmethod" },
    { "key": "B", "text": "@classmethod" },
    { "key": "C", "text": "@property" }
  ],
  "answer": "A",
  "allowIDK": true,
  "points": 5
}
```

**Possible responses:**
- "A" → 5 points (correct)
- "B" or "C" → 0 points (incorrect)
- "IDK" → 0 points (honest)

### Example 2: Multiple Choices with Partial Credit

```json
{
  "id": "q2",
  "type": "multiple_choices",
  "prompt": "Which are valid Python keywords? (Select all that apply)",
  "choices": [
    { "key": "A", "text": "class" },
    { "key": "B", "text": "function" },
    { "key": "C", "text": "import" },
    { "key": "D", "text": "include" },
    { "key": "E", "text": "def" }
  ],
  "answer": ["A", "C", "E"],
  "partialCredit": true,
  "allowIDK": true,
  "points": 9
}
```

**Possible responses:**
- ["A", "C", "E"] → 9 points (all correct)
- ["A", "C"] → 6 points (2/3 correct with partial credit)
- ["A", "C", "D"] → 3 points (2 correct - 1 wrong)
- ["IDK"] → 0 points (honest)

### Example 3: True/False with IDK

```json
{
  "id": "q3",
  "type": "true_false",
  "prompt": "Python supports tail call optimization.",
  "answer": false,
  "allowIDK": true,
  "points": 3
}
```

**Possible responses:**
- false → 3 points (correct)
- true → 0 points (incorrect)
- "IDK" → 0 points (honest)

### Example 4: Disable IDK (Force Answer)

```json
{
  "id": "q4",
  "type": "single_choice",
  "prompt": "What is 2 + 2?",
  "choices": [
    { "key": "A", "text": "3" },
    { "key": "B", "text": "4" },
    { "key": "C", "text": "5" }
  ],
  "answer": "B",
  "allowIDK": false,
  "points": 2
}
```

**UI shows only A, B, C (no IDK option)**

## Answer Format

### Response for single_choice
```json
{
  "questionId": "q1",
  "response": "A"
}
// or
{
  "questionId": "q1",
  "response": "IDK"
}
```

### Response for multiple_choices
```json
{
  "questionId": "q2",
  "response": ["A", "C", "E"]
}
// or
{
  "questionId": "q2",
  "response": ["IDK"]
}
```

### Response for true_false
```json
{
  "questionId": "q3",
  "response": true
}
// or
{
  "questionId": "q3",
  "response": "IDK"
}
```

## Benefits

### For Learners:
- Honest self-assessment
- Identify knowledge gaps
- Avoid false confidence from lucky guesses
- Better learning outcome tracking

### For Instructors:
- Clearer picture of student knowledge
- Identify commonly unknown topics
- Better curriculum adjustment
- More accurate assessment

## Implementation Notes

- IDK option rendered in UI automatically when `allowIDK` is true
- Selecting IDK clears any other selections (for multiple_choices)
- Scoring treats IDK as incorrect (0 points)
- Analytics can track IDK rate per question
- High IDK rate = topic needs more coverage