first working version
This commit is contained in:
howard
284
docs/multiple-choices-and-idk.md
Normal file
284
docs/multiple-choices-and-idk.md
Normal file
@@ -0,0 +1,284 @@
|
||||
# Multiple Choices and "I Don't Know" Option
|
||||
|
||||
## New Question Type: multiple_choices
|
||||
|
||||
### Purpose
|
||||
Allows questions with multiple correct answers (e.g., "Select all that apply").
|
||||
|
||||
### Format
|
||||
```json
|
||||
{
|
||||
"id": "q1",
|
||||
"type": "multiple_choices",
|
||||
"prompt": "Which of the following are mutable data types in Python? (Select all that apply)",
|
||||
"choices": [
|
||||
{ "key": "A", "text": "list" },
|
||||
{ "key": "B", "text": "tuple" },
|
||||
{ "key": "C", "text": "dict" },
|
||||
{ "key": "D", "text": "str" },
|
||||
{ "key": "E", "text": "set" }
|
||||
],
|
||||
"answer": ["A", "C", "E"],
|
||||
"partialCredit": true,
|
||||
"allowIDK": true,
|
||||
"points": 10
|
||||
}
|
||||
```
|
||||
|
||||
### Scoring
|
||||
|
||||
**Partial Credit (default):**
|
||||
```
|
||||
Score = (Correct Selections / Total Correct Answers) × Points
|
||||
```
|
||||
|
||||
Example:
|
||||
- Correct answers: A, C, E (3 total)
|
||||
- Student selects: A, C (2 correct)
|
||||
- Score = (2/3) × 10 = 6.67 points
|
||||
|
||||
**All or Nothing:**
|
||||
```json
|
||||
"partialCredit": false
|
||||
```
|
||||
- Gets full points only if ALL correct answers selected
|
||||
- Any mistake = 0 points
|
||||
|
||||
**With Wrong Selections:**
|
||||
- Penalty for selecting incorrect options
|
||||
- Score = max(0, (Correct - Wrong) / Total Correct × Points)
|
||||
|
||||
Example:
|
||||
- Student selects: A, C, D (2 correct, 1 wrong)
|
||||
- Score = (2 - 1) / 3 × 10 = 3.33 points
|
||||
|
||||
## "I Don't Know" Option
|
||||
|
||||
### Purpose
|
||||
- Encourages honest assessment
|
||||
- Prevents random guessing
|
||||
- Better measures actual knowledge
|
||||
- Can be scored differently (0 points vs penalty)
|
||||
|
||||
### Availability
|
||||
|
||||
**Automatically added to:**
|
||||
- `single_choice` questions
|
||||
- `multiple_choices` questions
|
||||
- `true_false` questions
|
||||
|
||||
**Not added to:**
|
||||
- `essay` questions (can leave blank)
|
||||
- `code_simple` / `code_exercise` (can leave blank)
|
||||
|
||||
### Format
|
||||
|
||||
**Enable (default):**
|
||||
```json
|
||||
{
|
||||
"type": "single_choice",
|
||||
"allowIDK": true,
|
||||
"prompt": "What is...?",
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
**Disable:**
|
||||
```json
|
||||
{
|
||||
"type": "single_choice",
|
||||
"allowIDK": false,
|
||||
"prompt": "What is...?",
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
### UI Behavior
|
||||
|
||||
**Single Choice:**
|
||||
```
|
||||
○ A. Option A
|
||||
○ B. Option B
|
||||
○ C. Option C
|
||||
○ ? I don't know
|
||||
```
|
||||
|
||||
**Multiple Choices:**
|
||||
```
|
||||
☐ A. Option A
|
||||
☐ B. Option B
|
||||
☐ C. Option C
|
||||
☐ ? I don't know (if checked, clears other selections)
|
||||
```
|
||||
|
||||
**True/False:**
|
||||
```
|
||||
○ True
|
||||
○ False
|
||||
○ I don't know
|
||||
```
|
||||
|
||||
### Scoring Rules
|
||||
|
||||
**"I don't know" selected:**
|
||||
- Treated as incorrect (0 points)
|
||||
- NOT penalized (better than guessing wrong)
|
||||
- Honest indicator of knowledge gaps
|
||||
|
||||
**Comparison:**
|
||||
- Wrong guess: 0 points + false confidence
|
||||
- "I don't know": 0 points + honest gap identification
|
||||
|
||||
## Complete Examples
|
||||
|
||||
### Example 1: Single Choice with IDK
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "q1",
|
||||
"type": "single_choice",
|
||||
"prompt": "Which decorator is used for static methods?",
|
||||
"choices": [
|
||||
{ "key": "A", "text": "@staticmethod" },
|
||||
{ "key": "B", "text": "@classmethod" },
|
||||
{ "key": "C", "text": "@property" }
|
||||
],
|
||||
"answer": "A",
|
||||
"allowIDK": true,
|
||||
"points": 5
|
||||
}
|
||||
```
|
||||
|
||||
**Possible responses:**
|
||||
- "A" → 5 points (correct)
|
||||
- "B" or "C" → 0 points (incorrect)
|
||||
- "IDK" → 0 points (honest)
|
||||
|
||||
### Example 2: Multiple Choices with Partial Credit
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "q2",
|
||||
"type": "multiple_choices",
|
||||
"prompt": "Which are valid Python keywords? (Select all that apply)",
|
||||
"choices": [
|
||||
{ "key": "A", "text": "class" },
|
||||
{ "key": "B", "text": "function" },
|
||||
{ "key": "C", "text": "import" },
|
||||
{ "key": "D", "text": "include" },
|
||||
{ "key": "E", "text": "def" }
|
||||
],
|
||||
"answer": ["A", "C", "E"],
|
||||
"partialCredit": true,
|
||||
"allowIDK": true,
|
||||
"points": 9
|
||||
}
|
||||
```
|
||||
|
||||
**Possible responses:**
|
||||
- ["A", "C", "E"] → 9 points (all correct)
|
||||
- ["A", "C"] → 6 points (2/3 correct with partial credit)
|
||||
- ["A", "C", "D"] → 3 points (2 correct - 1 wrong)
|
||||
- ["IDK"] → 0 points (honest)
|
||||
|
||||
### Example 3: True/False with IDK
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "q3",
|
||||
"type": "true_false",
|
||||
"prompt": "Python supports tail call optimization.",
|
||||
"answer": false,
|
||||
"allowIDK": true,
|
||||
"points": 3
|
||||
}
|
||||
```
|
||||
|
||||
**Possible responses:**
|
||||
- false → 3 points (correct)
|
||||
- true → 0 points (incorrect)
|
||||
- "IDK" → 0 points (honest)
|
||||
|
||||
### Example 4: Disable IDK (Force Answer)
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "q4",
|
||||
"type": "single_choice",
|
||||
"prompt": "What is 2 + 2?",
|
||||
"choices": [
|
||||
{ "key": "A", "text": "3" },
|
||||
{ "key": "B", "text": "4" },
|
||||
{ "key": "C", "text": "5" }
|
||||
],
|
||||
"answer": "B",
|
||||
"allowIDK": false,
|
||||
"points": 2
|
||||
}
|
||||
```
|
||||
|
||||
**UI shows only A, B, C (no IDK option)**
|
||||
|
||||
## Answer Format
|
||||
|
||||
### Response for single_choice
|
||||
```json
|
||||
{
|
||||
"questionId": "q1",
|
||||
"response": "A"
|
||||
}
|
||||
// or
|
||||
{
|
||||
"questionId": "q1",
|
||||
"response": "IDK"
|
||||
}
|
||||
```
|
||||
|
||||
### Response for multiple_choices
|
||||
```json
|
||||
{
|
||||
"questionId": "q2",
|
||||
"response": ["A", "C", "E"]
|
||||
}
|
||||
// or
|
||||
{
|
||||
"questionId": "q2",
|
||||
"response": ["IDK"]
|
||||
}
|
||||
```
|
||||
|
||||
### Response for true_false
|
||||
```json
|
||||
{
|
||||
"questionId": "q3",
|
||||
"response": true
|
||||
}
|
||||
// or
|
||||
{
|
||||
"questionId": "q3",
|
||||
"response": "IDK"
|
||||
}
|
||||
```
|
||||
|
||||
## Benefits
|
||||
|
||||
### For Learners:
|
||||
- Honest self-assessment
|
||||
- Identify knowledge gaps
|
||||
- Avoid false confidence from lucky guesses
|
||||
- Better learning outcome tracking
|
||||
|
||||
### For Instructors:
|
||||
- Clearer picture of student knowledge
|
||||
- Identify commonly unknown topics
|
||||
- Better curriculum adjustment
|
||||
- More accurate assessment
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- IDK option rendered in UI automatically when `allowIDK` is true
|
||||
- Selecting IDK clears any other selections (for multiple_choices)
|
||||
- Scoring treats IDK as incorrect (0 points)
|
||||
- Analytics can track IDK rate per question
|
||||
- High IDK rate = topic needs more coverage
|
||||
|
||||
Reference in New Issue
Block a user