01
AI6130 · Large Language Models
Evaluating Small LLMs on Multiple-Choice QA
Individual assignment
Scored four open models (1.1B to 4B parameters) on CommonsenseQA, OpenbookQA and PIQA with baseline, zero-shot and few-shot prompts. Prompt format mattered more than examples: a clear zero-shot prompt lifted Qwen2.5-3B from 32% to 80%, and few-shot added nothing.
- Qwen2.5-3B average accuracy, baseline vs zero-shot prompt
- 32% → 80%
- models, benchmarks and prompt settings evaluated
- 4 × 3 × 3
