Failure Taxonomy

model parse_status error_category count ratio_within_model
gemma3:4b success other 1335 0.9933035714285714
gemma3:4b failed invalid_or_missing_selected_article_id 9 0.006696428571428571
gemma4:latest success other 1247 0.999198717948718
gemma4:latest failed invalid_or_missing_selected_article_id 1 0.0008012820512820513
llama3.2:3b success other 1249 0.9293154761904762
llama3.2:3b failed invalid_or_missing_selected_article_id 95 0.07068452380952381
mistral:latest success other 1243 0.9959935897435898
mistral:latest failed invalid_or_missing_selected_article_id 5 0.004006410256410256
phi4-mini:3.8b success other 1298 0.9657738095238095
phi4-mini:3.8b fallback fallback_after_malformed_json 32 0.023809523809523808
phi4-mini:3.8b failed invalid_or_missing_selected_article_id 14 0.010416666666666666
qwen2.5:7b success other 1343 0.9992559523809523
qwen2.5:7b failed other 1 0.000744047619047619
qwen3:8b success other 1343 0.9992559523809523
qwen3:8b failed invalid_or_missing_selected_article_id 1 0.000744047619047619