Refideas Analyzer

A Python tool for analyzing Wikipedia articles that use the Refideas template to understand what types of reference sources are suggested for improving articles.

Template:Refideas
Used on: ~29,000 Wikipedia talk pages
Purpose: Suggest reference sources that editors could use to improve articles

Template:Refideas (also known as Refidea, RI, Suggested Sources, and 8 other aliases) is a Wikipedia template placed on article talk pages (the discussion pages associated with articles). Its purpose is to suggest reference sources that editors could use to improve the article.

When an editor adds {{Refideas}} to a talk page, they can list:

• URLs to useful articles or websites
• Links to archived copies of lost content
• Book references
• Academic papers
• News articles
• Citation templates
• And more

This template helps coordinate research efforts by suggesting where to find reliable sources. The template is used on approximately 29,000 Wikipedia talk pages across all topics—from music and video games to sports, biography, and science.

When placed on a talk page, the template renders like this:

📚 The following reference(s) may be useful when improving this article in the future:

• [News article]. Example.com.
• [Another news article]. Example.com.

On a talk page, an editor might add:

{{Refideas
|1=https://www.example.com/article News article
|2=https://www.example.com/journal Academic paper about the topic
|f1=https://archive.org/details/public-domain-book Public domain book
}}

This produces a formatted box listing the suggested references for improving the article.

Refideas Analyzer fetches a random sample of Wikipedia talk pages that use Template:Refideas (and its 11 aliases), extracts the reference suggestions, classifies them by type (URL, archive, citation template, news, etc.), and produces detailed reports.

Results from 380-page sample:

• 1,395 total reference entries analyzed
• Average of 3.7 references per article
• 51.9% URLs, 25.9% Archives, 15.6% Citation Templates

# Create virtual environment
uv venv --python 3.11
source .venv/bin/activate

# Install dependencies
uv pip install requests mwparserfromhell

# Run with default sample size (100 pages)
.venv/bin/python refideas_analyzer.py

Edit SAMPLE_SIZE in refideas_analyzer.py to change sample size:

SAMPLE_SIZE = 100  # Options: 10, 100, 380+ (tested)

The script generates timestamped output files (no overwrite):

• refideas_analysis_YYYYMMDD_HHMMSS.csv - Aggregated statistics
• refideas_report_YYYYMMDD_HHMMSS.txt - Detailed per-article reports
• cache_Template_Refideas.json - Cached page list (~28K pages)

• Template Parsing: Uses mwparserfromhell for reliable wikitext parsing
• Multiple Aliases: Handles all 11 template variants (Refideas, RI, Suggested sources, etc.)
• Reference Classification: 12 categories (URL, Archive, Citation Template, News, Book, Academic Journal, etc.)
• Topic Classification: Uses Lift Wing API for article topics
• Error Detection: Identifies malformed template usage (bullet list syntax)
• Caching: Page list cached to avoid repeated API calls

Total articles analyzed: 380
Total reference entries: 1,395
Average refs per article: 3.7
Template errors found: 54

--- Reference Type Distribution ---
  url                   :   724 ( 51.9%)
  archive               :   362 ( 25.9%)
  citation_template    :   218 ( 15.6%)
  news_organization   :    25 (  1.8%)
  other                :    20 (  1.4%)
  book                 :    14 (  1.0%)
  social_media         :    13 (  0.9%)
  wiki_article         :    13 (  0.9%)
  academic_journal     :     5 (  0.4%)
  government           :     1 (  0.1%)

--- Top 20 Domains (from URLs) ---
  punknews.org               :    47   (music/punk rock news)
  allmusic.com              :    28   (music reviews)
  thegazette.news           :    20   (newspaper)
  articles.mcall.com        :    18   (Allentown Morning Call)
  altpress.com               :    16   (alternative press)
  mmegi.bw                  :    14   (Botswana newspaper)
  mtv.com                   :    11   (music/entertainment)
  books.google.co.uk        :    10   (Google Books UK)
  polygon.com               :     9   (gaming)
  sundaystandard.info       :     9   (Botswana newspaper)
  ign.com                   :     8   (gaming)
  sportskeeda.com           :     8   (sports)
  exclaim.ca                :     7   (Canadian music)
  timeextension.com        :     7   (retro gaming)
  laut.de                   :     7   (German music)
  ftbl.com.au               :     7   (Australian football)
  snl24.com                 :     7   (news)
  newspapers.com            :     6   (newspaper archives)
  gamespot.com              :     6   (gaming)
  kotaku.com                :     6   (gaming)

--- Top 20 Article Topics ---
  Culture.Media.Media*            :    81
  Culture.Media.Music              :    66
  Culture.Internet_culture         :    54
  Culture.Media.Video_Games        :    44
  Culture.Sports                   :    19
  Culture.Biography.Biography*     :    18
  Culture.Literature               :    13
  STEM.STEM*                       :     9
  Geography.Regions.Americas.North_America:     8
  Culture.Visual_arts.Comics_and_Anime         :     7
  Culture.Visual_arts.Visual_arts*             :     5
  Geography.Regions.Europe.Europe*              :     5
  Geography.Regions.Asia.East_Asia              :     5
  Geography.Regions.Asia.Southeast_Asia          :     4
  Geography.Regions.Oceania                      :     4
  History_and_Society.Politics_and_government     :     3
  Culture.Media.Television                       :     3
  STEM.Biology                                   :     3
  Geography.Regions.Asia.West_Asia               :     2
  Culture.Philosophy_and_religion                :     2

This tool was "vibe coded" using OpenCode with the MiniMax M2.5 Free model. The development process evolved through two main modes:

• The user (a Wikipedia expert) described the nature of the Refideas template and its approximately 29,000 uses
• Together, we formed a strategy to understand how it's used and what sample size would be needed to build a statistically meaningful profile
• Discussed desired output format: per-article reports, aggregated statistics, reference type distribution, domain analysis, and article topic classification

We prototyped and tested several approaches, hitting various roadblocks:

1. Initial Parsing Issues - First attempts using regex to parse wikitext failed to handle complex nested templates. The user recommended mwparserfromhell, which proved essential for reliable parsing.

2. Topic Classification - The user suggested using the Lift Wing API for article topic classification, which provides useful categorical data (Culture.Media.Music, STEM.Biology, etc.).

3. Template Variants - The template appeared under many names. The user directed us to Wikipedia's Special:WhatLinksHere page, which revealed all 11 aliases (Refideas, Refidea, RI, Suggested sources, Ref ideas, Source ideas, Potential sources, Possible sources, Suggested refs, Refideas-nonotice, Refsuggestion).

4. Parameter Format Diversity - Users enter references in many ways: numbered parameters (|1=, |2=), URLs directly, citation templates ({{cite journal}}), bullet list syntax, and more. Each required different handling.

5. API Throttling - Wikipedia enforces strict rate limits. We implemented retry logic with exponential backoff, added delays between calls, and discovered that sample sizes of ~100-380 work reliably while 400+ may timeout.

6. Report Refinement - Added per-article detail reports, error detection for malformed templates (e.g., bullet list syntax instead of proper parameters), and timestamped output files to prevent overwriting previous runs.

• Random sampling approach (rather than full dataset) to balance statistical validity with API constraints
• Cache the ~28K page list to avoid repeated 5-minute fetches on subsequent runs
• 12-category reference classification system based on domain patterns
• Include template error detection as a "linting" feature

• refideas_analyzer.py - Main analysis script
• PRD.md - Product Requirements Document
• SKILLS.md - Technical skills and lessons learned
• README.md - This file

• Python 3.11+
• requests
• mwparserfromhell

MIT License

• mwparserfromhell - Excellent wikitext parsing
• Wikipedia API - Access to Wikipedia data
• Lift Wing API - Article topic classification