Jim Ashe, PhD

Applied AI · LLM Evaluation · Measurement

Mathematician and 20-year educator building and evaluating applied AI systems. I work at the intersection of LLM evaluation, reasoning-data quality, and measurement — designing rubrics and objective quality gates, debugging retrieval, and thinking rigorously about how AI and education systems are measured.

Selected work

A few projects that show how I build and evaluate AI and data systems — with an emphasis on measurement, correctness, and responsible data handling.

Course FAQ Bot Framework

A reusable, config-driven framework for turning any course's materials into an evaluated, privacy-safe RAG assistant — with human-in-the-loop feedback tooling and an evaluation loop. Proven end-to-end on a university capstone.

RAGLLM evaluationprivacy-by-designPython

Metric-Validity Counterexamples

Monte-Carlo existence proofs of how a single aggregate metric can mislead across populations — a literal Simpson's paradox, throughput-driven metric deflation, and non-uniform optimal effort allocation, with symbolic verification.

measurementstatisticsevaluationPython

CaseloadNotes

A Windows desktop app that automates the repetitive Salesforce/Outlook workflow of running a course caseload — batch actions across filtered students, templated email/text, at-rest encryption, a test suite, and a packaged build.

applied softwarePythondesktopautomation

CirclePack · contributor

Open-source research software for the creation, manipulation, and display of circle packings (in the sense of Thurston) — tied to my published mathematics on generalized branching.

math researchJavaopen source

Background

  • PhD, Mathematics — University of Tennessee (advisor: Ken Stephenson).
  • Published in The Journal of Analysis · arXiv:1607.03404 · Google Scholar.
  • 20+ years teaching mathematics, computer science, and data analytics to a large, diverse online student body.
  • Authored competency-based performance assessments — with evidence-to-objective mapping, separable rubrics, and objective quality gates — for WGU's BS in AI Engineering.
  • Focused on model evaluation, reasoning-data design, and assessment. Always happy to talk shop about eval methodology.