Adaptive Tutor with Bayesian Knowledge Tracing and LLM-Generated Practice

Solution for the use case Personalized Learning AI Tutor Platform.

AI tools
Python, Bayesian Knowledge Tracing, OpenAI Python SDK, OpenAI API (gpt-6-luna), Ollama (optional, local)
Language
Python
Published
2026-10-26

Approach

A Python tutor that tracks a mastery estimate per skill and decides what to practise next, with an LLM writing the questions and explanations.

  • Skill map: a curriculum JSON with skills and prerequisites (example: fractions).
  • Mastery tracking: Bayesian Knowledge Tracing updates the probability that a skill is mastered after every answer, using slip and guess probabilities.
  • Adaptive practice: the tutor picks the least-mastered skill whose prerequisites are mastered and asks for easy, medium or hard questions depending on mastery.
  • LLM via an OpenAI-compatible API: generates questions with short answers and hints, and explains wrong answers step by step. Grading itself is done locally in code.

Implementation steps

  1. Describe the subject as skills with prerequisites in a curriculum JSON and validate the references
  2. Implement the BKT update: Bayes step with slip and guess, then the learning transition
  3. Choose the next skill: the least-mastered one whose prerequisites have reached 0.95
  4. Ask an OpenAI-compatible chat model for a question, answer and hint as JSON, with difficulty set by mastery and recent questions excluded, retrying once on invalid output
  5. Grade the learner's answer locally, comparing numbers and fractions by value
  6. Update mastery and attempts, and on a wrong answer request a step-by-step explanation
  7. Save learner state to a JSON file after every answer
  8. Test offline: BKT against hand calculations, skill selection, grading and the tutor loop with a fake chat client

Code (Python)

"""The tutoring loop: pick a skill, ask a generated question, grade locally, update mastery."""
import json
from dataclasses import dataclass, field
from pathlib import Path

from . import bkt
from .curriculum import Curriculum, Skill
from .grading import is_correct
from .llm import Question, TutorLLM


@dataclass
class LearnerState:
    mastery: dict[str, float] = field(default_factory=dict)
    attempts: dict[str, int] = field(default_factory=dict)
    recent_questions: list[str] = field(default_factory=list)

    @classmethod
    def load(cls, path: Path) -> "LearnerState":
        return cls(**json.loads(path.read_text(encoding="utf-8"))) if path.exists() else cls()

    def save(self, path: Path) -> None:
        path.write_text(json.dumps(self.__dict__, indent=2), encoding="utf-8")


@dataclass
class Feedback:
    correct: bool
    mastery_before: float
    mastery_after: float
    explanation: str | None = None


class Tutor:
    def __init__(self, curriculum: Curriculum, llm: TutorLLM, state: LearnerState | None = None,
                 params: bkt.BKTParams = bkt.DEFAULT_PARAMS):
        self.curriculum, self.llm, self.params = curriculum, llm, params
        self.state = state or LearnerState()

    def mastery(self, skill_id: str) -> float:
        return self.state.mastery.get(skill_id, self.params.p_init)

    def next_skill(self) -> Skill | None:
        known = {sid: self.mastery(sid) for sid in self.curriculum.skills}
        return self.curriculum.next_skill(known)

    def ask(self, skill: Skill) -> Question:
        question = self.llm.question(skill, self.curriculum.subject, self.mastery(skill.id),
                                     self.state.recent_questions)
        self.state.recent_questions = (self.state.recent_questions + [question.text])[-20:]
        return question

    def submit(self, question: Question, answer: str) -> Feedback:
        before = self.mastery(question.skill_id)
        correct = is_correct(answer, question.answer)
        after = bkt.update(before, correct, self.params)
        self.state.mastery[question.skill_id] = round(after, 4)
        self.state.attempts[question.skill_id] = self.state.attempts.get(question.skill_id, 0) + 1
        explanation = None
        if not correct:
            explanation = self.llm.explain(self.curriculum.skills[question.skill_id], question, answer)
        return Feedback(correct, before, after, explanation)

Prompts

You write one practice question for the skill "Adding fractions with different denominators" in the subject "Fractions (grade 5 to 6 level)". Difficulty: medium, two steps. The answer must be short and unambiguous (a number, a fraction like 3/4, or one word). Avoid these recent questions: [...]. Answer with JSON only: {"question": str, "answer": str, "hint": str}.

Performance

Expectations, not measurements:

  • With the default parameters, three correct answers in a row take a skill from 0.2 to above 0.95; wrong answers lower the estimate.
  • The default BKT parameters are not fitted to data; fit them per skill on real answer logs before relying on the mastery numbers.
  • One chat request per question plus one per wrong answer.

Lessons learned

The learner answers one question at a time in the terminal and sees the skill and current mastery, can ask for a hint, and gets an immediate verdict. After a mistake the tutor explains the steps and the likely misconception. Mastery moves visibly (for example 60% to 89%), and the tutor moves on to new skills only when the prerequisites are mastered. Progress is saved between sessions.

Back to the use case · All AI use cases