Shikhu: Turn Your Codebase into Self-Tests and Track Your Knowledge Coverage

Who it is for

You have inherited a codebase you can never finish reading, or someone on the team keeps asking "who actually uses this part?". If your goal is to have an AI coding agent practice repeatedly on your own code and to turn "how much do I actually understand?" into a number you can track, Shikhu's approach fits better than one-off Q&A: it turns learning into question generation and answering, the tool draws questions from the repository and records which ones you got right and wrong, and the companion skills walk you through the key points in the files you name. It suits anyone willing to let an agent run a few rounds of practice on real code and to accept that "the model will read your code."

Your first attempt in 3 minutes

There is only one official path, and it starts with installing the command-line tool:

uv tool install shikhu

After installation, shikhu is on your PATH and runs in an isolated environment, so it does not clash with the dependencies in your project.

There are two prerequisites: first, uv in your environment; second, an OpenRouter API key -- put it in a .env at the repository root (loaded automatically), or export OPENROUTER_API_KEY in your shell, because every model call goes through OpenRouter.

Then run this at the root of the code you want to learn:

shikhu init

This step really does write files, so confirm before you run it: it writes a local database coverage.db (added to .gitignore automatically), generates .quizignore (used like .gitignore, to keep configuration or documentation files out of the question pool), and installs the two skills into your agent's skills directory.

Install location: the skills land in .claude/skills/ and/or .agents/skills/ if those directories already exist in the repository (depending on which family your agent belongs to); if neither exists, both are created. To reinstall later or install globally, use shikhu install-skill, adding --global for the global scope.

For the first trigger, pick something read-only and reversible: in a skill-capable coding agent, use /shikhu-study with a small file, or use /shikhu-inquiry to ask a question that touches only a single function. Before you ask, confirm that this question will bring an external model call (through OpenRouter, consuming quota), and send it only once you have confirmed.

There are three observable success signals: the agent's answers contain file names, function names, and call relationships that genuinely exist in your repository instead of generic tutorial prose; questions you have asked turn into quiz seeds and can come up again later when new questions are generated; and coverage.db and .quizignore appear at the project root.

Use cases

Case one: targeted self-study of a module.

Build the index first:

shikhu summarize

It writes a model-generated cached summary for every tracked file; one model call per file, and files whose contents have not changed are not processed again. The skills and shikhu map rely on these summaries to work out "which files should I be looking at for this question?". Once the index is ready, use /shikhu-study in the agent with the file or directory you want to learn, and the agent walks you through them one by one, following the locations the summaries point to.

Case two: turning everyday questions into quiz seeds.

In the agent, use /shikhu-inquiry to ask a question about your own code -- for example, where the parameters of an endpoint come from, or why a piece of logic is written the way it is. It answers, and at the same time records the question as a quiz seed; later, when the tool generates quizzes from the code, those seeds take part, and whether you answer correctly is tracked as knowledge coverage. If you want the questions to stay focused on real code, use .quizignore to exclude configuration and documentation files.

Limits

• You need to bring your own OpenRouter API key; without it, no step that involves model generation can be done.
• shikhu summarize is billed per file, so the bigger the repository, the slower and more expensive the first indexing run is; only files whose contents changed are processed again.
• Where the skills land depends on which skill directories already exist in the repository; if neither exists, both are created at the same time.
• coverage.db is written inside the project but is added to .gitignore automatically, so it is not synced to version control, and coverage and answer history are not shared across the team.
• The source explains how to use the command-line tool and the two skills, their commands, and their install locations; but it does not state which programming languages are supported, exactly how coverage is computed, or how the difficulty tiers are determined, so do not infer them from how ordinary testing tools behave.