Stop Guessing Why Your Prompt Isn't Working
PromptLab scores a prompt across 12 dimensions, suggests improved variants using distinct strategies, and compares the outputs of the original and the variants on test cases it generates from the prompt. The comparison is a starting point for your own review, not proof of how a prompt will perform in real use.
Why I built it
Three problems I kept hitting
The problem
- ❌A rewritten prompt with no explanation of what was wrong with the original
- ❌Evaluation set-ups that assume a labelled dataset I didn't have
- ❌No record of which variants I had tried, or how they compared
What PromptLab does
- ✅Scores each dimension with a rationale for why it is weak and a suggestion
- ✅Generates test cases from the prompt, so variants can be compared without a dataset (a first pass, not a substitute for testing on real inputs)
- ✅CLI-first, local-first — sessions saved, history browsable, works offline
Recorded example
What the analysis looks like
A recorded example of promptlab analyse on a weak prompt, shown as static text. Nothing runs on this page.
$ promptlab analyse "You are a helpful assistant. Answer questions."
Analysing prompt across 12 dimensions...
Overall Score: 1.75 / 5.0 ──────────────────── NEEDS WORK
6 critical issues · 4 low · 1 medium · 1 good
→ Run: promptlab improve "You are a helpful assistant..." --test
On this page the overall score is the unweighted mean of the 12 dimension scores (21 ÷ 12 = 1.75), and a dimension is Critical at 1/5, Low at 2/5, Medium at 3/5 and Good at 4/5 or above. The counts are taken from the rows above.
How It Works
Three Commands. End to End.
Analyse
promptlab analyse "your prompt"Runs a structured diagnostic across 12 dimensions. Each gets a score from 1–5, a rationale for why it's weak, and an actionable suggestion. Critical issues are flagged immediately.
Improve
promptlab improve "your prompt"Generates 3 improved variants using distinct strategies — structured enhancement, role & context expansion, and few-shot augmentation. Not random rewrites. Each change is explained.
Compare
promptlab improve "your prompt" --testGenerates test cases from your prompt, runs the original and the 3 variants against them, and scores each output with its reasoning. Use the result to decide what to review by hand: generated test cases and model-graded scores are not proof of real-world quality.
Diagnostic Framework
The 12 Dimensions
Every prompt is scored 1–5 across these dimensions.
Role Definition
Does the prompt define a clear expert persona?
Task Clarity
Is the primary task unambiguous and single-focused?
Output Format
Does it specify the desired structure and format?
Input Specification
Does it describe what inputs to expect?
Constraints
Are restrictions and limits stated explicitly?
Examples
Are few-shot examples provided to guide output style?
Tone & Style
Is the desired register and voice specified?
Edge Cases
Does it handle unexpected or ambiguous inputs?
Reasoning
Is chain-of-thought or step-by-step thinking instructed?
Context Management
Is the prompt self-contained with all context?
Specificity Balance
Specific enough without over-constraining?
Token Efficiency
Concise and free of redundant instructions?
Stack
Built With
It's open source. Use it, break it, improve it.
Built in spare time as a genuine tool I use for my own prompts. PRs welcome — especially new providers and diagnostic dimensions.