skill-comply

Use it when:: You've written a skill or rule and want to know whether the agent follows it, rather than assume it does.
It does:: Derives the expected steps from the file, writes scenarios at three prompt strictness levels, runs each with claude -p and classifies the tool calls.
You get:: A Markdown report with the spec, the prompts, a compliance score per scenario and a labelled timeline of tool calls.
Needs:: Claude Code and uv; full runs spend tokens on every scenario, while a dry run stops after the cheaper model writes the spec and scenarios.
Works on:: SKILL.md files, rule files and agent definitions, the last only for whether the agent gets invoked when expected.

Runs Claude Code headless once per scenario, working in a temporary folder with Bash and file edits pre-approved (nothing confines Bash to that folder), which spends tokens. Needs uv.

TL;DR

skill-comply tests instructions the way you'd test code. Given a skill, rule or agent file, its bundled Python script has a cheaper model, Haiku by default, extract the expected sequence of steps, then write scenarios whose prompts go from supportive to neutral to competing, so you can see whether the instruction holds even when the prompt doesn't back it. Each scenario runs in a fresh sandbox folder through Claude Code's headless mode, Sonnet by default, with its tool calls captured as stream-json. An LLM, not a regex, matches the calls to the expected steps, and their order is checked deterministically. The report holds the spec, the scenario prompts, each scenario's compliance score and the tool-call timeline with labels, and for steps with low compliance it suggests turning them into hooks. A --dry-run flag prints the spec and scenarios and skips the scenario runs, so only the generation model is called, and --model and --gen-model pick the models.

Summarizes the skill at d2ef268.

npx skills add https://github.com/affaan-m/ecc --skill skill-comply

skills.sh page Source on GitHub All skills from affaan-m/ecc

Total Downloads: 5,400 on skills.sh, Nov 9, 2026
Repo stars: 299,776 affaan-m/ecc
First seen: May 19, 2026 on skills.sh

Weekly Downloads

443

+73% over the 8 weeks to Nov 9

Weekly values

Week to	Installs
Sep 21	256
Sep 28	166
Oct 5	144
Oct 12	229
Oct 19	416
Oct 26	403
Nov 2	483
Nov 9	443

Category	category:skill-tools category:software-engineering Find, write, vet and route to other skills. Everything for building software: planning, coding, debugging, design, frameworks, cloud, agents and skills.
Type	type:tool Operates a CLI, API, app or bundled script to get something done.
Effect	effect:local Its own steps create or change files, git state or local tools.
Needs	needs:cli needs:claude-code A specific command-line tool or desktop app must be installed, such as gh, agent-browser, LibreOffice, Obsidian or a vendor CLI. Uses Claude Code features such as hooks or background agents.
Publisher	by:community Everyone else. Unverified vendor claims default to community.
Topics	skill-authoring testing verification
Stack	stack:claude

SKILL.md

From affaan-m/ecc at d2ef268. Checked daily, last changed Nov 6, 2026.

Frontmatter

name: skill-comply
description: Visualize whether skills, rules, and agent definitions are actually followed — auto-generates scenarios at 3 prompt strictness levels, runs agents, classifies behavioral sequences, and reports compliance rates with full tool call timelines. Use when checking whether agents actually follow the skills, rules, and definitions they were given, rather than assuming they do.
metadata:
  origin: ECC
tools: Read, Bash

skill-comply: Automated Compliance Measurement

Measures whether coding agents actually follow skills, rules, or agent definitions by:

Auto-generating expected behavioral sequences (specs) from any .md file
Auto-generating scenarios with decreasing prompt strictness (supportive → neutral → competing)
Running claude -p and capturing tool call traces via stream-json
Classifying tool calls against spec steps using LLM (not regex)
Checking temporal ordering deterministically
Generating self-contained reports with spec, prompts, and timelines

Supported Targets

Skills (skills/*/SKILL.md): Workflow skills like search-first, TDD guides
Rules (rules/common/*.md): Mandatory rules like testing.md, security.md, git-workflow.md
Agent definitions (agents/*.md): Whether an agent gets invoked when expected (internal workflow verification not yet supported)

When to Activate

User runs /skill-comply <path>
User asks "is this rule actually being followed?"
After adding new rules/skills, to verify agent compliance
Periodically as part of quality maintenance

Usage

# Full run
uv run python -m scripts.run ~/.claude/rules/common/testing.md

# Dry run (no cost, spec + scenarios only)
uv run python -m scripts.run --dry-run ~/.claude/skills/search-first/SKILL.md

# Custom models
uv run python -m scripts.run --gen-model haiku --model sonnet <path>

Key Concept: Prompt Independence

Measures whether a skill/rule is followed even when the prompt doesn't explicitly support it.

Report Contents

Reports are self-contained and include:

Expected behavioral sequence (auto-generated spec)
Scenario prompts (what was asked at each strictness level)
Compliance scores per scenario
Tool call timelines with LLM classification labels

Advanced (optional)

For users familiar with hooks, reports also include hook promotion recommendations for steps with low compliance. This is informational — the main value is the compliance visibility itself.