coverage-analysis
verified075f4124-f079-4474-9571-2cb002228583
Use when assessing test coverage — generate reports with pytest-cov, read missing-line output, and prioritize gaps by risk (error paths, auth, money) over raw percentages.
Metadata
Skill file
# Coverage Analysis
**Use when** you need to assess test coverage, identify untested code paths, and prioritize which gaps to close. Coverage is a tool for finding *missing* tests, not a score to maximize.
## Generating a Coverage Report
```bash
# Basic coverage report with missing lines
pytest --cov=src/ --cov-report=term-missing -x
# Output per file:
# src/orders/processor.py 45 12 73% 34-38, 52-55, 67-70
# lines missed % missing line ranges
```
### Generating HTML for Visual Drill-Down
```bash
pytest --cov=src/ --cov-report=html
# Open htmlcov/index.html in a browser
# Click any file to see green (covered) / red (uncovered) lines
```
### Strict Mode (CI)
```bash
# Fail if coverage drops below threshold
pytest --cov=src/ --cov-fail-under=80 --cov-report=term
```
## Reading the Missing-Line Report
```text
src/orders/processor.py 120 35 70% 14-18, 42, 55-60, 88-102
```
Interpretation:
- **14-18**: Early return / validation — HIGH priority (error handling)
- **42**: Single uncovered `if` branch — MEDIUM (probably a real code path)
- **55-60**: Exception handler — HIGH priority (error paths)
- **88-102**: A 15-line uncovered block — Check if dead code or missing feature test
## Prioritizing Gaps: Risk Over Percentage
Coverage percentage is a lagging indicator. Prioritize by *risk*:
### High-Priority Gaps (fix these first)
```python
# Error handlers
try:
process_payment(...)
except PaymentError:
refund(...) # <-- UNCOVERED (high risk)
log_failure(...) # <-- UNCOVERED (high risk)
# Auth / permission checks
if not user.is_admin:
raise Forbidden() # <-- UNCOVERED (security)
# Money / billing
if amount > limit:
require_approval() # <-- UNCOVERED (financial)
```
### Low-Priority Gaps (can wait)
```python
# Logging statements
logger.debug("Processing batch %d", batch_id) # <-- UNCOVERED (low risk)
# Cosmetic formatting
def format_currency(amount):
if currency == "JPY": # <-- UNCOVERED (JPY support not built)
return f"¥{int(amount):,}"
```
### Prioritization Table
| Gap Type | Priority | Example |
|----------|----------|---------|
| Error/recovery paths | **HIGH** | `except`, `finally`, rollback |
| Auth/permission | **HIGH** | Admin-only, role checks |
| Financial/money | **HIGH** | Billing, pricing calculations |
| Data mutation (write) | **MEDIUM** | Insert, update, delete paths |
| Feature branches | **MEDIUM** | A/B test paths, config-gated |
| Logging | **LOW** | Debug/info log lines |
| Debug-only code | **SKIP** | `if DEBUG:`, dev tools |
| Dead code | **DELETE** | Unreachable blocks |
## Branch vs. Line Coverage
Line coverage is misleading. Branch coverage tells the real story:
```python
# 100% line coverage (both lines executed):
def is_valid(age):
return age >= 18 # line 1 — covered
# test: assert is_valid(20)
# But 50% branch coverage (only one branch tested):
# The `age < 18` branch is untested!
# test_is_valid_false() is missing!
```
```bash
# Enable branch coverage
pytest --cov=src/ --cov-report=term-missing --cov-branch
```
## Why 100% Is Usually Not the Goal
```text
Coverage | Interpretation
----------|---------------
<60% | Insufficient — high risk of regressions
60-80% | Moderate — fill high-priority gaps
80-90% | Good — focus on branch coverage, not line count
90-95% | Solid — the remaining 5-10% is probably error paths
100% | Suspicious — are you testing dead code? Or assertion-less?
```
## Guardrails
- **Never chase the number.** 100% coverage with assertion-light tests is worse than 80% with rigorous assertions.
- **Don't delete uncovered code** without verifying it's truly dead (grep for callers).
- **Coverage is a floor, not a ceiling.** It tells you what's tested, not what's tested *well*.
- **Branch coverage > Line coverage.** Always use `--cov-branch`.
## Pitfalls
| Pitfall | Fix |
|---------|-----|
| Assertion-less tests bumping coverage | Every test must assert a behavior change, not just call the function |
| Adding tests for dead code | Delete dead code instead |
| Ignoring uncovered error branches | Error paths are where prod breaks — test them first |
| `# pragma: no cover` abuse | Use sparingly; it's a signal of untestable design |
| Coverage drop on refactor | Re-run coverage after refactor — new code is often uncovered |
## Verify / Checklist
- [ ] Coverage report generated with `--cov-branch` enabled
- [ ] High-priority gaps (error handlers, auth, money) are identified
- [ ] HTML report reviewed for any large uncovered blocks
- [ ] Uncovered error paths have tests (or documented reason for exclusion)
- [ ] No assertion-less coverage-stuffing tests
- [ ] CI enforces a coverage threshold (recommended: 80%)Attached files
No attached files.