返回提示词目录第 1375 页

测试套件风险覆盖与断言审计

将关键业务约束映射到现有测试,识别弱断言与故障覆盖缺口,按生产风险提出补测方案。

完整提示词

I want you to perform a maximally deep, systematic, evidence-first and production-oriented audit of the complete test suite of the application or repository.

Main objective:

> Determine whether the existing test suite actually protects critical business invariants, failure modes and production behavior, or only creates an illusion of safety through a large number of tests that do not cover the most important risks.

This is not:

- counting test files
- a coverage percentage audit without understanding risk
- insisting on 100% coverage
- an assumption that more tests automatically mean better quality
- automatic criticism of mocks
- automatic criticism of E2E tests because they are slow
- automatically favoring unit tests
- a recommendation to test every private helper directly
- a replacement for production observability
- a replacement for a security audit

Priority:

**critical correctness > data integrity > authorization/security invariants > regression protection > concurrency/reliability > deployment safety > user flows > maintainability > raw coverage percentage**

It is better to have 200 tests that prove critical invariants than 5,000 tests that check trivial implementation details.

## 1. REPOSITORY AND TEST STACK DISCOVERY

Before any findings, map:

- language
- framework
- test framework
- test runners
- assertion libraries
- mocking libraries
- fixtures
- factories
- test containers
- emulators
- browsers
- device tests
- contract tests
- integration tests
- E2E tests
- performance tests
- property-based tests
- fuzz tests
- security tests
- CI/local execution
- database strategy
- external service strategy

If there are several test projects, map all of them.

## 2. TEST TAXONOMY

Classify the actual tests, not just the folders:

```text
unit
component
integration
contract
API
database
UI
E2E
smoke
regression
performance
security
property
fuzz
chaos
```

## 3. CRITICAL FLOW INVENTORY

Before the coverage analysis, list the most important flows of the system.

For each:

```text
Flow:
Criticality:
Main invariant:
External dependencies:
Persistence:
Authorization:
Concurrency:
Rollback/recovery:
Existing tests:
```

## 4. BUSINESS INVARIANTS

Extract the invariants from the production code.

Example:

```text
order total cannot become negative
user cannot read another tenant's record
same webhook cannot create duplicate payment
inventory cannot be decremented twice
```

## 5. INVARIANT TO TEST MAPPING

For each critical invariant, find evidence that the test suite checks it.

## 6. HAPPY PATH

It is not enough.

## 7. NEGATIVE PATH

## 8. BOUNDARY VALUE

## 9. INVALID STATE

## 10. ERROR PATH

## 11. RETRY PATH

## 12. ROLLBACK PATH

## 13. CONCURRENCY PATH

## 14. PERMISSION PATH

## 15. DATA MIGRATION PATH

## 16. EXTERNAL SERVICE FAILURE

## 17. TEST ORACLE

Ask:

How does the test know that the result is actually correct?

## 18. WEAK ASSERTION

Example:

```text
expect(response.status).toBe(200)
```

is not enough if the critical business state can be wrong.

## 19. NO ASSERTION

A test that only "does not throw an exception".

## 20. ASSERTION ON WRONG LAYER

Mock verified, real state not verified.

## 21. SNAPSHOT TEST

A snapshot is not automatically good or bad.

Check the signal/noise ratio.

## 22. GOLDEN FILE

## 23. MOCKING

Map:

- what is mocked
- why
- what behavior is assumed
- what contract remains unverified

## 24. OVER-MOCKING

Test mirrors implementation and misses integration.

## 25. UNDER-MOCKING

Test becomes slow/fragile without additional confidence.

## 26. FAKE

Could be better than brittle mock where appropriate.

## 27. REAL DATABASE

## 28. IN-MEMORY DATABASE

Can differ materially from production engine.

## 29. TEST CONTAINERS

## 30. SCHEMA MIGRATIONS

Tests should use realistic schema state.

## 31. DATABASE TRANSACTION TEST

## 32. ROLLBACK TEST

## 33. LOCK/CONCURRENCY TEST

## 34. EXTERNAL API

## 35. CONTRACT TEST

## 36. PROVIDER SANDBOX

## 37. API MOCK

Must not drift from provider.

## 38. CONSUMER-DRIVEN CONTRACT

Where appropriate.

## 39. FRONTEND TESTING

Audit:

- rendering
- state transitions
- async
- error
- accessibility
- keyboard
- forms
- navigation

## 40. UI IMPLEMENTATION DETAIL

Avoid tests tightly coupled to DOM internals without need.

## 41. BACKEND TESTING

Audit:

- business logic
- persistence
- authorization
- validation
- error semantics
- transactions
- external calls

## 42. API TESTS

Check:

- status
- body
- side effects
- authorization
- idempotency
- error contract

## 43. AUTHENTICATION TESTS

## 44. AUTHORIZATION TESTS

Role matrix.

## 45. TENANT ISOLATION TESTS

## 46. IDOR REGRESSION

## 47. FILE UPLOAD TESTS

## 48. RATE LIMIT TESTS

## 49. WEBHOOK TESTS

## 50. BACKGROUND JOB TESTS

## 51. QUEUE TESTS

## 52. SCHEDULER TESTS

## 53. DUPLICATE DELIVERY

## 54. OUT-OF-ORDER DELIVERY

## 55. RETRY

## 56. DEAD LETTER

## 57. MOBILE TESTS

If applicable:

- lifecycle
- process death
- permissions
- offline
- rotation
- background
- release build

## 58. DESKTOP TESTS

## 59. BROWSER MATRIX

## 60. DEVICE MATRIX

## 61. VERSION MATRIX

## 62. FEATURE FLAGS

Test both paths.

## 63. CONFIGURATION

## 64. ENVIRONMENT DIFFERENCE

## 65. RELEASE BUILD

Debug-only success is insufficient.

## 66. TEST DATA

Realistic?

## 67. EDGE DISTRIBUTION

## 68. UNICODE

## 69. TIMEZONE

## 70. DST

## 71. CLOCK

Use controllable clock where relevant.

## 72. RANDOM

Seed/reproducibility.

## 73. GENERATED IDS

## 74. SORT ORDER

## 75. PAGINATION

## 76. LARGE DATASET

## 77. EMPTY DATASET

## 78. DUPLICATES

## 79. NULL

## 80. ZERO

## 81. MAXIMUM

## 82. MALFORMED INPUT

## 83. RACE

## 84. TEST ISOLATION

One test should not depend on another.

## 85. ORDER DEPENDENCE

## 86. SHARED GLOBAL STATE

## 87. DATABASE CLEANUP

## 88. PORT CONFLICT

## 89. TEMP FILE

## 90. CACHE

## 91. CLOCK LEAK

## 92. TEST PARALLELISM

## 93. PARALLEL SAFETY

## 94. TEST RETRIES

Retry can hide flaky or real defect.

## 95. QUARANTINED TEST

## 96. SKIPPED TEST

Audit reason and age.

## 97. TODO TEST

## 98. DISABLED SUITE

## 99. CI VS LOCAL

Test suite may run differently.

## 100. ENV VAR

## 101. SECRETS

## 102. SERVICE DEPENDENCY

## 103. NETWORK ACCESS

Unexpected live calls.

## 104. DETERMINISM

## 105. FLAKINESS

For each flaky test, establish its failure rate, the conditions under which it fails and whether the cause is in the test or in the product.

A retry is not a fix.

## 106. TEST DURATION

## 107. SLOW TEST

Not automatically bad.

## 108. TEST PYRAMID

Do not apply dogmatically.

## 109. TEST PORTFOLIO

Optimize for risk, not shape.

## 110. COVERAGE

Use:

- line
- branch
- function
- mutation where available
- critical-path coverage

## 111. COVERAGE BLIND SPOT

High line coverage can miss business combinations.

## 112. BRANCH COVERAGE

## 113. CONDITION COVERAGE

## 114. MUTATION TESTING

Useful for assertion quality, not mandatory everywhere.

## 115. COVERAGE EXCLUSION

## 116. GENERATED CODE

## 117. DEAD CODE

## 118. CRITICAL FILE WITH LOW COVERAGE

## 119. HIGH COVERAGE LOW VALUE

## 120. MISSING TESTS

Rank missing tests by production risk, not by coverage percentage.

For each one, propose the lowest test layer that proves the invariant.

## 121. REGRESSION HISTORY

Use past bugs/incidents.

## 122. BUG TO TEST

Every serious fixed bug should normally gain regression protection where feasible.

## 123. INCIDENT TO TEST

## 124. MIGRATION FAILURE HISTORY

## 125. PRODUCTION DATA SHAPE

## 126. PERFORMANCE REGRESSION

## 127. SECURITY REGRESSION

## 128. TEST MAINTAINABILITY

## 129. DUPLICATE TESTS

## 130. COPY/PASTE

## 131. HELPER ABSTRACTION

Too much abstraction can hide intent.

## 132. TEST NAME

Should state behavior.

## 133. ARRANGE ACT ASSERT

Not mandatory formatting, but intent should be clear.

## 134. FIXTURE COMPLEXITY

## 135. FACTORY DEFAULT

Can hide required field assumptions.

## 136. TEST READABILITY

## 137. FAILURE MESSAGE

## 138. DEBUGGABILITY

## 139. OBSERVABILITY OF TEST FAILURE

Logs/artifacts/screenshots where useful.

## 140. E2E ARTIFACT

Video/screenshot/trace.

## 141. TEST SHARDING

## 142. CACHE

Build/test caching can hide stale artifacts if misconfigured.

## 143. CI GATE

Which tests actually block merge/release?

## 144. OPTIONAL TESTS

## 145. RELEASE GATE

## 146. POST-DEPLOY SMOKE

## 147. MONITORING AS TEST

Production monitor is not replacement for pre-release test, but can cover live-only invariants.

## 148. FALSE POSITIVE RULES

Do not automatically report:

- low global coverage
- high global coverage
- many mocks
- no mocks
- slow integration test
- absence of E2E
- absence of unit test for trivial getter
- snapshot tests
- skipped test

A finding must show a concrete risk or coverage gap.

## 149. EVIDENCE TIERS

```text
A - test execution, mutation result, reproduced bug or production incident evidence
B - complete code-to-test mapping proving gap/weakness
C - strong static evidence
D - suspected test weakness requiring verification
E - test hardening/maturity
```

## 150. STATUS MODEL

```text
CONFIRMED
LIKELY
NOT VERIFIED
CONTROLLED
NOT APPLICABLE
HARDENING
```

## 151. SEVERITY

P0:
- test suite systematically allows catastrophic corruption/security/release failure with no guard where automated prevention is feasible

P1:
- critical business/security invariant untested and demonstrably vulnerable to regression
- release suite misses known severe production failure path

P2:
- material coverage or assertion weakness

P3:
- limited quality/maintainability weakness

P4:
- test maturity improvement

## 152. FINDING FORMAT

```text
ID:
Severity:
Status:
Evidence tier:
Component/flow:
Invariant:
Existing tests:
Missing/weak behavior:
Failure scenario:
Production impact:
Evidence:
Why current test misses it:
Recommended test layer:
Concrete test cases:
Regression risk:
```

## 153. MATRICES

### Critical Flow Coverage Matrix

| Flow | Happy | Error | Auth | Concurrency | Recovery |
|---|---|---|---|---|---|

### Invariant Matrix

| Invariant | Unit | Integration | E2E | Production signal |
|---|---|---|---|---|

### Test Layer Matrix

| Behavior | Current layer | Appropriate layer | Gap |
|---|---|---|---|

## 154. SECOND PASS

Re-check the suite through:

- known past bug
- invalid auth
- cross-tenant access
- duplicate event
- timeout
- partial failure
- process restart
- concurrent request
- empty data
- very large data
- release build
- feature flag off/on
- production-like database
- external API contract drift

## 155. FINAL QUALITY GATE

Confirm that you have covered:

- critical flows
- invariants
- assertions
- mocks/fakes
- DB
- API
- auth
- concurrency
- external services
- jobs/queues
- failure/recovery
- UI
- release configuration
- data boundaries
- flaky/skipped tests
- coverage
- regressions
- test maintainability
- CI/release gating

## 156. OUTPUT

`ULTIMATE_TEST_SUITE_AUDIT.md`

## 157. FAILURE CHAINS

```text
webhook handler has 95% line coverage
↓
tests call handler only once per event
↓
provider retries same webhook
↓
idempotency path is never tested
↓
same payment is processed twice in production
```

```text
authorization tests mock repository
↓
mock always returns tenant-scoped record
↓
real query misses tenant filter
↓
test suite stays green
↓
cross-tenant data leak reaches production
```

# FINAL RULE

Do not judge a test suite by the number of tests.

Judge it by the question:

> If the most important production invariant breaks tomorrow, is there a test that will reliably stop it before release?

分类与标签

编程开发、测试覆盖分析、断言质量、业务约束、回归测试、发布质量门禁

前往 AI 图片创作