返回提示词目录第 1351 页

智能体故障恢复与可靠性审计

审查智能体在超时、重试、崩溃和并发下的任务完成情况,形成含证据、修复建议与测试矩阵的报告。

完整提示词

I want a deep reliability audit of the AI agent with a focus on trajectory correctness, retries, loops, recovery, partial side effects, concurrency, nondeterminism and real task completion.

Main objective:

> Prove that the agent can reliably complete the task under failure conditions too, not only in an ideal happy-path demo.

This is not:

- a benchmark on happy-path demo tasks
- counting steps or tool calls as a measure of quality
- an assumption that retries, fallback or nondeterminism are failures by themselves
- an assumption that "done" from the model means the task is complete
- a redesign of the agent architecture, unless a concrete reliability defect requires it
- a claim of exactly-once execution without proof

Priority:

**duplicated or lost irreversible side effects > corrupted or lost state > authority violations during recovery > runaway execution > unrecoverable task failure > silent task failure > tail latency/cost > observability gaps**

## 1. RELIABILITY SLO

Define:

- success
- partial success
- safe failure
- unsafe failure

## 2. TASK CLASSES

Segment them.

## 3. COMPLETION CRITERIA

Model saying "done" is not completion.

## 4. AUTHORITATIVE VERIFICATION

## 5. TRAJECTORY

Record every step.

## 6. STEP FAILURE

## 7. PARTIAL SUCCESS

## 8. RETRY

## 9. IDEMPOTENCY

## 10. UNKNOWN OUTCOME

## 11. DUPLICATE TOOL CALL

## 12. DUPLICATE AGENT RUN

## 13. CRASH

## 14. RESUME

## 15. CHECKPOINT

## 16. STALE CHECKPOINT

## 17. STATE VERSION

## 18. RETRY AFTER CODE UPDATE

## 19. PROVIDER TIMEOUT

## 20. PROVIDER 5XX

## 21. RATE LIMIT

## 22. FALLBACK MODEL

## 23. QUALITY DEGRADATION

## 24. TOOL TIMEOUT

## 25. TOOL RATE LIMIT

## 26. TOOL PARTIAL EFFECT

## 27. TOOL SCHEMA ERROR

## 28. TOOL RESULT MALFORMED

## 29. NETWORK PARTITION

## 30. EXTERNAL CONSISTENCY

## 31. LONG-RUN TASK

## 32. LEASE EXPIRY

## 33. CONCURRENT WORKER

## 34. FENCING

## 35. DUPLICATE QUEUE DELIVERY

## 36. MESSAGE ORDER

## 37. STALE EVENT

## 38. LOOP

## 39. OSCILLATION

## 40. REPEATED ERROR

## 41. MAX STEPS

## 42. MAX WALL CLOCK

## 43. MAX COST

## 44. MAX TOKENS

## 45. MAX TOOL CALLS

## 46. ESCALATION

## 47. HUMAN TAKEOVER

## 48. CANCELLATION

## 49. CANCEL DURING TOOL

## 50. CANCEL AFTER EFFECT

## 51. USER DISCONNECT

## 52. BACKGROUND DURABILITY

## 53. MODEL NONDETERMINISM

## 54. REPEATED RUNS

## 55. SUCCESS RATE

## 56. CRITICAL FAILURE RATE

## 57. RETRY SUCCESS

## 58. MEAN STEPS

## 59. P95 STEPS

## 60. COST DISTRIBUTION

## 61. TAIL LATENCY

## 62. PATH EXPLOSION

## 63. LONG-CONTEXT DRIFT

## 64. SUMMARY DRIFT

## 65. MEMORY STALE

## 66. CONTEXT LOSS

## 67. TOOL DISCOVERY CHANGE

## 68. MODEL UPGRADE

## 69. PROMPT UPGRADE

## 70. REGRESSION SET

## 71. TRAJECTORY GOLDEN SET

## 72. CHAOS TESTING

Safe mocks/staging.

## 73. FAULT INJECTION

Inject:

- timeouts
- duplicate results
- malformed responses
- slow tools
- stale data

## 74. PROPERTY

Critical invariant should survive.

## 75. EXACTLY ONCE

Do not claim unless proven.

## 76. AT-LEAST-ONCE

Design for duplicate.

## 77. COMPENSATION

## 78. COMPENSATION FAILURE

## 79. RECONCILIATION

## 80. FINAL STATE CHECK

## 81. SIDE EFFECT LEDGER

## 82. RECEIPTS

## 83. OBSERVABILITY

## 84. TRACE CORRELATION

## 85. INCIDENT REPLAY

## 86. USER-VISIBLE FAILURE

## 87. ERROR MESSAGE

## 88. SAFE RETRY UX

## 89. RETRY BUTTON

Must not duplicate effect.

## 90. FALSE POSITIVE RULES

- multiple steps are not failure
- retries are not failure
- fallback is not failure
- nondeterminism is not failure

Finding requires measurable correctness/reliability impact.

## 91. EVIDENCE TIERS

```text
A - reproduced failure/fault injection/runtime trace
B - complete control-flow proof
C - strong static evidence
D - suspected failure
E - hardening
```

## 92. STATUS

```text
CONFIRMED
LIKELY
NOT VERIFIED
CONTROLLED
NOT APPLICABLE
HARDENING
```

## 93. SEVERITY

P0:
catastrophic autonomous reliability failure with irreversible impact

P1:
repeatable critical duplicate/lost side effect, runaway execution or unsafe recovery

P2:
material task failure under realistic faults

P3:
limited resilience/observability weakness

P4:
maturity/hardening

## 94. FINDING FORMAT

```text
ID:
Severity:
Status:
Evidence tier:
Task:
Fault:
Trajectory:
Checkpoint:
Retry:
External effect:
Expected state:
Actual state:
Detection:
Recovery:
Impact:
Evidence:
Fix:
Chaos/regression test:
```

## 95. RELIABILITY MATRIX

| Task | Happy path | Timeout | Duplicate | Crash/resume | Fallback |
|---|---|---|---|---|---|

## 96. FAILURE MATRIX

| Component | Fault | Detection | Retry | Safe? |
|---|---|---|---|---|

## 97. SECOND PASS

Force:

- every external call timeout once
- every write receive duplicate response
- model returns invalid tool args
- process crash after side effect
- process crash before checkpoint
- fallback model mid-task
- two workers same task
- cancellation at every stage
- memory missing
- context truncated

## 98. FINAL QUALITY GATE

Confirm:

- completion
- verification
- retries
- idempotency
- crash
- resume
- duplicates
- concurrency
- loops
- budgets
- cancellation
- compensation
- reconciliation
- fault injection
- metrics
- incident replay

## 99. OUTPUT

`AI_AGENT_RELIABILITY_AUDIT.md`

## 100. FAILURE CHAINS

```text
agent sends email
↓
SMTP accepted message
↓
network response lost
↓
tool reports timeout
↓
agent retries
↓
recipient gets duplicate email
```

```text
agent loop repeatedly calls search
↓
each result slightly changes summary
↓
model believes more research is needed
↓
no max-cost/step boundary
↓
task consumes extreme tokens without progress
```

# FINAL RULE

Agent reliability is not measured by the agent completing 20 ideal demo tasks.

It is measured by whether, under controlled failure conditions, it:

```text
does not lose state
does not duplicate effects
does not exceed its authority
does not loop indefinitely
and can prove the actual final result
```

分类与标签

编程开发、智能体审计、故障注入、幂等性检查、并发恢复、可靠性测试

前往 AI 图片创作