返回提示词目录第 1374 页

系统备份容灾与回滚审计

检查关键数据备份、恢复演练与部署回退能力,形成恢复缺口清单和整改方案。

完整提示词

# BACKUP, DISASTER RECOVERY AND ROLLBACK AUDIT

I want an ultimate forensic audit of backup, restore, disaster recovery, and rollback capabilities across the complete system.

Main objective:

> Determine whether the system can truly recover from data deletion, corruption, ransomware/admin compromise, bad migrations, broken deployments, region/provider failures, secret losses, and infrastructure destruction, rather than merely assuming that "backups exist."

Most critical rule:

> A backup that has never been restored is not proof of recoverability.

## 1. INVENTORY CRITICAL STATE

- primary database
- object/file storage
- user uploads
- configs
- secrets
- encryption keys
- queue state
- search index
- analytics
- source artifacts
- deployment artifacts
- DNS/IaC state

## 2. CLASSIFY

For each asset:

```text
Authoritative:
Rebuildable:
Backup required:
Recovery method:
```

## 3. DATABASE BACKUP

Verify:

- full backup
- incremental
- WAL/binlog
- snapshots
- PITR

## 4. FREQUENCY

Do not guess expected RPO.

If no requirement is specified:

**RPO NOT DEFINED**

## 5. RETENTION

How far back can the system be restored?

## 6. CORRUPTION WINDOW

If corruption goes undetected for 30 days while retention is 7 days:

backup does not solve the issue.

## 7. PITR

Granularity and retention limits.

## 8. RESTORE TEST

When was the last actual restore exercise conducted?

## 9. RESTORE IN ISOLATION

Never restore over production as a test.

## 10. DATA VALIDATION

A successful restore command output is insufficient.

Verify:

- tables
- row counts
- constraints
- application login
- critical reads/writes

## 11. ENCRYPTED BACKUP

Where is the encryption key stored?

## 12. LOST KEY

An encrypted backup without the corresponding key is unusable.

## 13. KEY BACKUP

However, the key must not reside unprotected alongside the backup.

## 14. ACCESS

Who has permission to read backups?

## 15. DELETE

Who has permission to delete backups?

## 16. SAME CREDENTIAL

If a single compromised admin credential can delete:

```text
production
+
all backups
```

the blast radius is catastrophic.

## 17. IMMUTABILITY

Object lock/WORM storage where justified.

## 18. SEPARATE ACCOUNT/PROJECT

For critical backup threat models.

Do not demand universally without justification.

## 19. OFFSITE

Same physical or provider failure domain.

## 20. CROSS-REGION

Only if a regional disaster recovery requirement exists.

## 21. FILE STORAGE

Database backups do not automatically incorporate uploaded files.

## 22. OBJECT STORAGE VERSIONING

Recovery from accidental overwrite or deletion.

## 23. LIFECYCLE

Version expiration schedules.

## 24. CDN

Is not a backup solution.

## 25. CACHE

If disposable, rebuild.

If holding authoritative state, backup and recovery required.

## 26. REDIS

Sessions/queues/locks vs durable business data.

## 27. QUEUE

Does losing queued messages equate to irrecoverable business data loss?

## 28. DLQ

Is not a backup.

## 29. SEARCH INDEX

Can it be reconstructed from the primary database?

## 30. ANALYTICS

Is analytics data critical to business continuity?

## 31. SECRET MANAGER

How to recover:

- deleted secret
- rotated key
- lost encryption key

## 32. SIGNING KEY

Loss can force global user logouts or completely break token verification.

## 33. PRIVATE TLS KEY

Certificate reissue procedure.

## 34. CODE SIGNING KEY

Loss or compromise response plan.

## 35. IaC STATE

Terraform state backup and versioning.

## 36. IaC SOURCE

Can rebuild infrastructure, but cannot restore data.

## 37. DNS

Zone file exports and configuration backups.

## 38. DEPLOYMENT ARTIFACT

Previous production artifact availability.

## 39. REGISTRY RETENTION

Ensure rollback container images are not pruned.

## 40. PACKAGE ARTIFACT

Desktop and mobile release archives.

## 41. DATABASE MIGRATION

Backing up prior to a destructive migration is insufficient if restore takes 12h while business RTO is 15min.

## 42. FORWARD FIX VS RESTORE

Restores can occasionally cause greater loss of recently committed data than forward fixes.

## 43. SELECTIVE RESTORE

Can a single tenant, table, or object be recovered?

## 44. FULL RESTORE

Complete system recovery.

## 45. LOGICAL BACKUP

Useful for granular and selective recovery.

## 46. PHYSICAL SNAPSHOT

Fast but coarse and provider-dependent.

## 47. CONSISTENCY

Across multiple datastores:

```text
DB
+
object storage
```

backup timestamps can be out of sync.

## 48. APPLICATION-CONSISTENT BACKUP

When required.

## 49. TRANSACTIONAL LINK

Database record pointing to specific object store versions.

## 50. PARTIAL RESTORE

Database restored to yesterday while object storage remains current.

## 51. CROSS-SYSTEM RPO

The slowest or retention-limited state dictates the actual recovery point.

## 52. DISASTER SCENARIOS

Model separately:

- accidental delete
- malicious delete
- corruption
- bad migration
- broken deploy
- ransomware/admin compromise
- provider outage
- region outage
- account lockout
- secret compromise
- key loss

## 53. ACCIDENTAL DELETE

The most frequent real-world recovery scenario.

## 54. SOFT DELETE

Is not a backup.

## 55. MALICIOUS DELETE

An attacker can permanently purge soft-deleted records.

## 56. BAD SCRIPT

Bulk database update corruption.

## 57. BUG CORRUPTION

Backups taken after a silent bug was introduced will preserve corrupted state.

## 58. DETECTION TIME

How quickly is silent corruption discovered?

## 59. BACKUP RETENTION > DETECTION WINDOW

If not, a permanent recovery gap exists.

## 60. RANSOMWARE/CLOUD ADMIN

Assume production credentials are fully compromised.

## 61. BACKUP CREDENTIAL SEPARATION

Can an attacker pivot to the backup storage plane?

## 62. PROVIDER ACCOUNT LOSS

Can the organization regain access to the hosting account?

Operational and business continuity planning.

## 63. REGION LOSS

Only evaluate if multi-region is an explicit disaster target.

## 64. MULTI-REGION DATA

Replication is not a backup.

## 65. REPLICATION

Corrupted data or deletions replicate immediately.

## 66. HA ≠ BACKUP

Crucial distinction.

## 67. BACKUP ≠ HA

Equally crucial.

## 68. DR ≠ ROLLBACK

Keep concepts distinct.

## 69. APPLICATION ROLLBACK

Redeploying the previous artifact.

## 70. DATABASE ROLLBACK

Often impossible following destructive schema modifications.

## 71. CONFIG ROLLBACK

Restoring previous configuration versions.

## 72. SECRET ROLLBACK

Never restore a known-compromised secret.

## 73. INFRA ROLLBACK

IaC previous state redeployment.

## 74. FEATURE FLAG

Can avoid full binary rollbacks.

## 75. DEPLOYMENT ROLLBACK

Must be tested in practice.

## 76. ROLLBACK AFTER MIGRATION

Critical verification point.

## 77. ROLLBACK AFTER NEW DATA FORMAT

The older application might not know how to parse data written by the newer release.

## 78. FORWARD COMPATIBILITY

Compatibility requirements.

## 79. QUEUE ROLLBACK

Old workers attempting to read new message payloads.

## 80. CACHE ROLLBACK

Serialized format mismatches.

## 81. RESTORE DURATION

Measure empirically, do not guess.

## 82. RTO

If undefined:

**RTO NOT DEFINED**

## 83. RESTORE BOTTLENECK

- download
- decompression
- DB import
- object restore
- DNS
- validation

## 84. COLD STORAGE

Retrieval delays and warming times.

## 85. LARGE DB

Restore durations grow non-linearly.

## 86. PARALLEL RESTORE

Potential optimization, not an automatic assumption.

## 87. RUNBOOK

Step-by-step procedural runbooks.

## 88. OWNER

Who has designated ownership of the restore procedure?

## 89. ACCESS DURING INCIDENT

Do incident responders maintain access if the primary IAM provider is impaired?

## 90. BREAK-GLASS

Procedures for emergency break-glass recovery.

## 91. CONTACTS

Provider escalation contacts and support tiers.

## 92. EVIDENCE PRESERVATION

Incident restores should not prematurely destroy forensic logs or evidence.

## 93. CLEAN RESTORE

If attacker persistence exists, restoring a compromised image or config reintroduces the attacker.

## 94. SECRET ROTATION AFTER RESTORE

Mandatory rotation following security compromise.

## 95. DR EXERCISE

Tabletop review vs actual technical restore execution.

## 96. TABLETOP

Useful for coordination, but not technical proof of recoverability.

## 97. GAME DAY

Controlled disaster simulation drills.

## 98. RESTORE AUTOMATION

Reduces manual operator error during crises.

## 99. AUTOMATION BUG

Recovery automation scripts must themselves be regularly tested.

## 100. BACKUP MONITORING

Alert on:

- failed backup
- stale backup
- size anomaly
- PITR disabled
- replication issue

## 101. "SUCCESS" CHECK

A 0-byte backup job can exit with code 0 if payload validation is omitted.

## 102. SIZE TREND

Valuable metric for anomaly detection.

## 103. RESTORE CHECKSUM

Cryptographic integrity verification.

## 104. BACKUP CATALOG

Maintain an accurate registry of existing backups.

## 105. OWNERSHIP

Designated owners for each recovery domain.

## 106. DOCUMENTATION DRIFT

Runbooks pointing to deprecated services, outdated paths, or stale credentials.

## 107. DEPENDENCY RESTORE

Third-party SaaS data may require dedicated export or backup processes.

## 108. EMAIL/CRM/EXTERNAL SYSTEM

Only if authoritative business data resides exclusively on external platforms.

## 109. USER EXPORT

Is not an adequate substitute for system-wide backups.

## 110. LEGAL RETENTION

Do not make legal assertions unless regulatory specifications are provided.

## 111. DATA DELETION REQUIREMENTS

Compliance-mandated deletions can conflict with immutable backup retention.

## 112. BACKUP SECURITY

Backups are often more lucrative targets for attackers than production databases due to unmonitored offline access.

## 113. PASSWORD HASHES

Sensitive exposure risks.

## 114. TOKENS

Older backups may contain unexpired API or refresh tokens.

## 115. SECRET ROTATION + OLD BACKUP

Restoring an older snapshot can resurrect rotated credentials or stale security state.

## 116. RESTORE SANITIZATION

Ensure revoked or compromised tokens remain revoked following a database restore.

## 117. USER PASSWORD CHANGE

Point-in-time recovery prior to a user password reset will revert to the old hash.

Security implications.

## 118. ACCOUNT DELETION

Restores can resurrect accounts deleted under privacy mandates.

Business and privacy handling requirements.

## 119. AUDIT LOG

Restores must preserve or reconcile chronological audit events.

## 120. ID GENERATORS

Rolling back database sequences can cause primary key collisions with external systems.

## 121. EXTERNAL SIDE EFFECTS

You cannot roll back dispatched emails or processed credit card payments.

## 122. PAYMENT RECONCILIATION

Database restores may drop records of transactions finalized by the payment gateway.

## 123. WEBHOOK REPLAY

Provider webhook events must be reconciled post-restore.

## 124. EVENT SOURCING

If event sourcing is utilized, the recovery model differs significantly.

## 125. THIRD-PARTY SOURCE OF TRUTH

Establish which system holds authoritative state.

## 126. DR MODE

Operating in a degraded, read-only mode during recovery.

## 127. MAINTENANCE PAGE

Preserving user experience and preventing erroneous writes during restoration.

## 128. DNS FAILOVER

If an alternate standby environment exists.

## 129. COLD STANDBY

Cost vs RTO trade-off.

## 130. WARM STANDBY

Warm standby considerations.

## 131. HOT STANDBY

Do not recommend automatically without cost justification.

## 132. REGION RESTORE

Can IaC plus offsite backups reconstruct an entire region?

## 133. ACCOUNT-LEVEL DISASTER

Separate account backup strategies apply primarily to high-assurance threat models.

## 134. SINGLE PROVIDER

Not inherently unacceptable for all architectures.

## 135. FINDING FORMAT

```text
ID:
Severity:
Status:
Evidence tier:
Asset:
Authoritative:
Failure scenario:
Backup mechanism:
Frequency:
Retention:
Restore tested:
Last known restore:
RPO requirement:
Actual estimated RPO:
RTO requirement:
Measured/estimated RTO:
Failure path:
Data loss:
Availability impact:
Security impact:
Evidence:
Root cause:
Remediation:
Restore verification:
```

## 136. SEVERITY

P0:
- critical data has no usable recovery and realistic catastrophic loss path exists
- backup encryption/key loss makes all recovery impossible
- one compromised credential can irreversibly destroy production + all backups and no secondary recovery exists

P1:
- backup exists but proven unusable
- actual RPO/RTO grossly violates explicit critical requirement
- rollback procedure can further corrupt/destroy production
- region/account failure has no recovery despite explicit DR requirement

P2:
- meaningful restore gap
- partial data class omitted
- stale/unverified runbook
- insufficient retention vs realistic corruption detection

P3:
- limited recovery weakness

P4:
- maturity/hardening

## 137. EVIDENCE

```text
A - successful/failed restore exercise or production incident evidence
B - complete backup/restore config and path
C - strong config evidence
D - inferred
E - maturity
```

## 138. STATUS AND FALSE POSITIVES

Status:

- **CONFIRMED** - tier A or B evidence shows the failure or exploit path.
- **LIKELY** - tier C evidence.
- **NOT VERIFIED** - depends on runtime state, settings or versions that could not be checked (tier D). Never present tier D as confirmed.
- **NOT APPLICABLE** - the component or pattern is not used.
- **CONTROLLED** - the risk exists but another control contains it.
- **HARDENING** - improvement without a current failure path (P4).

Recoverability is CONFIRMED only by a restore exercise or incident evidence (tier A); a complete configuration (tier B) proves that the backup path exists, not that the restore works within RPO/RTO.

False-positive rules:

- A single region or single provider is not a defect when the stated recovery requirements do not demand more.
- Missing point-in-time recovery is a finding only when a realistic corruption or deletion scenario needs a finer restore point than existing backups provide.
- Derived or rebuildable state (caches, search indexes, analytics copies) does not need backups if the rebuild path and its duration are acceptable.
- Retention shorter than a compliance guideline is a finding only when a requirement applies or realistic detection time exceeds it.
- An irreversible migration is not a rollback defect if a tested roll-forward path exists.

Do not report a missing best practice as a confirmed defect unless there is a concrete failure, exploit, correctness, reliability, or operational path.

## 139. OUTPUT

`BACKUP_DISASTER_RECOVERY_ROLLBACK_AUDIT.md`

## 140. MATRICES

### Critical State Matrix

| State | Authoritative | Backup | PITR | Restore tested |
|---|---|---|---|---|

### Failure Matrix

| Disaster | Data affected | Recovery | RPO | RTO |
|---|---|---|---|---|

### Rollback Matrix

| Component | Previous version available | Data compatible | Rollback tested |
|---|---|---|---|

### Credential Separation

| Identity | Prod delete | Backup read | Backup delete | KMS |
|---|---|---|---|---|

## 141. SECOND PASS - RESTORE EXERCISES

Mandatory review:

### Scenario A
Accidental deletion of a single table or critical resource.

### Scenario B
Bad migration corrupts active data.

### Scenario C
Production database completely destroyed.

### Scenario D
Object storage files or buckets deleted.

### Scenario E
Production cloud administrator account compromised.

### Scenario F
Backup encryption key unavailable or destroyed.

### Scenario G
Latest backup corrupted upon restore.

### Scenario H
Restoration required to a state from 24 hours ago.

### Scenario I
Application rollback required following a database migration.

### Scenario J
Complete cloud region or provider outage (if within scope).

For each scenario:

```text
Detection
↓
Decision
↓
Containment
↓
Restore
↓
Validation
↓
Traffic return
↓
Reconciliation
```

## 142. SECOND PASS - ACTUAL RESTORE

If a safe, isolated staging/test environment is available:

1. obtain a production-like backup
2. restore into a blank, isolated environment
3. launch the application stack
4. validate end-to-end data integrity
5. record actual elapsed recovery time
6. document all errors encountered

Never restore over production purely for audit validation.

## 143. SECOND PASS - CREDENTIAL COMPROMISE

Assume an attacker gains root/admin credentials in production.

Ask:

> Can the attacker delete backups, destroy KMS keys, and purge audit logs using those same credentials?

## 144. SECOND PASS - LONG-LIVED CORRUPTION

Assume a bug silently corrupts data for 30 days before detection.

Does existing retention allow recovery to an uncorrupted snapshot?

## 145. SECOND PASS - ROLLBACK REALITY

Deployment sequence:

```text
v1 -> migration -> v2
```

then evaluate conceptually or in staging:

```text
v2 -> v1
```

Does v1 still understand:

- schema
- new rows
- enum values
- queue messages
- cache
- config

## 146. FINAL QUALITY GATE

Prior to final submission, verify:

- every authoritative datastore
- file and object storage
- secrets and encryption keys
- deployment artifacts
- backup storage location
- access and deletion permissions
- PITR configuration
- retention windows
- encryption key recovery procedures
- verified restore tests
- measured/estimated restore duration
- explicit RPO/RTO requirements
- application validation following restore
- cross-store data consistency
- rollback feasibility following migrations
- queue, cache, and client compatibility
- attacker/admin compromise blast radius
- backup monitoring and alerting
- runbook ownership and freshness
- external side-effect reconciliation

# FINAL RULE

Looking for issues such as:

```text
database backup:
daily
↓
backup encrypted with customer-managed key
↓
same cloud admin identity can:
delete DB
delete backups
schedule KMS key deletion
↓
admin credential compromised
↓
production + backups + decryption capability lost
↓
recovery impossible
```

or:

```text
migration adds new enum values
↓
v2 writes new values
↓
deployment later fails
↓
team rolls back to v1
↓
v1 cannot deserialize new enum
↓
rollback deployment itself causes outage
```

or:

```text
DB is restored to yesterday
↓
payment provider was not rolled back
↓
provider has 500 successful payments
↓
restored DB remembers only 450
↓
system may retry/reconcile incorrectly
↓
financial state divergence
```

If backups exist but restore was never verified:

**RECOVERABILITY NOT VERIFIED.**

If RPO/RTO are not explicitly defined:

**REQUIREMENT NOT DEFINED.**

If only DR maturity improvements are identified without an empirical gap:

**P4 - HARDENING.**

分类与标签

编程开发、备份恢复、容灾演练、部署回滚、恢复目标

前往 AI 图片创作