设计生产数据脱敏测试集
围绕测试用途设计数据抽样与脱敏流水线,交付脚本、完整性检查及需专业确认的审批事项。
完整提示词
<context>
Production-shaped data finds bugs that hand-written fixtures never do: real distributions, skew, legacy rows from old versions and odd encodings. Copying production wholesale into test environments, though, is one of the most common causes of data leaks, because test systems have weaker access controls, logs and third-party integrations. The safe approach subsets (start from a sample of root entities such as accounts, then follow foreign keys so every reference resolves), masks every personal, payment and secret field in a way that keeps formats and relationships (the same email masked the same way everywhere, valid-looking but non-routable), and removes things that can act on the real world: real email addresses, phone numbers, webhook URLs, API keys and tokens. Masking must happen before data reaches the lower environment, and free-text fields (notes, comments, JSON blobs) are where personal data hides. Pseudonymised data is often still personal data under privacy law, so access to the result still needs controls.
</context>
<task>
Plan and script a test dataset from production.
Database: {{database}}
Purpose: {{purpose}}
{{#obligations}}
Obligations: {{obligations}}
{{/obligations}}
1. Read the schema, migrations or models. Classify every column: identifier, direct personal data (names, emails, phones, addresses, dates of birth, government ids), indirect or sensitive (health, location, free text, IP addresses), payment, secrets and tokens, external endpoints, business data. Flag free-text and JSON columns as needing inspection.
2. Design the subset: choose root entities and the sampling rule (random sample, plus deliberately included edge cases such as the largest accounts, oldest records, each status value), then follow foreign keys and implicit references (polymorphic ids, ids in JSON) so the subset is referentially complete. Exclude tables not needed for the purpose, such as audit logs.
3. Define masking per column: deterministic pseudonyms where joins or uniqueness depend on the value, format-preserving fake values, date shifting that keeps intervals, generalisation for sensitive values, and nulling or replacing secrets, tokens, webhook URLs and real contact details with non-routable test values (for example reserved example domains).
4. Write the pipeline: extract inside the production security boundary, mask there, and only then load to the target. Use the stack's tooling or SQL in the repository's style. Make it repeatable and idempotent, and keep masking keys or salts out of the repository.
5. Define checks before release: referential integrity, row counts, no value from a deny-list of real identifiers, regex scans for emails, phones and card numbers, a manual sample review of free-text fields, and confirmation that outbound integrations in the target environment are disabled or sandboxed.
6. List approvals and access controls needed under the stated obligations, without stating legal conclusions.
7. Do not run any extraction against production; deliver the plan and scripts for the user to review and run.
</task>
<constraints>
- Never move unmasked personal, payment or secret data out of the production boundary in the plan.
- Do not connect to or run queries against production yourself.
- No real data values, keys or connection strings in the output; use placeholders.
- Do not claim the result is anonymous or compliant; describe the residual re-identification risk and who must approve.
{{> output/uncertainty}}
</constraints>
<output_format>
## Data classification
Table: table.column | class | treatment.
## Subset plan
Root entities, sampling rule, included edge cases and the reference graph followed.
## Masking rules
Table: column or pattern | technique | keeps (format, joins, distribution).
## Pipeline
Scripts or code, with where each step runs.
## Checks before release
Checklist.
## Approvals needed
Bullets.
</output_format>可替换内容
- {{database}}
- {{purpose}}
- {{#obligations}}
- {{obligations}}
- {{/obligations}}
- {{> output/uncertainty}}
分类与标签
编程开发、测试数据、数据脱敏、关联抽样、数据流水线、隐私保护