返回提示词目录第 1292 页

配置数据集与模型版本管理

为项目规划数据与模型存储,生成版本追踪配置、训练流水线及持续集成和团队操作流程。

完整提示词

<context>
Data and model files do not belong in git: they bloat every clone and git cannot diff them meaningfully. DVC keeps the files in remote storage and commits small `.dvc` pointer files (or a `dvc.lock` for pipeline outputs) holding content hashes, so a git commit pins exactly which data and model went with which code. Compared with Git LFS, DVC adds pipelines (`dvc.yaml` stages with declared dependencies, parameters and outputs, re-run only when inputs change), metrics and experiment comparison, and works with ordinary cloud buckets. Common mistakes: running `dvc add` on a folder that is also tracked by git, forgetting to `dvc push` so colleagues get pointer files with no data behind them, storing cloud credentials in `.dvc/config` (which is committed) instead of `.dvc/config.local` or environment variables, adding files one by one instead of whole directories, and keeping raw data that is mutable in place, which breaks reproducibility. CI needs read access to the remote and a cache to avoid pulling gigabytes per run.

<project>
{{project}}
</project>
</context>

<task>
1. Inspect the repository read-only: current data and model files and their sizes, whether any are committed to git or LFS, existing scripts for preparing data and training, and how CI runs.
2. Propose the layout: which directories DVC tracks (raw data, processed data, models), which stay in git (code, small configs, params), and whether raw data is treated as immutable with new versions added rather than edited.
3. Initialise DVC and configure the remote, with credentials kept out of committed config (`.dvc/config.local`, environment variables or the cloud's own credential chain). Show the commands and the resulting committed config.
4. Track the data: `dvc add` on directories, commit the `.dvc` files and updated `.gitignore`. If large files are already committed to git, untrack them from git first and explain whether old history should be cleaned (a separate, confirmed step).
5. If the project has preparation or training steps, turn them into `dvc.yaml` stages with `deps`, `params` and `outs`, so `dvc repro` rebuilds only what changed, and add a metrics file if the project produces metrics.
6. Set up CI: read-only remote credentials from the CI secret store, `dvc pull` of only what the job needs, caching, and optionally `dvc repro` with a check that `dvc.lock` is up to date.
7. Write the daily workflow for the team: after changing data run `dvc add` or `dvc repro`, then `git commit`, then `dvc push` and `git push`; to go back to an old result, `git checkout <commit>` then `dvc checkout`.
8. Run what you can (init, add, status) and report what you verified; leave `dvc push` to the remote for the user to confirm.
</task>

<constraints>
- Never commit credentials; check that `.dvc/config` contains no secrets before committing.
- Do not rewrite git history or delete data files without explicit confirmation.
- Do not push to the remote storage without confirmation.
- State command flags you are unsure of as things to check in the DVC documentation.
</constraints>

<output_format>
## Findings
Current state of data and models.
## Layout
What DVC tracks and what git tracks.
## Setup
Commands run and files created.
## Pipeline
`dvc.yaml` stages, if any.
## CI
Steps and credentials handling.
## Team workflow
The daily commands in order.
## Verified
What was run and what remains for the user.
</output_format>

可替换内容

  • {{project}}

分类与标签

编程开发、数据版本管理、模型管理、训练流水线、持续集成

前往 AI 图片创作