返回提示词目录第 1171 页

构建可恢复的大模型批处理任务

为大量记录编写模型批处理代码,加入断点续跑、输出校验与成本控制,并提供试跑结果和操作手册。

完整提示词

<context>
Running a model over 50,000 rows looks like a for-loop and fails like a distributed system. The loop dies at row 31,000 and restarts from zero, paying twice. Unbounded concurrency hits rate limits and half the rows fail. Malformed outputs are written straight into the results table. Nobody runs a pilot, so a prompt bug is discovered after the whole budget is spent. Provider batch APIs cut cost substantially and remove rate-limit pain for work that can wait hours, but they have their own rules: request file formats, size limits, a completion window, per-request custom ids for matching results back, and partial failures to resubmit.
</context>

<task>
Build a batch job for: {{job}}, using {{provider_sdk}}.{{#limits}} Limits: {{limits}}.{{/limits}}

1. Read the existing per-item prompt, output schema and data access code in the repository. If there is no output schema, define one and validate against it.
2. Choose the execution mode and say why: the provider's batch API when the deadline allows and it is permitted, otherwise a concurrent runner with a bounded worker pool sized to the rate limits, with backoff on 429.
3. Make every record idempotent: a stable custom id per record, a results store keyed by that id with status (pending, done, failed, invalid) and the prompt version, so reruns skip done records and a changed prompt version can be rerun deliberately.
4. Checkpoint and resume: persist progress so the job restarts where it stopped. For the batch API, store batch ids and poll; on completion, map results back by custom id and resubmit failed or expired items.
5. Validate each output against the schema; store invalid outputs separately with the raw text and reason, retry them once, then leave them for review. Never overwrite source data in place.
6. Cost control: estimate cost before starting from token counts of a sample, print it, require a confirmation flag to run the full job, and stop when the running cost passes the cap.
7. Pilot: a `--limit` option; run it on 20 to 50 representative records, show the outputs and the invalid rate, and stop there for the user to check quality before the full run.
8. Report at the end: counts by status, tokens and cost, invalid examples, and how to rerun failures.
</task>

<constraints>
- Do not start the full run yourself; run only the pilot and leave the full run to the user's explicit go-ahead.
- Do not send data to the provider that the user has not said may be sent; ask if the records contain personal or confidential data.
- Use the real batch API request format for the stated SDK version; if unsure of a field, say so and point to the documentation.
- No orchestration framework unless the project already uses one.
{{> guardrails/verify-before-done}}
</constraints>

<output_format>
## Plan
Mode, concurrency or batch size, idempotency key, storage, in bullets.
## Code
Code blocks with file paths.
## Pilot
Command, real results and sample outputs, or a plain statement that it was not run.
## Cost estimate
The arithmetic from sample token counts, with prices as given or as symbols.
## Runbook
How to start, monitor, resume, rerun failures and change prompt versions.
## Open questions
Numbered, or "None".
</output_format>

可替换内容

  • {{job}}
  • {{provider_sdk}}
  • {{#limits}}
  • {{limits}}
  • {{/limits}}
  • {{> guardrails/verify-before-done}}

分类与标签

编程开发、模型批处理、断点续跑、输出校验、成本控制、接口调用

前往 AI 图片创作