> ## Documentation Index
> Fetch the complete documentation index at: https://docs.profy.cn/llms.txt
> Use this file to discover all available pages before exploring further.
# End-to-end: Batch document processing
> Turn a pile of mixed files into a spreadsheet that computes, with sampling checks and the script threshold
This covers a very common task that's easy to get wrong: you have a batch of files (contracts, invoices, résumés, weekly reports, CSV exports) and you need key fields extracted into a table you can filter and compute on.
There's basically one way to get it wrong: **asking the model to "read each one and summarize."** The first few come out accurate, then fields start going missing, and by the end the numbers are invented — with no way to tell which file it started failing on. This guide's whole point is avoiding that: **have the model write a script rather than read the files itself.**
## What you'll get
| Artifact | Notes |
| ----------------------- | -------------------------------------------------------------------------------------------------------- |
| `output/extracted.xlsx` | One row per file, one column per field, **numeric columns carry formulas rather than hardcoded results** |
| `output/extract.py` | The extraction script itself — reusable, auditable, fixable |
| `output/failures.md` | Every file that couldn't be processed, with the reason |
The third artifact matters as much as the first two. A batch process that only reports successes is one that silently swallows failures.
## The key call: read vs. script
| Situation | Approach | Why |
| --------------------------------------------- | ---------------------------------------- | ------------------------------------------------------------------------------- |
| ≤ 3 files, all different | Let the model read them directly | Script overhead exceeds the benefit |
| ≥ 5 files, consistent format | **Have the model write a script** | Consistency comes from code, immune to attention drift |
| Many files, mixed formats | Group first, one script per group | Let the model do the grouping; scripts do the processing |
| Semantic judgment needed (e.g. contract risk) | Script extracts, model judges row by row | Extraction needs determinism, judgment needs comprehension — keep them separate |
The threshold is around **five files**. It isn't a hard rule, it's an empirical line: past five, manual verification costs enough that you won't actually do it, so the process itself has to be verifiable.
## Prerequisites
Put them in the workspace. Per-file caps: 512 MB for documents, 20 MB for images, 50 MB for code files. Total usage counts against your cloud drive quota (10 GB on Free, more on paid tiers).
Scanned PDFs (image-based) and text PDFs are different animals. The former needs OCR, and accuracy degrades with layout complexity. Have the agent read one at random and confirm text comes out before going further.
List the field names and types (text/number/date). "Have a look and see what fields there are" produces a different field set per file, and you can't assemble a table from that.
## Steps
### Step 1: Sample the structure
Don't start with all of them. Look at three:
```text theme={null}
There are 47 purchase contract PDFs under uploads/. Don't batch-process yet.
Pick 3 at random, read them, and tell me:
1) Whether the structure is consistent (section order, field positions)
2) Where each of these fields lives — contract number, supplier, signing date,
total incl. tax, payment terms — and whether any is simply absent in some files
3) Whether any scanned (image-based) PDFs are mixed in
4) Based on these 3, how you'd approach batch extraction
```
Question 2's "simply absent" clause is the important one. A missing field and a failed extraction are different things and must be distinguished before the script is written.
### Step 2: Have it write a script, not do the work
```text theme={null}
Structure confirmed. Write output/extract.py to do the processing:
- Iterate over every PDF under uploads/
- Extract five fields: contract number, supplier, signing date,
total incl. tax, payment terms
- When a field can't be found, write an empty value and log it to the failure
list — don't skip the whole file and don't guess
- Emit two files:
* output/extracted.csv — one row per file, first column is the source filename
* output/failures.md — per entry: filename, missing fields, reason
Write the script only for now. Show it to me; I'll confirm before you run it.
```
Don't skip "show it to me first." The script is the only thing in this flow you can review in full — glancing at the regexes and field-location logic is much cheaper than reconciling 47 output rows afterwards.
### Step 3: Run five, then run everything
```text theme={null}
Script looks fine. Run it against the first 5 files only,
then print both extracted.csv and failures.md.
```
Verify those five rows against the source files. Continue to the full run only if **all five are correct**. If even one is wrong, fix the script — never hand-edit the output, because a hand-edited row makes you believe the script works.
```text theme={null}
All 5 verified. Run all 47. When done, tell me:
how many succeeded, how many failed, and the most frequent failure reason.
```
### Step 4: Convert to a spreadsheet with formulas
The CSV is an intermediate. The deliverable has to compute:
```text theme={null}
Convert output/extracted.csv to output/extracted.xlsx:
- Freeze the header row, bold the field names
- Format "total incl. tax" as a number with two decimals
- Add a totals row at the bottom using a **SUM formula** — don't write the
computed number in
- Add a "net of tax" column computed by formula from the total (13% tax rate),
again not hardcoded
- Highlight failed rows in yellow
```
"Use formulas, don't hardcode" has to be stated explicitly. The default tends to be writing the computed number straight into the cell — the spreadsheet looks identical, but change any input and the total won't move. This class of error surfaces after you've handed the file to someone else, which is the most expensive time to find it. See [Excel spreadsheets](/en/documentation/capabilities/office-xlsx).
### Step 5: Work the failure list
```text theme={null}
Look at failures.md. Classify the reasons, and for each class tell me:
- whether the script can be fixed to handle it (format variant, regex gap)
- or whether the file itself is the problem (scan, missing page, field truly absent)
- Fix the script for the first kind and re-run just those files
- List the second kind for me to handle manually
```
After this, everything left in the failure list should genuinely require a human. If script-fixable entries remain, the batch isn't finished.
### Step 6: Accept the work
1. Open the xlsx, pick 5 rows at random, verify each field against the source file
2. Change one "total incl. tax" cell and confirm the totals row and net column **move with it**
3. Confirm row count = success count, and success + failures = total file count
4. Open failures.md and confirm each entry has a specific reason, not "parse failed"
Item 3 catches the most problems: if the numbers don't add up, files were dropped silently.
## Boundaries and failure modes
| Symptom | Cause | Fix |
| ----------------------------------------------- | ------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| No text from scanned files | Image-based PDF, needs OCR | Group them separately for OCR; accuracy drops with layout complexity, so sample-verify |
| Only the beginning of a long file is read | The read tool truncates by lines (about 2000 by default) | Have the script read in chunks rather than the model reading in one go |
| Script times out mid-run | Single commands have a default time limit (about 120 seconds) | Run in batches, or have the script checkpoint progress so it can resume |
| Totals don't respond to edits | The result was hardcoded | Explicitly ask for a formula and redo that column |
| Same field formatted inconsistently across rows | The script doesn't normalize | Normalize dates/amounts inside the script rather than having the model fix rows |
| File count doesn't reconcile | Files were skipped silently | Require one output record per input file, success or failure |
| Upload rejected as too large | Over the per-file cap | 512 MB documents, 20 MB images; split before uploading |
| Upload rejected for quota | Cloud drive full | Clear old files or upgrade your plan |
## Variations
Split it in two: the script extracts source passages into a structured CSV, then the model reads those passages row by row and judges (e.g. "is this payment clause risky"). Don't let the model parse the PDF and judge in one pass — when it's wrong you can't tell whether it misread or misjudged.
Connect the Feishu knowledge base and read them directly without downloading. See [Feishu integration](/en/documentation/guides/feishu-integration).
Freeze the finished script and prompt into a scheduled task. Every run bills for real — see [Daily briefing](/en/documentation/guides/daily-briefing).
xlsx is for computing, the web is for reading. See [Publish a site](/en/documentation/guides/publish-a-site).
## Related pages
Formula-first principle and limits
Upload caps and quotas
Parameters and defaults for read/write/exec tools
Where scripts actually run