feat: enforce import provenance exposure policy (#2)
Merge BAT-017 / REQ-021 implementation after delivery-runner implementation, independent QC, and passing GitHub Actions CI.
This commit is contained in:
+111
-4
@@ -1,16 +1,123 @@
|
||||
# BAT-017 Implementation Evidence
|
||||
|
||||
- Status: not-started
|
||||
- Status: implemented-awaiting-qc
|
||||
- Created At (KST): 2026-07-08 11:33 KST
|
||||
- Last Updated (KST): 2026-07-09T13:16:10+09:00
|
||||
|
||||
## Implementation Summary
|
||||
|
||||
Pending.
|
||||
### Slice 01 — provenance foundation, import preview/write sanitization
|
||||
- Added a shared provenance policy module for source-hash labeling, allowlisted raw-payload JSON serialization, and CSV preview column filtering.
|
||||
- Stopped new import preview/import write paths from exposing or persisting full local source paths for:
|
||||
- KB Bank PDF
|
||||
- Toss Bank PDF
|
||||
- Toss Securities PDF
|
||||
- SeoulPay XLSX
|
||||
- CSV preview
|
||||
- Changed `import_batches.filename` new writes to `masked-basename#source_hash_prefix`.
|
||||
- Minimized `transactions.raw_payload` for current supported import sources through explicit allowlists.
|
||||
- Added targeted regression tests for path redaction and forbidden-field non-leak assertions.
|
||||
|
||||
## Changed Files
|
||||
- `jarca_ledger_mcp/application/imports_service.py`
|
||||
- `jarca_ledger_mcp/kb_bank_pdf.py`
|
||||
- `jarca_ledger_mcp/seoulpay_xlsx.py`
|
||||
- `jarca_ledger_mcp/toss_bank_pdf.py`
|
||||
- `jarca_ledger_mcp/toss_securities_pdf.py`
|
||||
- `jarca_ledger_mcp/provenance_policy.py`
|
||||
- `tests/test_import_provenance_policy.py`
|
||||
|
||||
Pending.
|
||||
## Execution Evidence
|
||||
- `python -m pytest -q tests/test_import_provenance_policy.py`
|
||||
- result: pass (`.....`)
|
||||
- `python -m ruff check jarca_ledger_mcp/application/imports_service.py jarca_ledger_mcp/kb_bank_pdf.py jarca_ledger_mcp/seoulpay_xlsx.py jarca_ledger_mcp/toss_bank_pdf.py jarca_ledger_mcp/toss_securities_pdf.py jarca_ledger_mcp/provenance_policy.py tests/test_import_provenance_policy.py`
|
||||
- result: pass (`All checks passed!`)
|
||||
- `pytest -q tests/test_import_provenance_policy.py tests/test_kb_bank_pdf.py tests/test_seoulpay_xlsx.py tests/test_toss_securities_pdf.py`
|
||||
- result: pass via `dev-impl`
|
||||
- `pytest -q tests/test_kb_bank_pdf.py tests/test_seoulpay_xlsx.py tests/test_toss_securities_pdf.py tests/test_service.py`
|
||||
- result: pass via `dev-impl`
|
||||
- `pytest -q tests/test_import_provenance_policy.py tests/test_kb_bank_pdf.py tests/test_seoulpay_xlsx.py tests/test_toss_securities_pdf.py tests/test_service.py`
|
||||
- result: pass via `dev-impl`
|
||||
- `pytest -q`
|
||||
- result: pass via `dev-impl`
|
||||
|
||||
## Evidence Notes
|
||||
- Targeted regression confirms preview responses no longer emit full absolute paths.
|
||||
- CSV preview regression confirms arbitrary raw columns such as `secret_note` and `local_path` are suppressed by default.
|
||||
- Import persistence regression confirms `import_batches.filename` stores masked basename plus source-hash prefix rather than full paths.
|
||||
- Import persistence regression confirms `transactions.raw_payload` stores only the source-specific allowlisted minimum fields for the implemented sources.
|
||||
|
||||
## Residual Risks / Follow-up
|
||||
- Slice 01 does not yet complete the MCP provenance exposure contract (`minimal` / `operator-summary` / `debug-gated`).
|
||||
- Import/read surfaces beyond the new-write preview/import boundary still need review for finance-agent privacy alignment.
|
||||
- Baseline docs still need BAT-017-specific updates once the exposure contract and final runtime boundary are settled.
|
||||
- Existing operating DB sanitization remains intentionally out of scope and must stay in the separate follow-up Todo.
|
||||
|
||||
## Notes
|
||||
- Implementation must not use Kanban unless the user explicitly re-allows it.
|
||||
|
||||
Implementation must not use Kanban unless the user explicitly re-allows it.
|
||||
### Slice 02A — provenance exposure helper + tests
|
||||
- Added explicit provenance exposure constants and helper APIs in `jarca_ledger_mcp/provenance_policy.py` for `minimal`, `operator-summary`, and `debug-gated`.
|
||||
- Added sanitizer behavior that keeps normalized provenance fields while suppressing full paths, raw document/raw text dumps, passwords, tokens, secrets, and non-allowlisted raw payload keys.
|
||||
- Added alias helpers `sanitize_provenance_envelope(...)` and `sanitize_provenance_for_exposure(...)` for bounded downstream adoption.
|
||||
- Added helper-level regression coverage in `tests/test_import_provenance_policy.py` for exposure-level behavior and forbidden-field suppression.
|
||||
|
||||
## Additional Changed Files (Slice 02A)
|
||||
- `jarca_ledger_mcp/provenance_policy.py`
|
||||
- `tests/test_import_provenance_policy.py`
|
||||
|
||||
## Additional Execution Evidence (Slice 02A)
|
||||
- Supervised impl child result
|
||||
- artifact: `.stagepilot/worker-supervision/impl-20260709T030219Z/final-result.json`
|
||||
- result: `done` (`child_exit_code=0`)
|
||||
- `python -m pytest -q tests/test_import_provenance_policy.py`
|
||||
- result: pass (`7 passed`)
|
||||
- `python -m py_compile jarca_ledger_mcp/provenance_policy.py tests/test_import_provenance_policy.py`
|
||||
- result: pass
|
||||
- `python -m ruff check jarca_ledger_mcp/provenance_policy.py tests/test_import_provenance_policy.py`
|
||||
- result: initially blocked in runner environment (`No module named ruff` from current Python)
|
||||
- Lead follow-up verification after environment check:
|
||||
- `. .venv/bin/activate && python -m ruff check jarca_ledger_mcp/provenance_policy.py tests/test_import_provenance_policy.py`
|
||||
- result: pass (`All checks passed!`)
|
||||
- `. .venv/bin/activate && python -m pytest -q tests/test_import_provenance_policy.py`
|
||||
- result: pass (`7 passed`)
|
||||
- `. .venv/bin/activate && python -m py_compile jarca_ledger_mcp/provenance_policy.py tests/test_import_provenance_policy.py`
|
||||
- result: pass
|
||||
- `. .venv/bin/activate && python -m ruff check .`
|
||||
- result: pass (`All checks passed!`)
|
||||
- `. .venv/bin/activate && python -m pytest -q`
|
||||
- result: pass (`55 passed, 4 skipped`)
|
||||
|
||||
## Slice 02B — import batch read-surface wiring + baseline docs
|
||||
- Wired `list_ledger_import_batches(...)` through the BAT-017 provenance sanitizer so default responses now return `minimal` provenance and scrub the raw `memo` field from user-facing payloads.
|
||||
- Added bounded operator-summary support for import batch listing so masked filename, source-hash prefix, sheet reference, and validation summary are available only when explicitly requested.
|
||||
- Added MCP adapter pass-through for `provenance_exposure` on `list_ledger_import_batches(...)`.
|
||||
- Added explicit `debug_provenance_gate=True` enforcement before `debug-gated` provenance can be requested from the import batch listing surface.
|
||||
- Extended provenance regression coverage for:
|
||||
- default/minimal import batch responses
|
||||
- explicit operator-summary import batch responses
|
||||
- debug-gated access denial without the explicit gate and allow-path with the gate
|
||||
- MCP adapter pass-through of the exposure level
|
||||
- Updated baseline docs (`docs/data-model.md`, `docs/project-structure.md`, `docs/runtime-flows.md`) to record the provenance envelope boundary, `raw_payload=minimize`, masked filename policy, and read-time exposure split.
|
||||
|
||||
## Additional Changed Files (Slice 02B)
|
||||
- `docs/data-model.md`
|
||||
- `docs/project-structure.md`
|
||||
- `docs/runtime-flows.md`
|
||||
- `jarca_ledger_mcp/adapters/mcp/server.py`
|
||||
- `jarca_ledger_mcp/application/imports_service.py`
|
||||
- `tests/test_import_provenance_policy.py`
|
||||
|
||||
## Additional Execution Evidence (Slice 02B)
|
||||
- `. .venv/bin/activate && python -m ruff check jarca_ledger_mcp/application/imports_service.py jarca_ledger_mcp/adapters/mcp/server.py tests/test_import_provenance_policy.py`
|
||||
- result: pass (`All checks passed!`)
|
||||
- `. .venv/bin/activate && python -m pytest -q tests/test_import_provenance_policy.py`
|
||||
- result: pass (`11 passed`)
|
||||
- `. .venv/bin/activate && python -m ruff check .`
|
||||
- result: pass (`All checks passed!`)
|
||||
- `. .venv/bin/activate && python -m pytest -q`
|
||||
- result: pass (`59 passed, 4 skipped`)
|
||||
|
||||
## Updated Residual Risks / Follow-up
|
||||
- Independent QC, PR creation/push, GitHub CI, and post-merge `confirm-req-implemented` remain pending.
|
||||
- StagePilot doctor entrypoint is not present in the current workspace/runtime; this should be classified during QC as environment/tooling debt unless the lead requires a separate setup action before closure.
|
||||
|
||||
+17
-2
@@ -1,11 +1,20 @@
|
||||
# BAT-017 Verification Evidence
|
||||
|
||||
- Status: not-started
|
||||
- Status: ready-for-qc
|
||||
- Created At (KST): 2026-07-08 11:33 KST
|
||||
|
||||
## Verification Commands
|
||||
|
||||
Pending.
|
||||
- `. .venv/bin/activate && python -m ruff check jarca_ledger_mcp/application/imports_service.py jarca_ledger_mcp/adapters/mcp/server.py tests/test_import_provenance_policy.py`
|
||||
- result: pass (`All checks passed!`)
|
||||
- `. .venv/bin/activate && python -m pytest -q tests/test_import_provenance_policy.py`
|
||||
- result: pass (`11 passed`)
|
||||
- `. .venv/bin/activate && python -m ruff check .`
|
||||
- result: pass (`All checks passed!`)
|
||||
- `. .venv/bin/activate && python -m pytest -q`
|
||||
- result: pass (`59 passed, 4 skipped`)
|
||||
- StagePilot doctor
|
||||
- result: blocked in current workspace/runtime because no discoverable `stagepilot-doctor` entrypoint or package was present; needs QC classification as pre-existing environment/tooling debt vs. closure blocker.
|
||||
|
||||
## Expected Gates
|
||||
|
||||
@@ -16,3 +25,9 @@ Pending.
|
||||
- fixture/log sensitive-field scan
|
||||
- affected dashboard smoke, if applicable
|
||||
- StagePilot doctor with pre-existing debt separated from BAT-017 findings
|
||||
|
||||
## Batch-specific verification notes
|
||||
- Full pytest includes the affected import/privacy regression suite and leaves 4 skips in place without introducing new failures.
|
||||
- Import batch debug-gated exposure now requires explicit `debug_provenance_gate=True`; the regression suite covers both deny-without-gate and allow-with-gate behavior.
|
||||
- BAT-017 touched MCP import batch exposure behavior but did not modify dashboard code paths directly; no additional dashboard-only code change was required in this slice.
|
||||
- QC still needs to independently inspect the changed surfaces, validate the documentation updates, and classify the missing StagePilot doctor entrypoint.
|
||||
|
||||
Reference in New Issue
Block a user