gitea-impl-agent/jarca-ledger-mcp
1
0
Fork 0
forked from jsfamily/jarca-ledger-mcp
Code Pull Requests Activity

refactor(jarca-ledger): implement BAT-010 backend core layering

Browse Source
This commit is contained in:
Jarca
2026-07-03 07:03:17 +09:00
parent 02380a88ab
commit 7ac25a6cf7
36 changed files with 4892 additions and 4113 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/runtime-flows.md
+53 -69
View File
@@ -4,19 +4,19 @@
- Baseline Seed: `.stage-pilot/bootstrap/baseline.yaml`
- Status: draft
- Owner: Jarca / project maintainer
- Last Updated (KST): 2026-06-30
- Source Discovery / Batch: bootstrap-baseline, BAT-006, BAT-007, BAT-008
- Last Updated (KST): 2026-07-03
- Source Discovery / Batch: bootstrap-baseline, BAT-006, BAT-007, BAT-008, BAT-010
- Primary Runtime: mixed
## Project Summary
- Jarca 가계부 전용 MCP 서버와 Streamlit 대시보드로 계좌/카드 거래, 예산, 분류 규칙, import, 현금흐름 분석을 SQLite에 저장하고 운영한다.
- Jarca 가계부 전용 MCP 서버와 Streamlit 대시보드로 계좌/카드 거래, 예산, 분류 규칙, import, 현금흐름 분석, 자산/스냅샷 운영을 SQLite에 저장하고 제공한다.
## Purpose
- This document is the current execution-flow baseline for the project.
- It is a cross-cutting reference document for Discovery, REQ, Batch Design, Batch Verification, and implementation updates.
## Scope
- Covers representative command, MCP, dashboard, import, and reconciliation flows.
- Covers representative command, MCP, dashboard, import, reconciliation, snapshot, and verification flows.
- Does not duplicate full acceptance criteria or low-level implementation details.
## Covered Entry Points
@@ -27,56 +27,48 @@
## Shared Components
- SQLite database at `~/.local/share/jarca/ledger.sqlite3` by default.
- `jarca_ledger_mcp.service` as the central domain/service layer.
- `jarca_ledger_mcp/service.py` compatibility facade over application modules.
- `jarca_ledger_mcp/infrastructure/sqlite/*` for DB path resolution, schema, migration, and connection lifecycle.
- `jarca_ledger_mcp/adapters/mcp/server.py` for MCP registration.
- `jarca_ledger_mcp/adapters/streamlit/service_gateway.py` for dashboard-facing service calls.
- Parser modules for KB/Toss/SeoulPay sources.
- Category and `cashflow_kind` semantics shared by reports and UI.
## Flow: `MCP server tool call`
1. Hermes MCP client invokes a tool exposed by `jarca_ledger_mcp.server`.
2. `server.py` forwards arguments to the matching `service.py` function.
3. `service.py` opens the SQLite database path from env/default, validates inputs, and performs the operation.
4. Service returns a dictionary payload with success data or error information.
5. Hermes surfaces the tool result to the user or uses it in a follow-up workflow.
1. Hermes MCP client invokes a tool exposed by `jarca_ledger_mcp.server` / `jarca_ledger_mcp.adapters.mcp.server`.
2. MCP adapter forwards arguments to the matching `jarca_ledger_mcp.service` facade function.
3. Service facade routes into the relevant application module.
4. Application code opens SQLite through `infrastructure/sqlite` helpers, validates inputs, and performs the operation.
5. Application returns a dictionary payload with success data or error information.
6. Hermes surfaces the tool result to the user or uses it in a follow-up workflow.
## Flow: `Streamlit dashboard operation`
1. Operator starts `streamlit run dashboard/app.py --server.address 0.0.0.0 --server.port 8501`.
2. Dashboard reads the configured/default SQLite database.
3. User navigates to 요약, 거래내역, 확인필요 큐, 미분류, 분류 규칙, 카테고리, 계좌, or Import pages.
4. UI calls service functions for reads, updates, imports, categorization, and manual linking.
5. UI displays success/error status and refreshes visible tables.
3. User navigates to 요약, 자산/마감, 거래내역, 확인필요 큐, 미분류, 분류 규칙, 카테고리, 계좌, or Import pages.
4. UI calls `jarca_ledger_mcp.adapters.streamlit.service_gateway` rather than giant implementation modules directly.
5. Gateway delegates to the service facade/application modules and returns data to the page.
6. UI displays success/error status and refreshes visible tables.
## Flow: `Dashboard operations hub overview`
1. Operator opens the `요약` page.
2. Dashboard resolves the KST current month and calls `get_monthly_cashflow_report(month)` using `transactions.transaction_date` and `ledger_categories.cashflow_kind` semantics.
2. Dashboard calls `get_monthly_cashflow_report(month)` using `transactions.transaction_date` and `ledger_categories.cashflow_kind` semantics.
3. Dashboard calls `get_asset_liquidity_report`, which uses the shared valuation helper: account-backed assets from `accounts.last_balance` plus active `manual_assets` rows when present.
4. If `manual_assets` and `asset_snapshots` are empty, the report keeps the REQ-005 account-only fallback and preserves existing totals/keys.
5. Dashboard renders sections in REQ-005 priority order: this-month actual income/expense/net/savings rate, asset liquidity, asset composition, budget/spend, operational warnings.
6. Missing `last_balance` or `manual_assets.valuation_amount` values are excluded from asset totals and surfaced as warnings; transaction sums are not used to estimate non-transaction assets.
6. Missing `last_balance` or `manual_assets.valuation_amount` values are excluded from asset totals and surfaced as warnings.
## Flow: `CI validation gates`
1. GitHub Actions starts `.github/workflows/ci.yml` on `pull_request`, `push` to `main`, or `workflow_dispatch`.
2. `lint` installs dev dependencies and runs `ruff check .` with the conservative BAT-007 required rule set.
3. `contract-regression` installs dev dependencies and runs `pytest tests/test_contract_regression.py`; the test uses synthetic pytest temp DBs and `tests/contract_snapshots/response_contract_v1_0.json` metadata for health/status, list, report, and mutation success envelope scopes.
4. `pytest-regression` installs the project with dev/dashboard extras and runs `pytest --ignore=tests/test_dashboard_app.py` so service/parser/regression failures fail the workflow.
5. `dashboard-app-test` runs `pytest tests/test_dashboard_app.py`; the AppTest test creates a synthetic fixture DB through `scripts/ci_fixture_db.py`, sets `JARCA_LEDGER_DB_PATH`, and asserts core dashboard pages render without Streamlit exceptions.
6. `dashboard-http-smoke` runs `scripts/ci_dashboard_http_smoke.py`; the wrapper creates a temp fixture DB, starts Streamlit on loopback, polls for HTTP 200, then terminates the process.
7. `db-safety-smoke` runs `scripts/ci_db_safety_smoke.py`; the wrapper creates temp source/backup/target DB files and invokes `scripts/ledger_restore_smoke.py` without touching the operating DB path.
8. Any required job failure fails the workflow. Stable required-check job names are `pytest-regression`, `dashboard-app-test`, `dashboard-http-smoke`, `db-safety-smoke`, `contract-regression`, and `lint`; actual branch protection toggle enforcement is configured outside the repository.
9. CI logs are status-oriented and synthetic-data-only; they must not contain real account identifiers, raw payloads, statement filenames, balances from the operating DB, or passwords.
## Flow: `Asset snapshot and net-worth read model`
1. Month-end manual close records item-level rows in `asset_snapshots` from the live authority sources: account `last_balance` rows and `manual_assets` valuations. BAT-008 stores the schema and service helper but does not expose snapshot creation as an MCP mutation.
2. `get_asset_snapshot_report(basis_date)` first checks for snapshot rows on the requested basis date.
3. If snapshot rows exist, the report computes bucket totals, total assets, total liabilities, and net worth from the stored rows.
4. If no snapshot rows exist, the report falls back to live authority sources (`accounts.last_balance` + active `manual_assets`) and marks `source='live_authority'`.
5. `get_net_worth_timeseries(start_date, end_date, months)` reads distinct stored snapshot basis dates and applies the same asset-buckets-minus-loan formula to every point.
6. `list_manual_assets` exposes current manual asset/liability rows for read-only MCP consumers. Manual asset CRUD and snapshot creation MCP tools remain future scope.
## Flow: `Asset snapshot and net-worth workflow`
1. Operator can open `자산/마감` in Streamlit or call snapshot tools through MCP.
2. `preview_asset_snapshot_close(basis_date)` computes authority-source items from `accounts.last_balance` + active `manual_assets` without DB side effects.
3. `close_asset_snapshot(basis_date, source_note, memo, correction_reason)` writes append-only snapshot rows with correction metadata when re-closing the same basis date.
4. `get_asset_snapshot_report(basis_date)` first checks stored snapshot rows; if none exist, it falls back to live authority sources and marks `source='live_authority'`.
5. `get_net_worth_timeseries(start_date, end_date, months)` reads stored snapshot basis dates and applies the same asset-buckets-minus-loan formula to every point.
## Flow: `Import preview and import`
1. Operator chooses or supplies a supported source file: KB Bank PDF, Toss Bank PDF, Toss Securities PDF, SeoulPay XLSX, or CSV.
1. Operator chooses a supported source file: KB Bank PDF, Toss Bank PDF, Toss Securities PDF, SeoulPay XLSX, or CSV.
2. Preview function parses the file and reports counts, sample rows, duplicate status, and classification preview.
3. On explicit import, service creates/fetches the target account, inserts non-duplicate transactions, writes raw provenance, and records an import batch.
4. Reports and dashboard views read the inserted rows using category and cashflow semantics.
3. On explicit import, application code creates/fetches the target account, inserts non-duplicate transactions, writes provenance, and records an import batch.
4. Reports and dashboard views read the inserted rows using shared category/cashflow semantics.
5. If an import is wrong, rollback uses import batch data and soft-deletes by default.
## Flow: `Categorization rule lifecycle`
@@ -86,7 +78,7 @@
4. Category `cashflow_kind` controls actual cashflow/report inclusion.
## Flow: `Cross-account and cancel/original reconciliation`
1. Preview functions search candidate pairs by amount, date, direction, account/source compatibility, and text similarity.
1. Preview helpers search candidate pairs by amount, date, direction, account/source compatibility, and text similarity.
2. Operator reviews candidates in the dashboard or runs the automatic matcher where confidence rules are satisfied.
3. Matched self-transfers become `이체 / 본인계좌이동`; investment movements become `이체 / 투자이동`; ambiguous rows remain `확인필요`.
4. Cancel/refund rows can be linked to original expense rows; memo tags preserve manual link evidence.
@@ -97,26 +89,23 @@
3. Risky import, rollback, categorization apply, reconciliation, or migration work is tested first against a synthetic fixture DB or repo-external validation DB copy selected through `JARCA_LEDGER_DB_PATH`.
4. Backup list/status/hash checks use `list_ledger_backups` and `get_ledger_backup_status`.
5. Restore execution is not an MCP flow. Restore smoke uses `scripts/ledger_restore_smoke.py` or the runbook and requires explicit approval before overwriting an existing target DB.
6. Verification evidence must not include full transaction dumps, full merchant/counterparty dumps, account identifiers, raw_payload, original financial file contents, or passwords.
## Flow: `Statement-unavailable manual balance reconciliation`
1. Operator follows the dcy-002 safety standard first: capture DB status, create or confirm a recent backup, and run preview/reconcile on a validation DB before operating on the production DB.
2. Operator marks accounts with `statement_status`: `manual_only` for statement-unavailable accounts, `partial_import` for accounts that require manual supplementation, or `import_supported` where normal statement import is sufficient.
3. Operator records manual evidence transactions with `source='manual'`, positive amount, date, account, and non-sensitive memo/source context.
4. Operator runs `get_balance_reconciliation_report` from MCP/service or the dashboard account page. If `baseline_at` is `YYYY-MM`, the snapshot time is that month end at 23:59 KST; ad-hoc reconciliation uses the user-supplied snapshot timestamp.
5. The report compares `baseline_balance`/`last_balance` with the latest `balance_after` if available, otherwise with net signed account transactions. Absolute tolerance is 1,000 KRW; 0.1% of baseline is reference-only.
6. `OK` rows need no immediate triage. `REVIEW_NEEDED` rows present manual decision candidates: `계좌이동`, `실지출`, `실수입`, `투자이동`, `분석제외`, `확인필요`.
7. This flow does not run automatic scheduling, notifications, restore, or auto-confirming mutation.
## Flow: `CI validation gates`
1. GitHub Actions starts `.github/workflows/ci.yml` on `pull_request`, `push` to `main`, or `workflow_dispatch`.
2. `lint` runs `ruff check .` with the conservative required rule set.
3. `contract-regression` runs `pytest tests/test_contract_regression.py` with synthetic pytest temp DBs and committed contract snapshots.
4. `pytest-regression` runs the non-dashboard regression suite.
5. `dashboard-app-test` runs `pytest tests/test_dashboard_app.py` using `scripts/ci_fixture_db.py` synthetic DB setup.
6. `dashboard-http-smoke` starts Streamlit against a synthetic DB and polls readiness.
7. `db-safety-smoke` runs restore smoke through temp source/backup/target DB paths only.
8. Any required job failure fails the workflow. Stable required-check job names remain `pytest-regression`, `dashboard-app-test`, `dashboard-http-smoke`, `db-safety-smoke`, `contract-regression`, and `lint`.
## Flow: `Streamlit browser evidence capture`
1. Operator runs `scripts/streamlit_browser_evidence.py` from the repository root.
2. The script creates a synthetic fixture DB under the evidence output directory and refuses to use the default operating DB path.
3. Streamlit starts with `JARCA_LEDGER_DB_PATH=<fixture.sqlite3>` on a local loopback port.
4. `/usr/bin/google-chrome-stable` runs in headless mode through Chrome DevTools Protocol and opens target dashboard pages such as `분류 규칙`, `확인필요 큐`, and `Import`.
5. After page title text is rendered, the script stores PNG screenshots, DOM snapshots, and `manifest.json` in the evidence directory.
6. UI/dashboard-impacting batches must attach browser screenshot/snapshot evidence. Non-UI batches may treat it as optional, and docs-only batches do not require browser evidence.
7. If Chrome/browser capture is unavailable, AppTest + HTTP 200 fallback evidence is allowed only as `PASS_WITH_NOTES` with the environment failure and remaining browser-evidence item recorded.
8. Browser screenshot/DOM evidence is not a required GitHub Actions gate in BAT-006. Run it locally for visual/layout-sensitive dashboard changes, while non-visual backend/CI changes can rely on pytest/AppTest/HTTP/DB safety gates.
1. Operator runs `scripts/streamlit_browser_evidence.py` or starts Streamlit against a synthetic fixture DB.
2. The script/flow refuses to use the default operating DB path for evidence capture.
3. Browser smoke opens key pages such as `요약`, `자산/마감`, `분류 규칙`, `확인필요 큐`, and `Import`.
4. Verification records rendered headings/content and browser console JS error absence.
5. UI/dashboard-impacting batches must attach browser screenshot/DOM evidence or equivalent browser smoke notes.
## Flow Constraints
- Preview before broad import or categorization mutation where supported.
@@ -125,20 +114,14 @@
- Avoid destructive migrations without a backup/approval plan.
- Before broad DB mutation, capture DB status evidence and create or confirm a recent backup.
- Restore execution is script/runbook only and must not be exposed as an MCP tool.
- Manual balance reconciliation must remain preview/report-first unless a future REQ approves persistent review routing and confirmation mutation.
- Streamlit browser evidence must use synthetic/validation DBs only and must not screenshot/log real account numbers, balances, source financial filenames, passwords, full merchant/counterparty dumps, or raw payloads.
- CI gates must use synthetic/temp DBs only. The default operating DB path (`~/.local/share/jarca/ledger.sqlite3`) is forbidden for AppTest, HTTP smoke, and DB safety smoke wrappers.
- Contract regression must use synthetic/temp DBs only and must not expose runtime contract-version fields in MCP success payloads unless a future approved REQ changes the policy.
- Branch protection required-check drift should be audited against the six stable workflow job names documented in `docs/interface-contract.md`; changing a job name requires an explicit REQ/BAT note because it can break external repository settings.
- Dashboard and MCP asset valuation reports must use the shared service valuation policy: `accounts.last_balance` plus `manual_assets`, asset buckets minus loan, and missing valuation exclusion.
- Snapshot read tools are read-only in BAT-008. Snapshot creation and manual asset CRUD MCP mutation tools require a future REQ.
- Sensitive passwords are runtime inputs and must not be committed.
- Streamlit browser evidence must use synthetic/validation DBs only and must not screenshot/log real account numbers, source financial filenames, passwords, full merchant/counterparty dumps, or raw payloads.
- CI gates must use synthetic/temp DBs only. The default operating DB path is forbidden for AppTest, HTTP smoke, and DB safety smoke wrappers.
- Top-level `service.py`, `db.py`, and `server.py` import paths remain compatibility surfaces until a future approved REQ retires them.
## Current Gaps / Planned Changes
- Browser-level evidence is available locally through `scripts/streamlit_browser_evidence.py`; it remains outside required CI gates by design.
- Contract-regression and lint gates are now part of the required-check contract for CI; full MCP tool-surface snapshot coverage remains deferred beyond the representative health/status/list/report/mutation scope.
- Dedicated reconciliation workbench and persistent review-decision queue are deferred beyond REQ-003.
- Separate asset snapshot/manual asset tables and automatic market/real-estate valuation are deferred beyond REQ-005.
- Browser-level evidence exists locally and remains outside required CI gates by design.
- Phase 4+ write/import/advanced workflow service thinning and narrower adapters are deferred to GitHub Project TODO items.
- HTTP adapter and remote ops re-architecture remain deferred.
## Update Triggers
- Update this document when a command entry point changes.
@@ -151,5 +134,6 @@
- 2026-06-29 KST: REQ-003/BAT-003 기준으로 statement-unavailable 수동 잔액대조 flow 추가.
- 2026-06-29 KST: REQ-004/BAT-004 기준으로 Streamlit browser evidence capture flow 및 UI evidence gate 기준 추가.
- 2026-06-29 KST: REQ-005/BAT-005 기준으로 dashboard operations hub 및 `liquidity_bucket`/`last_balance` 자산 유동성 flow 추가.
- 2026-06-30 KST: REQ-006/BAT-006 기준으로 GitHub Actions CI validation gates, synthetic fixture DB boundary, AppTest/HTTP/DB safety smoke flow, local browser evidence boundary 추가.
- 2026-06-30 KST: REQ-007/REQ-008/BAT-007 기준으로 `contract-regression`/`lint` CI required-check flow, contract snapshot scope, branch protection drift boundary 추가.
- 2026-06-30 KST: REQ-006/BAT-006 기준으로 GitHub Actions CI validation gates, synthetic fixture DB boundary, AppTest/HTTP/DB safety smoke flow 추가.
- 2026-06-30 KST: REQ-007/REQ-008/BAT-007 기준으로 contract-regression/lint CI required-check flow 추가.
- 2026-07-03 KST: REQ-013/REQ-014/BAT-010 기준으로 service facade → application, db shim → infrastructure/sqlite, MCP/Streamlit adapter seam flow를 반영했다.