jarca-ledger-mcp/docs/data-model.md

# Jarca Ledger MCP Data Model

- Baseline Mode: observed
- Baseline Seed: `.stage-pilot/bootstrap/baseline.yaml`
- Status: draft
- Owner: Jarca / project maintainer
- Last Updated (KST): 2026-07-09
- Source Discovery / Batch: bootstrap-baseline, BAT-008, BAT-009, REQ-021/BAT-017

## Project Summary
- Jarca 가계부 전용 MCP 서버와 Streamlit 대시보드로 계좌/카드 거래, 예산, 분류 규칙, import, 현금흐름 분석을 SQLite에 저장하고 운영한다.

## Purpose
- This document captures the current data-model baseline for future Discovery, REQ, Batch, and Release work.
- It focuses on durable entities, state rules, consistency rules, and persistence boundaries rather than implementation-only details.

## Persistence Backend
- SQLite database at `~/.local/share/jarca/ledger.sqlite3` by default.
- Override path through `JARCA_LEDGER_DB_PATH`.
- Schema initialization and migrations are managed by `jarca_ledger_mcp/db.py`.

## Entity: LedgerAccount
- Source of Truth: `ledger_accounts`
- Purpose: Represents bank, card, cash, investment, or loan accounts used to attach transaction evidence and balances.
- Lifecycle: created -> updated -> inactive/archived
- Key Fields:
  - `id` :: text :: stable account identifier
  - `name` :: text :: operator-facing account name
  - `type` :: text :: bank/card/cash/investment/loan classification
  - `institution` :: text :: financial institution or source
  - `account_number_masked` :: text :: masked account identifier
  - `holder_name` :: text :: owner context for transfer matching
  - `last_balance` :: integer :: latest known balance when available
  - `last_synced_at` :: text :: 기준 잔액 스냅샷 시각. 정기 대조는 월말 23:59 KST, 수시 대조는 사용자 입력 시각
  - `statement_status` :: text :: `unknown`, `import_supported`, `manual_only`, `partial_import`
  - `liquidity_bucket` :: text :: `cash`, `investment`, `deposit`, `real_estate`, `loan`, `other`; 자산 유동성/구성/순자산 집계의 계좌 단위 권위 소스
- Relationships:
  - `transactions.account_id` links rows to accounts.
- State Rules:
  - `active=false` hides accounts from default lists without deleting history.
  - `manual_only` is the statement-unavailable class; `partial_import` remains eligible for manual supplementation and balance reconciliation.
  - Balance reconciliation is report/preview first: compare `baseline_balance`/`baseline_at` against the latest `balance_after` or net account transaction delta, with 1,000 KRW absolute tolerance.
  - 계좌 기반 자산/부채는 `last_balance`를 현재 평가 권위 소스로 사용하고, `last_synced_at`을 기준일, `memo`를 출처 메모로 보존한다.
  - 거래로 산출되지 않는 비거래 자산/부채는 `ManualAsset`에 별도 저장한다. 계좌와 수기 자산을 같은 항목으로 중복 등록하지 않는다.
  - `liquidity_bucket='loan'`은 자산 구성 비율 분모에서 제외하고 총부채/순자산 산식에만 반영한다. `last_balance`가 없는 계좌/자산은 임의 거래 합계로 추정하지 않는다.

## Entity: ManualAsset
- Source of Truth: `manual_assets`
- Purpose: Stores non-transaction-derived assets/liabilities such as real estate, deposits, loans, and other manually valued holdings that should not be mixed into account rows.
- Lifecycle: created -> revalued/updated -> inactive/archived
- Key Fields:
  - `id` :: text :: stable manual asset identifier
  - `name` :: text :: operator-facing asset/liability name
  - `liquidity_bucket` :: text :: `cash`, `investment`, `deposit`, `real_estate`, `loan`, `other`
  - `valuation_amount` :: integer nullable :: latest manually entered valuation/balance; null means excluded from valuation totals
  - `valuation_date` :: date text :: valuation basis date
  - `currency` :: text :: default `KRW`
  - `source_note` :: text :: valuation evidence/source memo
  - `memo` :: text :: operator note
  - `active` :: integer/bool :: inactive rows are excluded from default read reports
- Relationships:
  - Referenced by `asset_snapshots.source_type='manual_asset'` and `source_id` when a month-end valuation is snapshotted.
- State Rules:
  - `manual_assets` is the live authority source for non-transaction assets/liabilities.
  - Create/update mutation validates name, date, liquidity bucket, source evidence fields, and stores changes additively in the existing row; inactive rows are hidden from default valuation reports.
  - Soft delete/deactivate (`active=false`) is the default deletion semantic so historical `asset_snapshots.source_id` remains reproducible. Hard delete is exceptional cleanup only.
  - Missing `valuation_amount` rows are excluded; transaction sums are not used as a substitute.
  - `loan` bucket values are treated as liabilities and subtracted from asset-bucket totals.

## Entity: AssetSnapshot
- Source of Truth: `asset_snapshots`
- Purpose: Append-only item-level record of a basis-date asset/liability valuation produced from the live authority sources.
- Lifecycle: appended at month-end/manual close -> read for point-in-time reports and timeseries
- Key Fields:
  - `id` :: text :: snapshot row identifier
  - `basis_date` :: date text :: valuation date represented by the row
  - `source_type` :: text :: `account` or `manual_asset`
  - `source_id` :: text :: source entity identifier when available
  - `name` :: text :: snapshot display name copied from the source at capture time
  - `amount` :: integer nullable :: snapshotted valuation amount
  - `liquidity_bucket` :: text :: bucket copied from the source at capture time
  - `valuation_source`, `source_note`, `memo` :: text :: evidence and operator context
  - `snapshot_run_id` :: text :: month-end close run identifier for rows created together
  - `correction_of_run_id` :: text :: previous close run corrected by this append-only run when applicable
  - `correction_reason` :: text :: operator reason required for same-date re-close/correction
  - `is_correction` :: integer/bool :: marks correction close rows
  - `closed_at` :: text :: close execution timestamp copied across the run
  - `created_at` :: text :: snapshot row creation timestamp
- Relationships:
  - Represents a historical record of `LedgerAccount.last_balance` or `ManualAsset.valuation_amount` rather than a new primary input source.
- State Rules:
  - Snapshot rows are append-only. Existing rows are not destructively overwritten.
  - `preview_asset_snapshot_close` computes the source item list, bucket totals, total assets/liabilities, net worth, and warnings without writing rows.
  - `close_asset_snapshot` writes one run of item-level rows for a basis date from live authority sources. Same-date re-close requires `correction_reason`, appends a new correction run, and links to the previous `snapshot_run_id` when available.
  - Read reports choose the latest run for a basis date when run metadata exists, preserving original and correction rows for audit while avoiding double counting.
  - Basis-date snapshot reports prefer stored `asset_snapshots` rows when rows exist for the date; otherwise read reports can fall back to live authority sources.

## Entity: Transaction
- Source of Truth: `transactions`
- Purpose: Represents imported or manually entered financial event evidence.
- Lifecycle: created/imported -> categorized/reconciled -> soft-deleted or retained
- Key Fields:
  - `transaction_date` :: date text :: ledger date
  - `posted_date` / `transaction_datetime` :: text :: source timing detail where available
  - `type` :: text :: `income`, `expense`, `transfer`, `refund`, or `adjustment`
  - `amount` :: integer :: positive KRW amount
  - `category` / `subcategory` :: text :: household classification
  - `merchant`, `counterparty`, `description`, `payment_method` :: text :: evidence and display fields
  - `source`, `external_id`, `import_batch_id`, `raw_summary`, `raw_payload` :: text :: provenance and audit fields
  - `raw_payload` write policy is `minimize`: only source-specific allowlisted normalized keys may be persisted; raw document dumps, full file paths, passwords, tokens/secrets, and arbitrary parser columns are excluded.
  - `memo` :: text :: operator notes and reconciliation tags
  - `status` :: text :: active/deleted/review lifecycle state where used
- Relationships:
  - belongs to LedgerAccount through `account_id`
  - linked to ImportBatch through `import_batch_id`
  - interpreted with LedgerCategory through `category`/`subcategory`
- State Rules:
  - Amount is stored positive; `type` determines direction.
  - Cancellation/refund rows should use `refund` type when confirmed.
  - Manual link evidence should be memo-tagged rather than silently rewritten.

## Entity: LedgerCategory
- Source of Truth: `ledger_categories`
- Purpose: Defines category/subcategory labels and their cashflow semantics.
- Lifecycle: seeded/created -> updated -> inactive/deleted
- Key Fields:
  - `category` :: text
  - `subcategory` :: text
  - `cashflow_kind` :: text :: `income`, `expense`, `transfer`, `investment`, `debt_payment`, `exclude`, `unknown`
  - `sort_order`, `memo`, `active`
- Relationships:
  - Referenced by Transaction classification.
  - Targeted by CategoryRule results.
- State Rules:
  - `cashflow_kind` controls inclusion in actual household cashflow reports.
  - Transfer, investment, exclude, and unresolved kinds are not actual household spending unless a report explicitly asks for account movements.

## Entity: CategoryRule
- Source of Truth: `category_rules`
- Purpose: Automatic classification policy for transactions.
- Lifecycle: created -> previewed/applied -> updated/deactivated/deleted
- Key Fields:
  - `match_field` :: text :: field to inspect
  - `pattern` :: text :: match pattern
  - `match_type` :: text :: contains/equals/regex
  - `category` / `subcategory` :: text :: classification target
  - `priority` :: integer :: deterministic ordering
  - `amount_min`, `amount_max`, `day_of_month` :: integer nullable :: optional constraints
  - `person`, `is_fixed`, `confidence`, `memo`, `active`
- Relationships:
  - Applies to Transaction rows.
  - Should target valid or intentional category/subcategory labels.
- State Rules:
  - Preview should precede broad application when changing classification behavior.
  - More specific/high-priority rules should avoid accidentally overriding unrelated evidence.

## Entity: ImportBatch
- Source of Truth: `import_batches`
- Purpose: Tracks grouped imports for audit and rollback.
- Lifecycle: previewed -> imported -> optionally rolled back
- Key Fields:
  - `id` :: text :: import batch identifier
  - `source` :: text :: source kind
  - `filename` :: text :: masked basename plus source-hash prefix for operator correlation; full local source paths must not be stored on new writes
  - `memo` :: text/json :: compact batch metadata such as source hash, validation summary, or source-specific sheet/page references used for read-time provenance shaping
  - `inserted_count`, `skipped_duplicate_count`, `created_at`, `rolled_back_at`
- Relationships:
  - Transaction rows reference `import_batch_id`.
- State Rules:
  - Rollback should soft-delete by default.
  - Duplicate imports with `skip_duplicates=True` may insert zero rows; that is not automatically an error.
  - Read surfaces should expose sanitized provenance envelopes: default `minimal`, explicit operator-facing `operator-summary`, and explicit debug-gated diagnostics without full document/path leakage.

## Entity: Budget
- Source of Truth: budget table/service functions
- Purpose: Monthly category/subcategory/person budget target.
- Lifecycle: created/updated -> compared against actual spending
- Key Fields:
  - `month`, `category`, `subcategory`, `person`, `amount`, `currency`, `memo`
- Relationships:
  - Compared against Transaction aggregates by report functions.

## Entity: LedgerBackupArtifact
- Source of Truth: filesystem under `/root/.local/share/jarca/backups/ledger/` by default
- Purpose: Timestamped SQLite backup artifact used for operational recovery and validation smoke checks.
- Lifecycle: created -> listed/status-checked -> retained/pruned by policy -> optionally restored by script/runbook after approval
- Key Evidence Fields:
  - `path` :: text :: backup file path outside repository
  - `size_bytes` :: integer :: file size metadata
  - `mtime` :: text :: filesystem modification time
  - `sha256` :: text :: file checksum
  - `integrity_check` :: text :: SQLite integrity result
  - `table_counts` / `aggregate_totals` :: metadata-only validation evidence
- State Rules:
  - Keep backups from the most recent 30 days or newest 20 files.
  - Backup artifacts and validation DB copies must not be committed to git.
  - Restore execution is not a persisted MCP operation and requires script/runbook approval controls.

## Entity: ReviewDecision
- Source of Truth: `review_decisions`
- Purpose: 공통 review/workbench envelope for cross-transfer review rows, reconciliation targets, and uncategorized recommendation items.
- Lifecycle: pending -> decided/held/needs_recheck/ignored -> pending release when operator clears hold/recheck.
- Key Fields: `id`, `target_type`, `target_id`, `status`, `decision_type`, `reason_code`, `memo`, `operator`, `decided_at`, `review_after`, sanitized JSON `target_payload`, `created_at`, `updated_at`.
- State Rules: decisions are non-destructive and do not rewrite accounting fields by themselves. Existing read contracts are preserved by attaching additive `review_status` / `review_decision` metadata. `target_payload` omits raw payloads, passwords, tokens, secrets, and source file paths.

## Entity: ReconciliationSession
- Source of Truth: `reconciliation_sessions`
- Purpose: 계좌별/기준일별 잔액대조 session close/open state and difference evidence.
- Lifecycle: open -> closed, with `needs_recheck`/`ignored` available for operator triage.
- Key Fields: `id`, `account_id`, `basis_date`, `expected_balance`, `actual_balance`, `difference`, `status`, `closed_at`, `source_note`, `memo`, `operator`, `created_at`, `updated_at`.
- State Rules: account/basis-date rows are upserted idempotently; close records `closed_at`; destructive delete/reset workflow is not exposed.

## Consistency Rules
- Actual cashflow reports include only categories whose `cashflow_kind` represents actual income or expense according to report semantics.
- Account movement summaries should be separate from actual household income/expense summaries.
- Cross-account auto-matching requires same date, same amount, opposite direction, and compatible source/owner evidence.
- External counterparty securities transfers are review items unless matched or explicitly confirmed.
- Import evidence fields are retained for audit and future correction.
- DB corrections should be narrow and memo-tagged.
- Dashboard/MCP asset and net-worth reports use a shared valuation helper: account-backed items from `accounts.last_balance` plus non-transaction items from `manual_assets`.
- Net worth is asset buckets (`cash`, `investment`, `deposit`, `real_estate`, `other`) minus absolute `loan` liabilities; missing valuations are excluded and reported, never inferred from transactions.
- Review workbench state is sourced from `review_decisions`; pending/held/recheck counts must not be inferred by overwriting raw transaction evidence.
- Reconciliation operating state is sourced from `reconciliation_sessions`; last close time is the max closed session `closed_at`.

## Persistence Notes
- Schema changes should be additive/idempotent unless a destructive migration is explicitly approved and backed up.
- Operational backup/status evidence may include only path, size, mtime, checksum, integrity, row counts, and aggregate totals.
- Validation DB copies should live outside the repository and should be selected explicitly via `JARCA_LEDGER_DB_PATH` when used.
- Passwords for encrypted imports are runtime-only inputs and are not part of the persisted model.
- Raw payloads may contain sensitive financial evidence and should not be committed to git; BAT-017 constrains new writes to minimized allowlisted payloads only and leaves legacy row cleanup to a separate approved follow-up.
- ReviewDecision `target_payload` is sanitized to omit `raw_payload`, passwords, tokens, secrets, and file paths before persistence.

## Current Gaps / Planned Changes
- Formal schema migration documentation should be added before larger DB changes.
- Some UI workflows need final E2E verification against real rows.
- Statement-unavailable account handling is tracked by REQ-003/BAT-003; REQ-020/BAT-016 adds dedicated reconciliation sessions and persistent review-decision queue metadata.
- BAT-009 implements manual asset CRUD mutation and month-end snapshot preview/close/correction service/MCP/dashboard workflow. Future work may add automatic market/real-estate valuation or scheduled month-end close.

## Change Log
- 2026-07-09 KST: REQ-021/BAT-017 기준으로 import provenance exposure policy, `transactions.raw_payload=minimize`, `import_batches.filename` masked-basename persistence, and explicit debug gate expectations를 반영.
- 2026-06-26 22:24 KST: Created by `bootstrap-baseline` from repository observation.
- 2026-06-27 10:17 KST: BAT-002/REQ-002 기준으로 LedgerBackupArtifact와 metadata-only evidence 정책 추가.
- 2026-06-29 KST: REQ-003/BAT-003 기준으로 `accounts.statement_status`와 수동 잔액대조 preview 모델 추가.
- 2026-06-29 KST: REQ-005/BAT-005 기준으로 `accounts.liquidity_bucket`, `last_balance` 기반 자산 유동성/부채/순자산 규칙 추가.
- 2026-06-30 KST: REQ-009/REQ-010/BAT-008 기준으로 `manual_assets`, `asset_snapshots`, 권위 소스(`accounts.last_balance` + `manual_assets`), 순자산 산식/미평가 제외 정책 추가.
- 2026-07-01 KST: REQ-011/REQ-012/BAT-009 기준으로 manual asset CRUD, snapshot preview/close, append-only correction metadata, latest-run read policy 추가.
- 2026-07-08 KST: REQ-020/BAT-016 기준으로 `review_decisions`, `reconciliation_sessions`, review 상태값, payload 최소보관/마스킹 규칙, 운영 허브 집계 source-of-truth를 추가.