fix: ledger write idempotency for TimescaleDB hypertable (BAT-016 rework)

BAT-016's initial implementation passed only against mocked repositories and
broke on a real database. Two defects fixed:

1. insert_ledger_trade used `ON CONFLICT (portfolio_id, order_id, execution_id)
   WHERE order_id IS NOT NULL` but portfolio_ledger is a TimescaleDB hypertable,
   which cannot have that partial unique index (must include the partitioning
   column recorded_at). The statement errored against Postgres. Replaced with
   application-level idempotency: SELECT-before-insert inside the existing
   `SELECT ... FOR UPDATE` portfolio row lock, which serializes concurrent
   same-portfolio writes (race-safe). Duplicate returns current authoritative
   portfolio state (dcy-006 OQ-4 b).

2. `with self._connect() as conn:` closes the shared psycopg3 connection on exit,
   leaving the reused repository connection unusable. Switched to
   `with conn.transaction():` (commits without closing).

Added tests/test_ledger_consolidation_integration.py — real-Postgres integration
tests (new / duplicate / negative-balance) that close the mock-only verification
gap. Full suite: 376 passed.

SDLC: REQ-036/037 CHG-20260605-03 reconcile OQ-7 (ii) (DB partial unique →
application-level idempotency) since the hypertable makes the DB constraint
infeasible. REQ-036/037/038 → Implemented; BAT-016 → release-candidate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-05 22:11:17 +09:00
co-authored by Claude Opus 4.8
parent 44f2102d26
commit 2a19b7f058
11 changed files with 414 additions and 63 deletions
@@ -1,6 +1,6 @@
# REQ-037: 신 path 회계 책임 단일화 + 응답에 portfolio 상태 포함
- Status: Approved
- Status: Implemented
- Type: Interface
- Priority: High
- Owner: Justin Ko
@@ -16,7 +16,7 @@
- **잔고/수량 부족 거부 검증을 적용하지 않는다 — 음수 `cash_balance`/`asset_quantity` 모두 허용**(dcy-006 OQ-3). 음수 검증을 제거한 회계 헬퍼를 ledger 흐름 전용으로 추출하거나 inline 구현(기존 `apply_buy`/`apply_sell`은 레거시 전용으로 보존, R-8).
- `portfolio_ledger`에 INSERT(REQ-036 신 스키마 사용). broker 식별자 `order_id`/`execution_id`/`side`/`executed_at` + 갱신 후 잔고 snapshot(`cash_balance_after`, `asset_quantity_after`, `total_value_after`) 기록.
- portfolios 테이블 UPDATE(`cash_balance`, `asset_quantity`, `last_trade_at`, `updated_at`).
- duplicate 감지(dcy-006 FR-15): `ON CONFLICT (portfolio_id, order_id, execution_id) WHERE order_id IS NOT NULL DO NOTHING`로 전환(기존 `ON CONFLICT ON CONSTRAINT uq_ledger_trades_idempotency` 대체). PostgreSQL partial index inference 사용.
- duplicate 감지(dcy-006 FR-15, CHG-20260605-03 재정의): **application-level idempotency** — `SELECT ... FOR UPDATE`로 portfolios row를 잠근 상태에서 `(portfolio_id, order_id, execution_id)`로 기존 portfolio_ledger row를 SELECT한다. 존재하면 duplicate(INSERT/UPDATE 없이 현재 권위 상태 응답), 없으면 INSERT. portfolio_ledger가 TimescaleDB hypertable이라 DB partial UNIQUE index/`ON CONFLICT`가 불가능하므로(REQ-036 CHG-20260605-03) DB 제약 대신 row lock 직렬화로 race-safe 보장. order_id NULL(레거시/수동)은 dedup 조회 생략 → append-only. 트랜잭션은 `conn.transaction()` 사용(psycopg3 `with conn:`는 공유 connection을 close하므로).
- `status="new"`(신규 row): 거래 반영 후 권위 상태를 응답한다(FR-2).
- `status="duplicate"`(재호출): ledger row 재생성 없이 duplicate 호출 시점의 portfolios 테이블 현재 권위 상태를 SELECT로 조회해 응답한다(FR-3, dcy-006 OQ-4 (b)).
- `LedgerTradeResponse`([app/schemas/v1.py](../../app/schemas/v1.py))에 `cash_balance_after: float`, `asset_quantity_after: float` 두 필드를 응답 root에 flat으로 비파괴 추가한다(FR-1, dcy-006 OQ-2 (b)). 기존 응답 필드는 모두 보존.
@@ -39,7 +39,7 @@
- `docs/runtime-flows.md` — ledger write flow에 회계 적용 + portfolio 갱신 + 트랜잭션 lock 단계 + portfolio_ledger 단일 원장 명문화
- Modules:
- `app/api/v1_portfolios.py:record_ledger_trade_v1` — 통합 함수 호출 + 응답 확장
- `app/repositories/postgres.py` — `insert_ledger_trade` 재정의(portfolio_ledger insert + portfolios UPDATE 단일 트랜잭션 + FOR UPDATE + ON CONFLICT partial), `_row_to_ledger_*` 통합
- `app/repositories/postgres.py` — `insert_ledger_trade` 재정의(portfolio_ledger insert + portfolios UPDATE 단일 트랜잭션 + FOR UPDATE + application-level SELECT-before-insert idempotency + `conn.transaction()`), `_row_to_ledger_*` 통합
- `app/repositories/portfolio_repository.py` — interface 시그니처 갱신
- `app/domain/models.py` — `LedgerEntry` 단일화, `LedgerTrade` 삭제
- `app/domain/services.py` 또는 신규 — 음수 검증 제거한 ledger 전용 회계 헬퍼 추출
@@ -57,6 +57,67 @@
## Change Log
### CHG-20260605-04
- Date: 2026-06-05
- Author: Justin Ko
#### Change Summary
- Status: Approved → Implemented
#### Intent
- BAT-016 confirm-batch-verification 완료. 신 path 회계 책임 단일화 + 응답 cash_balance_after/asset_quantity_after + duplicate 현 상태 + 도메인/repository 통합 구현. 실 DB 통합 테스트(new/duplicate/음수) 통과, AC-1~7 검증. 전체 376 passed. CHG-20260605-03(ON CONFLICT→application-level idempotency + conn.transaction()) 정합 반영.
#### Acceptance Criteria Delta
- No change
#### Impacted Area Delta
- No change
#### Delivery Trace Delta
- Batches:
- BAT-016 (release-candidate)
- Releases:
- REL-003 (예정)
#### Revalidation Impact
- Reverification Needed: no
- Existing Implementation Invalidated: no
- Status Recommendation: implemented
- Follow-up Action: REL-003 배포 시 trex-core 응답 명세 정합 최종 확인
#### Validation Plan
- 실 DB 통합 테스트 + pytest 376 passed (완료)
### CHG-20260605-03
- Date: 2026-06-05
- Author: Justin Ko
#### Change Summary
- Requirement 재정의: duplicate 감지를 `ON CONFLICT ... WHERE order_id IS NOT NULL`(DB partial index)에서 **application-level SELECT-before-insert(under FOR UPDATE lock)**로 변경. portfolio_ledger가 TimescaleDB hypertable이라 partial unique index/ON CONFLICT가 불가능(REQ-036 CHG-20260605-03). 추가로 트랜잭션을 `conn.transaction()`으로 처리(psycopg3 `with conn:` connection close 결함 회피). Status Approved 유지.
#### Intent
- BAT-016 verify에서 실 DB 미동작(ON CONFLICT 에러 + 공유 connection close) 2건 결함 발견. hypertable 현실에 맞춰 idempotency 설계 정합 + 결함 수정.
#### Acceptance Criteria Delta
- No change (AC-1~7 그대로; AC-2 duplicate idempotency는 application layer로 동일 충족). 검증 방식만 mock → 실 DB 통합 테스트 추가.
#### Impacted Area Delta
- Modules: `app/repositories/postgres.py` insert_ledger_trade — ON CONFLICT 제거 + SELECT-before-insert + `conn.transaction()` 반영
- Tests: `tests/test_ledger_consolidation_integration.py`(실 DB) 추가
#### Delivery Trace Delta
- Batches: BAT-016 (in-delivery)
- Releases: none (REL-003 등 분리 예정)
#### Revalidation Impact
- Reverification Needed: yes (실 DB 통합 테스트로 재검증 — 376 passed)
- Existing Implementation Invalidated: no (구현이 본 재정의에 맞춰 수정됨)
- Status Recommendation: keep-current
- Follow-up Action: BAT-016 verification 실 DB evidence 반영
#### Validation Plan
- pytest 376 passed(통합 3 포함) + 실 DB new/duplicate/음수 통합 테스트
### CHG-20260605-02
- Date: 2026-06-05
- Author: Justin Ko
@@ -1,6 +1,6 @@
# REQ-038: 레거시 buy/sell endpoint deprecated 표기
- Status: Approved
- Status: Implemented
- Type: Interface
- Priority: Low
- Owner: Justin Ko
@@ -39,6 +39,34 @@
## Change Log
### CHG-20260605-03
- Date: 2026-06-05
- Author: Justin Ko
#### Change Summary
- Status: Approved → Implemented
#### Intent
- BAT-016 confirm-batch-verification 완료. 레거시 buy/sell `deprecated=True` + `Deprecation: true` 헤더 구현, 동작 보존. AC-1~4 검증(`tests/test_auth.py::test_legacy_buy_sell_routes_emit_deprecation_header` 포함). 전체 376 passed.
#### Acceptance Criteria Delta
- No change
#### Impacted Area Delta
- No change
#### Delivery Trace Delta
- Batches:
- BAT-016 (release-candidate)
- Releases:
- REL-003 (예정)
#### Revalidation Impact
- Reverification Needed: no
- Existing Implementation Invalidated: no
- Status Recommendation: implemented
- Follow-up Action: 실제 endpoint 제거는 후속 정리 트랙
### CHG-20260605-02
- Date: 2026-06-05
- Author: Justin Ko