38 Commits
Author SHA1 Message Date
Hermes Agent 80cc6c0d73 chore(release): bump version to 0.4.0
Publish verified container image / publish (push) Canceled after 0s
Publish verified container image / verify (push) Canceled after 0s
2026-08-26 22:23:11 +09:00
genichin 81624931cf Merge pull request #75 from fourblocks2020/copilot/feat-alerts-notify-reconcile-warned
feat(alerts): emit first `reconcile_warned` intent rejection alarm from `IntentService`
2026-08-26 14:53:25 +09:00
copilot-swe-agent[bot]andgenichin ac3f27241a docs(proofline): restore line-0008 canonical approved artifacts verbatim
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-26 05:29:56 +00:00
copilot-swe-agent[bot]andgenichin 75406af970 test: make context-failure regressions deterministic and simplify alarm log handling
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-26 05:07:35 +00:00
copilot-swe-agent[bot]andgenichin 1ca9e60c98 fix(intents): isolate alarm context lookup and sanitize alarm failure logs
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-26 05:02:19 +00:00
copilot-swe-agent[bot]andgenichin 5dd3d57f86 test: harden reconcile warning alarm coverage and daemon wiring assertion
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-26 04:21:17 +00:00
copilot-swe-agent[bot]andgenichin a9e88f9770 feat(intents): emit warn alarm on first reconcile_warned rejection
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-26 04:19:14 +00:00
copilot-swe-agent[bot] 98c38327b2 Initial plan 2026-08-26 04:16:07 +00:00
genichin 2e314232e3 Merge pull request #73 from fourblocks2020/copilot/feat-alerts-publish-reconciliation-quantity-mismat
feat(alerts): publish B/C reconciliation quantity mismatches to trex.alerts with durable deduplication
2026-08-26 10:03:24 +09:00
copilot-swe-agent[bot]andgenichin 21f8fdd4b6 fix(ac-0024): injective C identity via SHA-256 length-prefix digest + PostgreSQL migration/round-trip tests
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-26 00:38:02 +00:00
copilot-swe-agent[bot]andgenichin c246fca206 fix(ac-0024): add type annotation for notification_store in SnapshotScheduler
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-26 00:22:15 +00:00
copilot-swe-agent[bot]andgenichin f1864a6aa8 fix(ac-0024): address 4 blockers: NUL byte in C identity, instance_key isolation, clear_warned gate, production wiring
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-26 00:20:09 +00:00
copilot-swe-agent[bot]andgenichin 153a563f6d feat(alerts): publish reconciliation quantity mismatches to trex-notifier (req-0007/ac-0024)
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-25 14:50:29 +00:00
copilot-swe-agent[bot] baae02d2af Initial plan 2026-08-25 14:33:03 +00:00
Hermes Agent 8d0942fb30 docs(proofline): approve issue 64 alert specification 2026-08-25 23:27:41 +09:00
genichin 24a64b55d1 Merge pull request #72 from fourblocks2020/copilot/bug-fix-portfolio-hub-quantity-discrepancy
AC-0023: Block C-case quantity mismatch slots from strategy RPC; fail-closed on warn-store failure
2026-08-25 22:54:17 +09:00
copilot-swe-agent[bot]andgenichin f46c2efd8b AC-0023: fix Ruff F401/F841 in test_slots_handler_quantity_mismatch.py
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-25 13:40:22 +00:00
copilot-swe-agent[bot]andgenichin 49eb20bd51 AC-0023: fail-closed intent warn-store; C-case-only verify clear; clear_failed on partial error
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-25 13:27:03 +00:00
copilot-swe-agent[bot]andgenichin 1179bdfe23 AC-0023: filter C-case quantity mismatch slots from strategy RPC, fail-closed on warn_store failure, symbol-aggregate verify clear
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-25 13:17:06 +00:00
copilot-swe-agent[bot] d9205fe151 Initial plan 2026-08-25 13:06:45 +00:00
genichin d18f098b27 Merge pull request #71 from fourblocks2020/copilot/bug-send-broker-authoritative-holding-quantity
slots RPC: make holding `qty` broker-authoritative with fail-closed availability semantics
2026-08-25 21:34:58 +09:00
copilot-swe-agent[bot]andgenichin d4a968ee1a fix: normalize snapshot read failures to unavailable in slots RPC path
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-25 12:25:39 +00:00
copilot-swe-agent[bot]andgenichin f394f1560d chore: finalize broker qty metadata consistency
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-24 14:19:26 +00:00
copilot-swe-agent[bot]andgenichin 30bcbe1d12 feat: use broker-authoritative qty in slots RPC holdings
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-24 14:15:43 +00:00
copilot-swe-agent[bot] 832dcb5fbb Initial plan 2026-08-24 14:08:01 +00:00
genichin 131d694ce4 Merge pull request #70 from fourblocks2020/copilot/feat-alerts-persist-unresolved-skipped-fills 2026-08-24 22:54:07 +09:00
copilot-swe-agent[bot]andgenichin 169f222042 fix: tighten ledger skip CLI auditing
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-24 13:42:47 +00:00
copilot-swe-agent[bot]andgenichin 48dfc3e76f fix: harden skipped ledger resolution paths
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-24 13:38:04 +00:00
copilot-swe-agent[bot]andgenichin b88b21b772 fix: harden ledger skip invariant
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-24 13:24:15 +00:00
copilot-swe-agent[bot]andgenichin 76cde2edd0 feat: persist skipped ledger fills
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-24 13:23:06 +00:00
copilot-swe-agent[bot] df0258bc36 Initial plan 2026-08-24 13:10:39 +00:00
genichin 231e8e2eec Merge pull request #69 from fourblocks2020/copilot/feat-cli-add-ledger-backfill-command
Add evidence-backed `ledger backfill` operator command
2026-08-24 20:09:05 +09:00
copilot-swe-agent[bot]andgenichin 8f74c6b67d chore: address review feedback
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-24 10:27:49 +00:00
copilot-swe-agent[bot]andgenichin 3e74d35299 test: validate ledger backfill flow
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-24 10:25:35 +00:00
copilot-swe-agent[bot]andgenichin e2c4bbba34 feat: add ledger backfill command
Co-authored-by: genichin <4282227+genichin@users.noreply.github.com>
2026-08-24 10:24:05 +00:00
copilot-swe-agent[bot] f124e9509c Initial plan 2026-08-24 10:16:22 +00:00
Hermes Agent fbfe026a7d docs(proofline): revise SOR unknown venue ledger AC 2026-08-24 10:27:31 +09:00
Hermes Agent 9da8c27e52 fix(ledger): retain SOR fills with unknown venue 2026-08-24 09:47:21 +09:00
56 changed files with 6318 additions and 245 deletions
+4 -4
View File
@@ -7,10 +7,10 @@ status: active
## Criterion
core는 original `order_venue`와 broker-authoritative actual `execution_venue`를 서로 다른 의미로 보존해야 한다. `order_venue`는 KRX/NXT/SOR이며 submit, ack, slot, restart와 cancel에 사용한다. actual venue는 KRX/NXT 또는 미확정 NULL이며 KIS의 검증된 `ORD_EXG_GB`와 명시 exchange-id allowlist에서만 결정하여 fill, ledger와 outbox에 기록한다. `SOR`, 누락, 미지원 코드 또는 wall-clock time으로 actual venue를 추론해서는 안 된다. historical outbox SOR는 `order_venue=SOR`, actual NULL로 변환하고 migration replay는 신규 정상 조합을 오염시키지 않아야 한다.
core는 original `order_venue`와 broker-authoritative actual `execution_venue`를 서로 다른 의미로 보존해야 한다. `order_venue`는 KRX/NXT/SOR이며 submit, ack, slot, restart와 cancel에 사용한다. actual venue는 KRX/NXT 또는 미확정 NULL이며 KIS의 검증된 `ORD_EXG_GB`와 명시 exchange-id allowlist에서만 결정하여 fill, ledger와 outbox에 기록한다. 완료된 SOR fill의 actual venue가 확인되지 않으면 core는 `venue=NULL`, `order_venue=SOR`로 정상 ledger outbox work item을 enqueue해야 한다. `SOR`, 누락, 미지원 코드 또는 wall-clock time으로 actual venue를 추론해서는 안 된다. historical outbox SOR는 `order_venue=SOR`, actual NULL로 변환하고 migration replay는 신규 정상 조합을 오염시키지 않아야 한다.
## Verification
- 성공 조건: SOR order의 actual KRX/NXT fill이 `(order_venue=SOR, venue=KRX|NXT)`로 기록되고 cancel/restart는 SOR를 유지한다. historical SOR는 `(SOR,NULL)`이며 정상 `(order=KRX, actual=NXT)`는 migration 반복 후 보존된다.
- 실패·경계 조건: actual SOR 저장, unknown code를 KRX/NXT로 추론, actual venue로 slot order venue 덮어쓰기, NULL actual SOR fill의 거짓 ledger 기록, AUTO·NULL order 저장 또는 migration replay 오염은 FAIL이다.
- 검사 방법: KIS parser tests, execution ledger tests, `tests/adapters/persistence/test_postgresql_ledger_outbox.py`, `tests/adapters/persistence/test_migration_and_drift.py`를 실행한다. 실제 PostgreSQL에서 historical conversion, replay, round-trip과 check violation을 검증하고 전체 회귀·Ruff 및 독립 read-only QC를 수행한다.
- 성공 조건: SOR order의 actual KRX/NXT fill은 `(order_venue=SOR, venue=KRX|NXT)`로 기록되고, actual venue가 확인되지 않은 완료 SOR fill은 `(order_venue=SOR, venue=NULL)`인 durable outbox 1건을 생성해 기존 execution-ID 멱등 `LedgerWriter` 경로로 기록된다. cancel/restart는 SOR를 유지하며, historical SOR는 `(SOR,NULL)`이고 정상 `(order=KRX, actual=NXT)`는 migration 반복 후 보존된다.
- 실패·경계 조건: actual SOR 저장, unknown code를 KRX/NXT로 추론, actual venue로 slot order venue 덮어쓰기, actual venue가 NULL인 완료 SOR fill의 outbox 미생성·원장 유실, AUTO·NULL order 저장 또는 migration replay 오염은 FAIL이다.
- 검사 방법: KIS parser tests와 KIS daily-ccld SOR fixture의 execution→outbox→`LedgerWriter` 회귀·replay idempotency, `tests/adapters/persistence/test_postgresql_ledger_outbox.py`, `tests/adapters/persistence/test_migration_and_drift.py`를 실행한다. 실제 PostgreSQL에서 historical conversion, replay, round-trip과 check violation을 검증하고 전체 회귀·Ruff 및 독립 read-only QC를 수행한다.
+16
View File
@@ -0,0 +1,16 @@
---
id: "ac-0020"
status: active
---
# 완료됐지만 원장 기록 불가인 fill의 durable skip 감사 기록
## Criterion
core는 완료된 broker fill이 정상 `ledger_outbox` 경로로 들어갈 수 없을 때 WARN 로그만 남기고 끝내서는 안 된다. genuinely non-recordable 사례(예: `portfolio_id` 누락, `execution_id` 누락, 지원하지 않는 actual execution venue)마다 durable `ledger_skip` 감사 레코드 1건을 멱등하게 유지해야 하며, 이 레코드는 최소한 daemon instance, slot ID, portfolio ID, symbol, order ID, execution ID(있으면), side, qty, price, execution time, `order_venue`, `execution_venue` 또는 명시적 unknown 상태, machine-readable `skip_reason`, 생성/갱신/해결 timestamp, resolution state를 포함해야 한다. 이 경로는 broker·slot·portfolio 수량을 자동 보정하거나 mutate하지 않아야 하며, retryable ledger delivery failure는 계속 `ledger_outbox`와 `LedgerWriter` 재시도 의미론을 사용해야 한다.
## Verification
- 성공 조건: completed fill이 `missing_portfolio_id`, `missing_execution_id`, `unsupported_execution_venue` 등으로 정상 원장 기록이 불가능하면 slot transition 이후에도 `ledger_skip`에 동일 fill 식별자의 unresolved row가 남고, 같은 fill을 다시 관찰해도 duplicate row가 생기지 않는다. 운영자는 instance/date/portfolio/resolution state 조건으로 unresolved row를 조회할 수 있다.
- 실패·경계 조건: completed-but-unrecordable fill이 ephemeral log에만 남음, skip row가 retryable delivery failure row를 대체함, skip 처리 중 slot/broker/portfolio 수량이 암묵적으로 바뀜, unknown execution venue alone(SOR actual NULL)는 정상 outbox 경로 대신 skip으로 빠짐은 FAIL이다.
- 검사 방법: execution ledger unit tests, PostgreSQL `ledger_skip` store round-trip/filter/resolve tests, CLI skipped-ledger query tests, targeted Ruff/pytest와 ProofLine validate를 수행한다.
+16
View File
@@ -0,0 +1,16 @@
---
id: "ac-0021"
status: active
---
# skipped-ledger 알림과 해결 이벤트
## Criterion
unresolved `ledger_skip` 레코드는 best-effort·non-blocking 방식으로 deduplicated `trex.alerts` 이벤트를 발행해야 하며, notifier 부재나 transport 오류가 fill lifecycle을 막아서는 안 된다. 동일 unresolved event는 중복 알림 없이 1회만 알리고, 이후 정상 ledger delivery가 성공하거나 operator가 명시적으로 reconcile/resolve 하면 해당 `ledger_skip` 레코드는 resolved 상태로 전이되고 distinct resolved alert event를 발행해야 한다.
## Verification
- 성공 조건: 새로운 unresolved skip 1건은 `trex.alerts`에 `ledger_skip_unresolved` alarm 1건을 만들고, manual backfill 또는 operator resolve 후에는 동일 skip에 대해 `ledger_skip_resolved` alarm이 별도로 발행된다. 알림 publish 실패는 swallow되며 durable state 전이는 유지된다.
- 실패·경계 조건: unresolved skip마다 중복 알림이 반복 발행됨, resolved 전이가 알림 없이 묵살됨, alert publish failure가 fill/backfill/resolve 경로를 abort시킴은 FAIL이다.
- 검사 방법: in-memory alarm transport tests, CLI ledger backfill resolution tests, skipped-ledger resolve command tests, PostgreSQL resolution-state verification을 실행한다.
+19
View File
@@ -0,0 +1,19 @@
---
id: "ac-0022"
status: active
---
# strategy slots RPC broker-authoritative holding quantity
## Criterion
`trex.slots.req.<instance>`의 **전략용 slots 응답에 포함되는** `holding` 슬롯(AC-0023에 따라 C-case 수량 불일치로 제외된 slot 제외)의 `qty`는 요청한 instance에 결합된 KIS account broker holdings(`hldg_qty`)를 authoritative source로 사용해야 한다. `daily_slot.qty`와 portfolio-hub 집계값은 각각 `slot_quantity`, `asset_quantity` 진단 필드로 분리 유지한다. broker 수량을 확보하지 못했거나 freshness 허용시간을 초과하면 core/hub mirror fallback 없이 `qty=null`, `qty_source="unavailable"`, `broker_quantity_available=false`, `broker_quantity_captured_at=null`을 반환해 전략이 fail-closed하도록 해야 한다. confirmed broker zero는 `qty=0`, `qty_source="broker"`로 unavailable과 구분되어야 한다. 이 read path는 주문 submit/cancel, ledger write, slot lifecycle mutation을 유발하지 않아야 한다.
**Scope note:** AC-0023에 따라 C-case(`case_code=C`) 수량 불일치로 전략용 slots RPC에서 제외된 slot에는 이 AC의 broker-authoritative `qty` 응답 형식을 적용하지 않는다. AC-0023 제외가 AC-0022 broker-qty 형식보다 우선한다.
## Verification
- 성공 조건: mismatch fixture(`broker=440`, `slot=435`, `hub=435`)에서 holding row가 `qty=440`, `slot_quantity=435`, `asset_quantity=435`, `qty_source="broker"`를 반환한다. broker zero fixture는 `qty=0`으로 유지된다.
- 실패·경계 조건: broker unavailable/stale fixture에서 `qty`가 slot/hub 값으로 대체되면 FAIL이다. 이 경우 `qty=null`과 명시적 unavailable metadata를 반환해야 한다.
- 격리 조건: 다중 instance 테스트에서 각 `trex.slots.req.<instance>`는 해당 instance account snapshot만 참조하며 타 instance broker/snapshot/slot 데이터가 섞이지 않아야 한다.
- 비침습 조건: slots RPC 호출 전후로 slot 상태 전이, broker mutation, ledger write side effect가 없어야 한다.
+25
View File
@@ -0,0 +1,25 @@
---
id: "ac-0023"
status: active
---
# C-case quantity mismatch — strategy RPC exclusion and intent blocking
## Criterion
KIS 계좌 잔고(`hldg_qty`)와 같은 instance가 관리하는 동일 symbol의 portfolio-hub `asset_quantity` 합계가 일치하지 않으면(`case_code=C` reconcile-warning 활성), 해당 symbol의 모든 `holding` slot을 `trex.slots.req.<instance>` 응답에서 제외해야 한다. 수량 대조는 **instance_key + symbol 단위 합계**로 수행하며, 개별 portfolio 수량을 KIS 종목 합계와 직접 비교해서는 안 된다. 응답에는 `excluded_slots.reconcile_quantity_mismatch` 필드로 제외된 slot 수를 포함한다. warn-store를 읽지 못하면 경고 유무를 모르는 상태로 slot 목록을 반환하지 않고 `degraded=true` / `reason="warn_store_unavailable"`으로 응답해야 한다(fail-closed). 신규 buy/sell intent는 durable warning을 재확인하여 `reconcile_warned` 사유로 거부한다. C-case 경고 해제는 symbol 단위 합계가 다시 일치한 경우에만 수행하며, 해당 symbol에 걸린 모든 C-case 경고를 함께 해제한다. 이 경로는 주문 submit/cancel, ledger write, slot lifecycle mutation을 유발하지 않아야 한다.
## Verification
- 성공 조건: KIS 440주, hub portfolio A 200주 + B 240주(합계 440) → A·B 모두 반환되고 `excluded_slots.reconcile_quantity_mismatch=0`.
- 불일치 조건: KIS 440주, hub A 200주 + B 235주(합계 435) → A·B 모두 제외되고 `excluded_slots.reconcile_quantity_mismatch=2`. 다른 symbol 정상 slot은 같은 응답에 포함된다.
- 단일 portfolio 불일치: hub 단일 portfolio qty != KIS → 해당 symbol 전체 slot 제외.
- fail-closed (slots RPC): warn-store가 `list_c_case_warned()` 중 예외를 throw하면 `slots=[], degraded=true, reason="warn_store_unavailable"` 반환.
- fail-closed (intent): intent 처리 시 `warn_store.is_warned()` 호출이 실패하면 `reconcile_warning_unavailable` 사유로 거부해야 한다 — fail-open 허용 불가.
- intent race: 마지막 slots RPC 이후 경고가 설정된 경우에도 intent 처리 시점의 durable warning을 재확인해 `reconcile_warned`로 거부.
- 해제: symbol 합계가 다시 일치하면 해당 symbol의 모든 **C-case 경고만** 함께 해제. 같은 symbol의 B-case 등 다른 case 경고는 보존해야 한다.
- 해제 대상: HOLDING 뿐 아니라 모든 상태의 slot에 걸린 C-case 경고를 포함한다.
- 해제 실패: 해제 도중 예외가 발생하면 `result='clear_failed'`를 반환하고 완전 성공으로 보고하지 않아야 한다(부분 성공 = 실패).
- 시간이 지났다는 이유만으로 자동 해제하지 않음.
- 비침습 조건: slots RPC 호출 전후로 slot 상태 전이, broker mutation, ledger write side effect가 없어야 한다.
- AC-0022 우선순위: C-case로 제외된 slot에는 broker-authoritative `qty` 응답 형식(AC-0022)을 적용하지 않는다.
+18
View File
@@ -0,0 +1,18 @@
---
id: "ac-0024"
status: active
---
# Reconciliation 수량 불일치 탐지·변경·해소 알림
## Criterion
`trex-core`는 B-case(core slot quantity와 broker quantity 불일치)를 stable slot별로, C-case(같은 instance·symbol의 portfolio-hub `asset_quantity` 합계와 broker quantity 불일치)를 `instance_key + symbol`별로 하나의 durable notification lifecycle로 추적해야 한다. 새 mismatch와 동일 identity의 case별 비교 quantity tuple 변경은 sanitized `trex.alerts` detected/changed event를 각각 한 번 발행하고, 성공한 publish fingerprint를 process restart 뒤에도 보존하여 unchanged scheduler tick을 억제해야 한다. B-case는 operator `sync`, C-case는 operator `verify`의 authoritative durable warning clear에서만 resolved event를 한 번 발행해야 하며 scheduler 관찰만으로 recovery를 발행해서는 안 된다. Event는 lifecycle state, `instance_key`, `symbol`, `case_code`, broker quantity와 case별 비교 quantity를 포함하고 account number/hash, credential, token, DSN, raw payload와 exception 본문을 포함해서는 안 된다. Publish 성공 뒤에만 fingerprint를 전진시키고 publish 또는 notification-state persistence 실패는 reconciliation, warning set/clear, sync/verify, scheduler와 slot lifecycle을 중단하거나 바꾸지 않아야 한다. Resolved publish 실패 시 notification state가 writable이면 pending transition을 durable하게 보존하여 후속 retry opportunity에서 재발행한다. Notification-state persistence 자체가 실패하면 authoritative warning clear를 유지하고 event 유실 가능성을 sanitized log로 남긴다.
## Verification
- 성공 조건: 새로운 B-case slot mismatch는 B detected event 한 건, 새로운 C-case symbol aggregate mismatch는 portfolio 수와 무관하게 C detected event 한 건을 만든다. 동일 fingerprint의 반복 tick은 추가 event를 만들지 않고 quantity tuple 변경은 changed event 한 건을 만든다. process restart 모사 뒤 unchanged mismatch는 억제되며, B `sync`와 C aggregate `verify` clear는 각 identity의 resolved event 한 건만 만든다.
- publish 실패 조건: transport failure에서는 detected/changed/resolved fingerprint가 전진하지 않아야 한다. detected/changed는 다음 동일 detection에서 재시도하고, resolved는 notification state가 writable한 경우 warning clear를 유지하면서 pending transition을 durable하게 보존해 후속 retry opportunity에서 재시도해야 한다. Notification-state persistence 실패도 business path를 중단하지 않아야 하며 resolved event 유실 가능성을 credential·DSN·raw exception 없는 log로 남겨야 한다.
- 경계 조건: C-case 동일 symbol의 여러 portfolio/slot이 중복 event를 만들거나, scheduler의 단순 quantity 일치가 durable warning clear·resolved event를 만들거나, B slot 간 fingerprint가 섞이거나, restart 뒤 unchanged event가 재발행되거나, publish 전 fingerprint 기록으로 event가 유실되면 FAIL이다.
- 보안·비변경 조건: 모든 lifecycle event의 allowlist fields와 rendered payload에 account number/hash, credential, token, DSN, raw KIS/portfolio-hub payload 및 exception 본문이 없어야 한다. 알림 전후 broker call mutation, portfolio-hub write, slot lifecycle/quantity mutation과 기존 AC-0023 blocking/clear semantics 변화가 없어야 한다.
- 검사 방법: in-memory alarm transport와 restart 가능한 notification-state fixture로 B/C 최초 탐지, C multi-portfolio aggregation, unchanged suppression, quantity change, restart, publish failure retry, B sync resolution, C verify resolution, duplicate clear와 sensitive-field absence를 검증한다. focused reconciliation·warning·alarm tests, persistence migration/round-trip tests와 전체 repository regression 및 Ruff를 실행한다.
+22
View File
@@ -0,0 +1,22 @@
---
id: "ac-0025"
status: active
---
# 재조정 경고에 따른 전략 주문 요청 최초 거부 알림
## Criterion
`trex-core`는 해당 slot에 해제되지 않은 재조정 경고가 남아 있어 `ReconcileWarningStore.is_warned(slot_id)`가 참을 반환하고, 그 때문에 외부 전략의 새로운 매수 또는 매도 요청을 `status="rejected"`, `reason="reconcile_warned"`로 거부한 경우, 그 결과를 `IntentStore.finalize()`에 저장한 뒤 해당 `intent_id`에 대한 경고를 한 건 발행해야 한다. 경고는 `trex.alerts`로 보내며 `severity="warn"`, `code="strategy_intent_rejected"`를 사용한다. 같은 `intent_id`가 다시 들어오거나 최초 요청을 처리하는 도중 중복 요청이 들어오더라도 경고를 다시 발행해서는 안 된다. 반면, 같은 slot을 대상으로 하더라도 `intent_id`가 다르면 각각 별개의 최초 거부로 보고 경고를 발행해야 한다.
경고에는 `instance_key`, `intent_id`, `intent_type`, `reason="reconcile_warned"`, `broker_submitted=false`를 반드시 포함한다. `slot_id`, `portfolio_id`, `symbol`, `strategy_id`는 신뢰할 수 있는 값이 확인된 경우에만 포함하며, 알 수 없는 값을 추정하거나 원본 요청에서 그대로 복사해서는 안 된다. 매수 요청의 `strategy_id`와 매도 요청의 `sell_strategy_id`는 경고에서 공통 필드인 `strategy_id`로 표현한다.
정상 승인된 요청, `reconcile_warned` 이외의 사유로 거부된 요청, 최종 결과 저장에 실패한 요청, 그리고 safety-sell·kill-switch·watchdog 같은 내부 안전 경로에서는 이 경고를 발행하지 않는다. 경고 발행기가 없거나 부가 정보를 조회하지 못하거나 메시지 전송이 실패하더라도 이미 결정된 거부 결과, RPC 응답, 재조정 경고, slot 상태 및 broker·ledger·portfolio-hub 동작을 중단하거나 변경해서는 안 된다. 경고와 이 처리 경로의 로그에는 계좌 식별정보, credential, token, password, DSN, 원본 요청, 전략의 `extra`, 외부 시스템의 원본 payload 또는 예외 원문을 포함해서는 안 된다.
## Verification
- 정상 동작: 재조정 경고가 설정된 slot에 새로운 매수 요청과 매도 요청을 각각 제출하면, 최종 거부 결과가 저장된 뒤 `strategy_intent_rejected` 경고가 `warn` 등급으로 요청마다 한 건씩 발행된다. 각 경고에는 필수 필드와 `broker_submitted=false`가 들어 있다. 매수의 `strategy_id`와 매도의 `sell_strategy_id`는 값이 있을 때 경고의 `strategy_id`로 나타나며, `slot_id`, `portfolio_id`, `symbol`은 저장소에서 확인된 값만 포함된다.
- 중복 및 제외 조건: 같은 `intent_id`를 같은 프로세스에서 반복하거나 기존 영속 intent 저장소로 서비스를 다시 구성한 뒤 반복해도 경고는 추가로 발행되지 않고 저장된 기존 결과가 그대로 반환된다. 재조정 경고가 설정된 같은 slot에 서로 다른 `intent_id` 두 건을 제출하면 경고는 두 건 발행된다. 최초 요청을 처리하는 도중 들어온 중복 요청과 `duplicate_in_flight`, 정상 승인, `reconcile_warning_unavailable`, `capacity_full`, 입력 검증 실패 등 `reconcile_warned`가 아닌 결과는 경고를 만들지 않는다.
- 처리 순서 및 장애 조건: 호출 순서를 기록하는 테스트용 저장소와 경고 발행기를 사용해 `IntentStore.finalize()`가 경고 발행보다 먼저 완료되는지 확인한다. 최종 결과 저장이 실패하면 경고를 발행하지 않는다. 경고 발행기가 없거나 실패를 반환하거나 전송 계층에 장애가 발생해도 원래의 `reconcile_warned` 거부 결과와 RPC 응답 본문은 유지된다. 이 과정에서 broker 주문 제출·취소, slot 상태 전이, 재조정 경고 설정·해제, ledger 기록 또는 portfolio-hub 쓰기가 추가로 발생하지 않는다.
- 보안 및 호환성: 원본 요청과 `extra`에 credential·token·account·DSN·exception 식별용 값을 넣더라도 경고 제목, 필드, 직렬화된 전송 내용과 출력 로그에 해당 값이나 금지된 key가 나타나지 않는다. 부가 정보를 읽지 못하거나 값이 없으면 해당 선택 필드만 생략하고 필수 필드와 거부 결과는 유지한다. 운영 daemon의 `IntentService`에는 기존 공용 `AlarmEmitter`가 연결되며, 경고 발행기 없이 `IntentService`를 직접 생성하는 기존 사용 방식도 계속 동작한다.
- 검사 방법: `tests/application/test_reconcile_warning_blocking.py` 또는 별도의 intent 경고 테스트에 매수·매도, 같은 `intent_id`와 다른 `intent_id`, 재시작을 모사할 수 있는 intent 저장소, 처리 순서, 발행 실패, 경고 필드 허용 목록과 부작용 없음 조건을 추가한다. 이어서 `tests/application/test_messaging_intent_consumer.py`, `tests/application/test_alarm_emitter.py`, daemon 연결 테스트, AC-0023·AC-0024 관련 회귀 테스트, 전체 저장소 테스트, Ruff, ProofLine validation과 `git diff --check`를 실행한다.
+2 -2
View File
@@ -1,5 +1,5 @@
{
"schema_version": 1,
"next_line_number": 7,
"next_ac_number": 20
"next_line_number": 9,
"next_ac_number": 26
}
+2 -2
View File
@@ -29,14 +29,14 @@ buy/sell intent의 venue 누락과 explicit null을 SOR로 정규화하고 calle
- SOR 단독, 누락, unknown code 또는 time으로 actual venue를 추론하지 않는다. SOR 주문에서 actual이 확인되지 않으면 거짓 ledger actual을 만들지 않는다.
- historical outbox의 SOR는 `order_venue=SOR`, actual `venue=NULL`로 보존한다. NULL-only backfill로 migration replay가 정상 order/execution 조합을 재해석하지 않게 한다.
- omitted와 explicit null wire input, KRX/NXT/SOR preservation, AUTO rejection, KIS payload, partial fill·cancel·restart, outbox round-trip 및 migration semantics를 repository tests와 실제 PostgreSQL에서 검증한다.
- 기존 active AC-0018과 AC-0019의 exact 의미를 변경하지 않고 이 Requirement의 구현 evidence를 해당 criterion에 `satisfy`로 결속한다.
- 기존 active AC-0018과 AC-0019를 이 Requirement의 구현 evidence에 `satisfy`로 결속한다. 이후 requester가 명시 승인한 AC revision은 해당 criterion의 최신 canonical scope다.
## Constraints
- actual venue, terminal execution state 또는 broker acceptance를 wall-clock이나 default로 추론하지 않는다.
- cancel·restart는 stored original venue와 original order identity를 사용한다.
- KIS raw payload, credential, token 또는 connection string을 public wire·document evidence에 기록하지 않는다.
- active AC-0018과 AC-0019의 본문·status를 이 satisfy binding으로 변경하지 않는다.
- 이 satisfy binding 자체만으로 active AC-0018과 AC-0019의 본문·status를 변경하지 않는다. requester가 명시 승인한 revision은 예외다.
- TVR session/markets 판단과 strategy policy는 변경하지 않는다.
- Normative Design Documents: 없음. 이 REQ와 만족 대상으로 결속한 active AC-0018·AC-0019가 normative specification을 소유한다.
+60
View File
@@ -0,0 +1,60 @@
---
id: "dcy-0007"
status: confirmed
---
# Reconciliation 수량 불일치 알림
## Problem
`trex-core`는 broker와 core slot의 보유수량이 다른 B-case, portfolio-hub와 broker의 종목별 보유수량 합계가 다른 C-case를 reconciliation warning으로 기록한다. 그러나 두 mismatch 경로는 `AlarmEmitter.publish_alarm()`을 호출하지 않아 `trex.alerts` event가 만들어지지 않고, `trex-notifier`가 운영자에게 Telegram 알림을 전달할 수 없다. 현재 scheduler가 같은 mismatch를 반복 탐지하므로 단순 publish 추가는 scheduler tick마다 중복 메시지를 만들 수 있으며, 수량이 바뀌거나 mismatch가 해소된 사실도 별도 lifecycle event로 구분되지 않는다.
기대 상태는 B/C mismatch의 최초 탐지와 material quantity change를 정제된 `trex.alerts` event로 알리고, 기존 warning 해소 권한에 따라 mismatch가 실제로 해소되면 recovery/resolved event를 정확히 한 번 알리는 것이다. 알림 상태나 publish 실패가 reconciliation, warning persistence·clear, slot lifecycle 또는 broker/portfolio-hub 수량을 변경하거나 중단해서는 안 된다.
## Evidence
- 원본 요청: GitHub issue #64 `https://github.com/fourblocks2020/trex-core/issues/64`는 B/C mismatch의 최초 탐지·material change·해소 알림, unchanged warning 억제, 정제된 instance key·symbol·case code·비교 수량, best-effort publish와 B/C·변경·해소 테스트를 요구한다. issue는 2026-08-23 생성되었고 조사 시점에 comment 없이 open 상태였다.
- 조사 기준 `trex-core` revision은 `24a64b55d17263ade8d59f30af50d2a0e5220eae`이며 working tree는 Line 시작 전 clean이었다. issue #64 또는 reconciliation alarm을 참조하는 open/all PR 검색 결과는 없었다.
- `trex_core/application/reconciliation.py:100-199`의 `reconcile_core_holdings()`는 `alarm_emitter`를 받지만 B-case에서는 structured WARNING log와 `warn_store.set_warned(case_code="B")`만 수행한다. 같은 함수의 A-case broker-flat 경로만 `core_holding_broker_flat` alarm을 발행한다.
- `trex_core/application/reconciliation.py:202-280`의 `reconcile_positions()`는 portfolio-hub `asset_quantity`를 symbol별로 합산해 broker 수량과 비교하고 C-case warning을 저장하지만 `alarm_emitter` parameter나 publish 호출이 없다.
- `trex_core/application/snapshot_scheduler.py:306-334`는 C-case `reconcile_positions()`에 emitter를 전달하지 않고 B-case `reconcile_core_holdings()`에만 emitter를 전달한다. 두 reconciliation 호출의 예외는 scheduler를 중단하지 않도록 각각 swallow·log된다.
- `trex_core/application/alarm_emitter.py:35-93`은 allowlist된 caller fields로 `trex.alerts` envelope를 만들고 `TransportUnavailableError`와 그 밖의 exception을 swallow하는 best-effort 경계를 이미 제공한다.
- `trex_core/application/portfolio.py:356-446`, `trex_core/adapters/portfolio/reconcile_warning_store.py:14-98`, migration `016_create_slot_reconcile_warning.sql`은 active warning을 slot별 durable row로 저장하고 동일 slot의 case/reason을 upsert한다. 현재 contract에는 alert fingerprint, last-notified quantity, resolved event 또는 publish 상태가 없고 `clear_warned()`는 row를 삭제한다.
- active AC-0023은 `instance_key + symbol` 기준 C-case 합계 비교, 전략 RPC 제외, intent 차단과 authoritative 합계 재검증 후 C-case warning clear를 소유한다. active AC-0021은 별도 `ledger_skip`의 deduplicated unresolved/resolved alert lifecycle을 소유하지만 reconciliation mismatch는 범위에 포함하지 않는다. active AC-0012의 dedupe는 미확정 cancel alarm에 한정된다.
- `portfolio-hub` canonical source revision `5d6a0ad013900668930fe947f342e1ad1824edaa`의 `app/schemas/v1.py:302-338`은 `PortfolioItem.asset_quantity`와 portfolio list response를 정의하고, `app/api/v1_portfolios.py:315-363`은 service token의 daemon ownership으로 portfolio 목록을 반환한다. 이는 C-case가 소비하는 portfolio별 수량 source와 instance 경계를 확인한다. 해당 checkout에서 `uv run python -m pytest -o addopts='' -q tests/test_v1_portfolios_list.py tests/test_v1_portfolios_daemon_assignment.py`는 `37 passed, 3 warnings`였다.
- 현재 baseline focused 검증 `uv run pytest -q tests/application/test_reconcile_warning_blocking.py tests/application/test_alarm_emitter.py`는 `58 passed`였다. 기존 tests는 B/C warning·logging과 generic alarm envelope를 각각 검증하지만 B/C alarm, unchanged suppression, quantity change와 resolved event를 결합해 검증하지 않는다.
- Line scaffold preflight 전에 upstream AC-0022의 추가 `## Scope note` heading 때문에 ProofLine 0.9.1 validation이 실패했다. 문구를 바꾸지 않고 heading만 inline scope note로 정규화한 뒤 `proofline validate`와 `proofline line init --dry-run`이 통과했다. 이는 새 capability와 무관한 `housekeeping`이다.
## Scope
- AC admission은 reconciliation mismatch의 운영자-visible notification lifecycle을 독립적으로 검증할 장기 active criterion을 `create`하는 후보이며 next allocator 기준 예상 ID는 AC-0024다. AC-0023의 blocking·clear authority나 AC-0021의 ledger-skip alert 의미를 확장 해석해 `satisfy`하지 않는다.
- B-case core-slot↔broker quantity mismatch와 C-case portfolio-hub symbol aggregate↔broker mismatch의 새 탐지에서 `AlarmEmitter`를 통해 sanitized `trex.alerts` event를 발행한다.
- alert dedupe identity는 B-case에서 slot별, C-case에서 `instance_key + symbol`별이다. 동일한 active mismatch의 반복 scheduler tick은 Telegram message로 이어질 event를 재발행하지 않는다. 동일 identity의 case별 비교 quantity tuple이 바뀌면 material change event를 한 번 발행한다.
- recovery/resolved event는 기존 durable warning clear 경로인 B-case operator `sync`, C-case operator `verify`의 symbol aggregate 재검증에서만 발행한다. scheduler의 수량 일치 관찰은 durable warning을 자동 clear하거나 recovery를 발행하지 않는다.
- unchanged suppression은 process restart 뒤에도 유지하는 durable dedupe다. event fingerprint는 alarm publish가 성공한 뒤에만 기록하며 publish 실패 시 reconciliation을 중단하지 않고 다음 detection에서 다시 시도할 수 있어야 한다.
- event에는 최소 `instance_key`, `symbol`, `case_code`, broker quantity와 case별 비교 quantity를 포함하되 account number/hash, credential, token, DSN, raw broker/portfolio-hub payload와 exception 본문은 포함하지 않는다.
- alarm publish 실패는 best-effort로 swallow되어 detection, warning persistence·clear, scheduler 진행과 slot 상태에 영향을 주지 않는다.
- repository-local tests에서 B/C 최초 탐지, unchanged suppression, 비교 수량 변경, resolution, publish failure, 민감 필드 부재를 독립적으로 검증한다.
## Out of Scope
- B/C mismatch 판정식, `case_code` 의미, AC-0023의 C-case 전략 RPC 제외·intent 차단과 기존 warning clear authority 변경
- broker, core slot 또는 portfolio-hub 수량의 자동 보정과 operator `sync`/`verify` 승인 절차 변경
- A-case broker-flat 및 기존 alarm inventory의 의미 변경
- `trex-notifier` 자체의 Telegram routing·message rendering 구현, RabbitMQ topology 변경, production credential·배포·실계좌 검증
- push, PR, merge, release, issue closure와 production migration
## Risks and Unknowns
- slot별 warning row를 그대로 event identity로 사용하면 C-case의 동일 `instance_key + symbol`에 속한 여러 portfolio/slot이 Telegram 알림을 중복 생성할 수 있다. 반대로 symbol 단일 identity는 B-case에서 여러 slot의 독립 수량 차이를 잃을 수 있다.
- 현재 warning upsert는 `updated_at`과 reason을 매 tick 갱신하고 clear 시 row를 삭제하므로, 단순 timestamp 비교로는 unchanged suppression이나 restart-safe resolved event를 증명할 수 없다.
- event fingerprint를 alarm publish 성공 뒤에만 기록해야 하므로 publish 성공 판정과 durable dedupe write 사이의 failure window에서 duplicate 가능성이 있다. 이 경계는 reconciliation transaction과 결합하지 않으면서 retry 가능한 at-least-once 성질을 보존하도록 구현·검증해야 한다.
- C-case 해소는 AC-0023에 따라 symbol aggregate 재검증과 C-case-only clear가 필요하다. scheduler의 process-local `SnapshotStore.warning` clear를 durable recovery authority로 오인하면 premature recovery event가 발생할 수 있다.
### Confirmed Decisions
- B-case dedupe identity는 slot별이고 C-case dedupe identity는 `instance_key + symbol`별이다.
- recovery/resolved authority는 기존 durable clear 경로(B=`sync`, C=`verify`)로 제한한다.
- material change는 동일 identity의 case별 비교 quantity tuple 변경이다.
- unchanged suppression은 restart-safe durable dedupe이며 event fingerprint는 publish 성공 뒤에만 기록한다. 실패한 publish는 business path를 중단하지 않고 다음 detection에서 재시도한다.
- 요청자 genichin이 2026-08-25 위 결정을 명시하고 dcy-0007 승인을 요청했다. Open Questions는 모두 canonical Scope와 Risks에 반영되어 해소되었다.
+3
View File
@@ -0,0 +1,3 @@
---
id: "line-0007"
---
+50
View File
@@ -0,0 +1,50 @@
---
id: "req-0007"
status: approved
discovery: "dcy-0007"
criteria:
create:
- "ac-0024"
update: []
retire: []
satisfy: []
---
# Reconciliation 수량 불일치 알림
## Objective
`trex-core`가 B-case(core slot↔broker)와 C-case(portfolio-hub symbol aggregate↔broker) 수량 불일치의 최초 탐지, material quantity change와 authoritative resolution을 중복 폭주 없이 `trex.alerts`에 발행하게 한다. 알림 lifecycle은 process restart 뒤에도 이어지고 publish 실패는 다음 detection에서 재시도할 수 있어야 하며, reconciliation·warning·slot·broker·portfolio-hub의 기존 동작과 권한을 중단하거나 변경하지 않는다.
## Scope
- B-case mismatch는 stable slot identity별로 추적하고, C-case mismatch는 `instance_key + symbol`별로 추적한다. C-case의 동일 symbol에 여러 portfolio/slot이 있어도 하나의 aggregate mismatch lifecycle로 취급한다.
- B-case 비교 quantity tuple은 core slot quantity와 broker quantity이고, C-case 비교 quantity tuple은 같은 instance·symbol의 portfolio-hub `asset_quantity` 합계와 broker quantity다.
- 이전에 성공적으로 알린 fingerprint가 없는 active mismatch는 detected event를 발행한다. 같은 identity와 같은 quantity tuple의 scheduler 반복 탐지는 event를 재발행하지 않는다.
- active mismatch의 case별 quantity tuple이 바뀌면 changed event를 한 번 발행하고 성공한 새 fingerprint를 durable하게 보존한다.
- B-case recovery/resolved event는 operator `sync`가 broker-authoritative quantity로 warning을 clear하는 경로에서만 발행한다. C-case recovery/resolved event는 operator `verify`가 `instance_key + symbol` aggregate 일치를 재검증하고 C-case warning을 clear하는 경로에서만 발행한다.
- scheduler가 일치 quantity를 관찰하거나 process-local warning을 clear하는 것만으로 durable warning을 자동 clear하거나 recovery event를 발행하지 않는다.
- detected·changed·resolved event는 lifecycle state를 구분하고 `instance_key`, `symbol`, `case_code`, broker quantity 및 case별 비교 quantity를 포함한다. B-case identity에는 stable slot identifier를 포함할 수 있으나 account number/hash, credential, token, DSN, raw external payload 또는 exception 본문은 포함하지 않는다.
- successful publish 뒤에만 해당 lifecycle fingerprint를 durable하게 기록한다. detected/changed publish가 실패하면 다음 동일 detection에서 재시도한다. resolved publish가 실패해도 warning clear는 진행하며, notification state가 writable이면 pending resolved transition을 보존하여 후속 retry opportunity에서 재시도한다.
- 기존 `AlarmEmitter`와 `trex.alerts` envelope·routing contract를 재사용하고 새 event가 `trex-notifier`의 기존 alarm 소비 경계에 도달하게 한다.
- B/C 최초 탐지, C-case 다중 portfolio 단일 event, unchanged suppression, quantity change, restart, B sync resolution, C verify resolution, publish failure retry와 sensitive field 부재를 repository-local automated tests로 검증한다.
## Constraints
- AC-0023의 C-case `instance_key + symbol` aggregate 판정, strategy RPC 제외, intent 차단, C-case-only clear와 operator authority를 완화하거나 대체하지 않는다.
- B/C warning의 `case_code`, mismatch 판정식, slot lifecycle 및 broker/portfolio-hub quantity는 알림을 위해 변경하지 않는다.
- notification state는 restart-safe하고 replay-safe한 persistence로 관리해야 하며 기존 migration runner의 반복 적용을 견뎌야 한다.
- publish success와 durable fingerprint write 사이의 failure window에서는 at-least-once delivery에 따른 duplicate가 가능하지만, fingerprint를 publish 시도 전에 기록하여 event를 영구 유실시키는 방식은 허용하지 않는다.
- alarm publish 및 notification-state persistence failure는 reconciliation, warning set/clear, operator sync/verify와 scheduler를 abort시키지 않는 best-effort 경계여야 한다. resolved publish가 실패하고 notification state가 writable이면 pending transition을 보존하지만, notification-state persistence 자체가 실패하면 authoritative warning clear를 되돌리거나 지연하지 않으며 해당 event 유실 가능성을 sanitized log로 남긴다.
- 기존 alarm consumer와 wire compatibility를 유지하며 RabbitMQ topology 또는 `trex-notifier` message rendering 변경을 요구하지 않는다.
- credential, account 식별정보, token, DSN 및 raw KIS/portfolio-hub payload를 test fixture, event, log 또는 evidence에 기록하지 않는다.
- Normative Design Documents: 없음. 이 REQ와 AC-0024가 normative specification을 소유한다.
## Non-Goals
- B/C mismatch 판정, warning persistence·blocking·clear policy 또는 AC-0023 수정
- scheduler 관찰에 의한 automatic warning clear 또는 recovery 발행
- broker/core/portfolio-hub quantity 자동 보정과 operator `sync`/`verify` 승인 절차 변경
- A-case broker-flat 및 기존 alarm inventory의 semantics 변경
- `trex-notifier`의 Telegram routing·rendering 구현, RabbitMQ topology 변경과 production delivery 검증
- live credential·account·database·RabbitMQ·portfolio-hub 연결, production migration, deployment, release 또는 issue closure
+61
View File
@@ -0,0 +1,61 @@
---
id: "dcy-0008"
status: confirmed
---
# Reconcile-warned strategy intent rejection alerts
## Problem
활성 reconciliation warning이 있는 slot에 외부 strategy가 신규 buy 또는 sell intent를 제출하면 `IntentService`는 `reconcile_warned` 사유로 안전하게 거부하지만, 현재는 RPC 응답과 `intent_log` 외에 운영자에게 즉시 전달되는 신호가 없다. Strategy가 마지막 slots RPC 이후 발생한 warning을 알지 못한 채 intent를 제출하는 race에서도 거부 자체는 정상 동작하지만, 운영자는 반복되는 거부와 전략 정체의 원인을 별도 조사 전까지 인지하기 어렵다.
기대 상태는 durable intent idempotency와 기존 reconciliation 차단 권한을 유지하면서, 새 `intent_id`가 최초로 `reconcile_warned` 최종 거부된 시점에만 allowlist 기반 `trex.alerts` 경고를 best-effort로 한 건 발행하는 것이다. 동일 `intent_id`의 retry는 저장된 결정을 그대로 반환하되 추가 경고를 만들지 않아야 하며, 알림 실패가 intent 거부·응답·저장 또는 다른 business path를 바꾸어서는 안 된다.
## Evidence
- GitHub issue [#74](https://github.com/fourblocks2020/trex-core/issues/74)는 실운영에서 warning 활성 뒤 strategy intent가 거부됐지만 즉시 알림이 없어 원인 파악이 지연된 사례를 기록하고, 최초 `reconcile_warned` 거부 알림·중복 억제·민감정보 제외·non-blocking 동작을 요구한다. 확인 시 issue는 open이고 후속 comment나 연결된 PR은 없었다.
- 현재 `trex_core/application/intents.py`의 `submit_buy()`와 `submit_sell()`은 `IntentStore.try_insert()`를 dedup gate로 호출한 뒤 admission 결과를 `finalize()`하고 반환한다. Duplicate `intent_id`는 `_dedup()`에서 기존 결정을 반환하므로 새 admission을 실행하지 않는다.
- 같은 파일의 buy·sell warning gate는 `warn_store.is_warned(slot_id)`가 참이면 `IntentDecision(status="rejected", reason="reconcile_warned")`을 반환한다. `IntentService` constructor에는 `alarm_emitter` 의존성이 없고, 이 최종 거부를 `trex.alerts`에 발행하는 호출도 없다.
- `trex_core/runtime/daemon.py`는 하나의 `AlarmEmitter`를 생성해 ledger, kill-switch, scheduler와 execution service에 주입하지만 `IntentService`에는 주입하지 않는다. 따라서 production intent path에는 재사용 가능한 publisher가 존재하지만 wiring이 빠져 있다.
- `trex_core/application/alarm_emitter.py`의 `publish_alarm()`은 `severity`로 `critical | warn | info`를 사용하고 publish exception을 삼켜 호출자의 business path를 중단하지 않는 best-effort 계약이다. `docs/interface-contract.md`도 warning wire literal을 `warn`으로 정의한다.
- `trex_core/adapters/persistence/postgresql.py`의 `PostgreSQLIntentStore`는 `intent_id` unique insert를 durable dedup gate로 사용하고 final `status`, `slot_id`, `reason`을 저장한다. 별도 notification state 없이도 “최초 처리된 intent만 발행 후보”라는 경계는 이 기존 gate로 정할 수 있다.
- `tests/application/test_reconcile_warning_blocking.py`와 `tests/application/test_slots_handler_quantity_mismatch.py`는 buy/sell `reconcile_warned` 거부와 last-RPC race를 검증하지만 alarm 발행을 단언하지 않는다. `tests/application/test_messaging_intent_consumer.py`는 duplicate `intent_id`가 저장된 동일 결정을 반환하는 idempotency를 검증한다.
- 현재 `main` revision `2e314232e3073381b6fc9ee268ae8e12fbcc8749`에서 관련 baseline인 `tests/application/test_reconcile_warning_blocking.py`, `tests/application/test_slots_handler_quantity_mismatch.py`, `tests/application/test_messaging_intent_consumer.py`, `tests/application/test_alarm_emitter.py`를 실행한 결과 `84 passed`였다.
- Git history에서 `c193388`은 external strategy buy/sell에 warning gate를 의도적으로 추가하면서 safety-sell·kill-switch·watchdog가 `IntentService`를 우회하도록 정했다. 이번 Line은 그 차단 정책이나 내부 안전 경로를 변경하지 않고, 이미 확정된 외부 intent 거부의 운영 가시성만 추가한다.
## Scope
- AC admission은 새 `AC-0025` **create**로 분류한다. 가장 가까운 `AC-0023`은 C-case strategy RPC 제외·intent blocking·warning clear 권한을 소유하고, `AC-0024`는 B/C mismatch 자체의 detected/changed/resolved lifecycle을 소유한다. 이번 동작은 “외부 strategy intent가 실제로 차단된 최초 사건”이라는 별도 운영 신호이며 독립 PASS/FAIL과 장기 active 가치가 있으므로 기존 AC의 update 또는 satisfy로 흡수하지 않는다. `update`, `retire`, `satisfy`, `release evidence` 대상은 없다.
- `IntentService`의 buy·sell 처리에서 새 `intent_id`의 최종 decision이 `status="rejected"` 및 `reason="reconcile_warned"`로 확정되고 `IntentStore.finalize()`가 성공한 뒤에만 `AlarmEmitter.publish_alarm()`을 한 번 호출한다.
- Alarm은 기존 wire 계약에 맞춰 `severity="warn"`, code `strategy_intent_rejected`를 사용한다. Fields는 최소 allowlist인 `instance_key`, `intent_id`, `intent_type`, `reason`, `broker_submitted=false`를 포함하고, 요청 또는 canonical slot에서 안전하게 확인 가능한 경우에만 `slot_id`, `portfolio_id`, `symbol`, normalized `strategy_id`를 추가한다.
- Buy의 canonical `slot_id` 입력 또는 legacy `portfolio_id` mapping과 sell의 `slot_id` 입력 모두에서 가능한 slot context를 수집하되, context 조회 실패나 optional field 부재는 원래 `reconcile_warned` decision을 바꾸거나 알림을 위해 새로운 mutation을 유발하지 않는다.
- 동일 `intent_id`의 duplicate/retry는 기존 durable `intent_log` 결정을 반환하고 alarm을 재발행하지 않는다. 최초 처리 중 관찰되는 `processing`/`duplicate_in_flight` 응답도 별도 alarm 대상이 아니다.
- Alarm publish 성공 여부와 무관하게 원래 거부 decision과 RPC 응답은 유지된다. Publisher 부재와 transport failure는 non-blocking이며 별도 durable outbox, notification-state row 또는 retry loop를 만들지 않는다.
- Alarm title·fields·rendered log에는 credential, token, account number/hash, DSN, raw strategy payload, raw broker/provider payload와 raw exception body를 포함하지 않는다.
- Production daemon은 이미 생성하는 동일 `AlarmEmitter`를 `IntentService`에 주입한다. Optional dependency를 사용하는 직접 application test와 non-daemon construction은 기존 동작을 유지한다.
- Verification은 buy와 sell의 최초 경고 각 한 건, 같은 `intent_id` retry의 추가 경고 0건, 다른 `intent_id`의 독립 경고, admitted 및 다른 rejection reason의 무발행, publish failure의 non-blocking, payload allowlist·`broker_submitted=false`, production wiring과 기존 warning blocking/idempotency regression을 포함한다.
## Out of Scope
- `AC-0023`의 B/C warning 판정, slots RPC 제외, buy/sell 차단, fail-closed 및 authoritative clear semantics 변경.
- `AC-0024`의 mismatch detected/changed/resolved alarm과 `reconcile_notification_state` 변경 또는 통합.
- Strategy retry·backoff·pause·circuit-breaker 정책, strategy 또는 TVR wire schema 변경, 새로운 broker order submit/cancel 동작.
- Safety-sell, kill-switch, watchdog 등 `IntentService`를 우회하는 내부 안전 경로의 차단 또는 알림.
- 다른 rejection reason에 대한 범용 intent rejection alarm, 알림 집계·rate limit·severity escalation.
- Alarm delivery를 위한 durable outbox, 별도 notification table, 재시작 후 재발행 또는 publish retry 보장. `finalize()` 성공과 alarm publish 사이 process crash로 인한 알림 유실 가능성은 이 Line에서 해소하지 않는다.
- `trex-notifier`의 message formatting·routing·delivery channel 변경과 운영 배포 자체.
## Risks and Unknowns
### Risks
- `IntentStore.finalize()` 뒤 best-effort publish를 수행하므로 process가 그 사이 종료되거나 transport가 실패하면 durable decision은 남지만 alarm은 유실된다. Issue가 outbox/retry를 제외했으므로 이 Line은 business path 독립성을 우선하며, delivery 보장이 필요해지면 별도 Requirement로 다룬다.
- Buy는 canonical `slot_id`와 legacy `portfolio_id` 두 입력을 허용하고 sell은 `sell_strategy_id`를 사용한다. Context 정규화가 admission logic을 복제하면 drift 위험이 있으므로, 알림은 decision에 영향을 주지 않는 helper와 allowlist output으로 제한하고 양쪽 입력 형태를 테스트해야 한다.
- 기존 `AlarmEmitter`는 여러 subsystem이 공유한다. Intent-specific 실패 처리나 logging 변경이 공용 emitter 계약을 확장하지 않도록 production wiring과 publisher failure test를 분리해야 한다.
- Rejection payload에 raw request 또는 `extra` dict를 복사하면 민감정보가 유출될 수 있다. 허용된 scalar field만 명시적으로 구성하고 forbidden key/value sentinel을 rendered payload와 log에서 검사해야 한다.
### Assumptions
- Issue #74의 “경고(WARNING)”는 현재 interface contract의 wire literal `severity="warn"`을 의미한다.
- “최초”의 canonical identity는 기존 `IntentStore.try_insert()`가 소유하는 `intent_id`이며, process-local set이나 새로운 persistence를 추가하지 않는다.
- `strategy_intent_rejected`는 issue가 제시한 이 기능의 canonical alarm code로 사용하며, notifier-side inventory 등록이 별도로 필요하더라도 core의 event contract를 변경하지 않는다.
+3
View File
@@ -0,0 +1,3 @@
---
id: "line-0008"
---
+51
View File
@@ -0,0 +1,51 @@
---
id: "req-0008"
status: approved
discovery: "dcy-0008"
criteria:
create:
- "ac-0025"
update: []
retire: []
satisfy: []
---
# Reconcile-warned strategy intent rejection alerts
## Objective
`trex-core`가 활성 reconciliation warning 때문에 외부 strategy의 신규 buy 또는 sell intent를 `reconcile_warned`로 최종 거부하면, 해당 `intent_id`의 최초 처리에서만 원인을 식별할 수 있는 sanitized warning event를 `trex.alerts`에 한 건 발행하게 한다. 기존 durable intent idempotency와 warning 차단 권한을 유지하고, duplicate intent 및 무관한 decision의 알림을 억제하며, 알림 실패가 intent decision·저장·응답 또는 broker 동작에 영향을 주지 않도록 한다.
## Scope
- `IntentService`가 새 `intent_id`를 처리하여 buy 또는 sell decision을 `status="rejected"`, `reason="reconcile_warned"`로 결정하고 `IntentStore.finalize()`가 성공한 뒤에만 warning event를 발행한다.
- 동일 `intent_id`의 duplicate/retry는 기존 `IntentStore.try_insert()`와 저장된 final decision을 canonical dedup authority로 사용한다. Duplicate path, 처리 중인 intent 및 process-local cache는 새 event를 만들지 않는다.
- 서로 다른 `intent_id`가 같은 warned slot을 대상으로 각각 거부되면 각 intent는 독립적인 최초 거부 event 한 건을 만든다.
- Event는 기존 `AlarmEmitter`와 `trex.alerts` envelope·routing contract를 재사용하고 `severity="warn"`, `code="strategy_intent_rejected"`를 사용한다.
- Event fields의 필수 allowlist는 `instance_key`, `intent_id`, `intent_type`, `reason="reconcile_warned"`, `broker_submitted=false`다. Canonical slot 또는 intent에서 안전하게 확인된 값이 있을 때만 `slot_id`, `portfolio_id`, `symbol`, `strategy_id`를 추가하며, unavailable optional field는 추정하거나 raw payload에서 복사하지 않고 생략한다.
- Buy의 `strategy_id`와 sell의 `sell_strategy_id`는 event의 공통 `strategy_id` field로 정규화한다. Buy의 canonical `slot_id` 또는 legacy `portfolio_id` 입력과 sell의 `slot_id` 입력 모두에서 같은 allowlist 계약을 적용한다.
- Optional context 조회 실패, publisher 부재 및 alarm transport failure는 원래 final decision과 RPC response를 변경하거나 intent 처리를 실패시키지 않는다. 알림을 위해 slot, warning, intent, broker, ledger 또는 portfolio-hub 상태를 mutation하지 않는다.
- Production daemon은 이미 생성하는 동일 `AlarmEmitter`를 `IntentService`에 연결한다. 직접 application construction과 non-daemon entry point는 optional publisher 경계에서 기존 동작과 호환된다.
- Buy·sell 최초 거부, duplicate 억제, 서로 다른 intent의 독립 발행, 다른 decision의 무발행, publish failure, optional context, payload 보안과 production wiring을 repository-local automated tests로 검증한다.
## Constraints
- AC-0023이 소유하는 B/C warning 판정, strategy RPC 제외, buy/sell `reconcile_warned` 차단, warn-store fail-closed 및 authoritative clear semantics를 완화·대체·확장하지 않는다.
- AC-0024의 mismatch detected/changed/resolved lifecycle, fingerprint, `reconcile_notification_state`와 operator `sync`/`verify` recovery authority를 변경하거나 재사용하지 않는다.
- Alarm은 `IntentStore.finalize()`보다 먼저 발행해서는 안 된다. Finalize가 실패하거나 final decision이 확정되지 않은 경우 알림을 성공한 거부로 보고해서는 안 된다.
- Publisher return value나 failure는 저장된 rejection을 rollback하거나 RPC status/reason을 바꾸지 않는다. 별도 durable outbox, notification-state persistence 또는 retry loop를 추가하지 않는다.
- Event title, fields와 이 경로가 남기는 rendered log에는 account number/hash, credential, token, password, DSN, raw strategy `extra`, raw request, raw KIS/broker/portfolio-hub payload 또는 raw exception body가 없어야 한다.
- `broker_submitted=false`는 거부가 broker submission 전에 확정됐음을 나타내며, 이 기능이 broker submit/cancel 호출을 새로 만들거나 기존 호출 순서를 바꾸어서는 안 된다.
- Safety-sell, kill-switch, watchdog처럼 `IntentService`를 우회하는 내부 안전 경로는 계속 reconciliation warning gate와 이 strategy rejection alarm의 대상이 아니다.
- 기존 intent RPC response schema, intent idempotency, slot transition과 alarm wire compatibility를 유지하며 RabbitMQ topology 또는 notifier rendering 변경을 요구하지 않는다.
- Normative Design Documents: 없음. 이 REQ와 AC-0025가 normative specification을 소유한다.
## Non-Goals
- 다른 rejection reason의 범용 strategy intent alarm, reason별 severity 정책, rate limit·aggregation·escalation
- Strategy retry·backoff·pause·circuit-breaker 정책과 strategy/TVR wire schema 변경
- B/C mismatch 탐지·해소 알림과 strategy intent 거부 알림의 lifecycle 또는 persistence 통합
- Alarm의 exactly-once delivery, process crash 복구, durable outbox, publish retry와 재시작 후 재발행
- Warning 자동 해제, quantity 자동 보정, broker order submit/cancel, ledger write 또는 portfolio-hub write
- Safety-sell·kill-switch·watchdog 내부 경로의 신규 알림
- `trex-notifier` message formatting·delivery channel, RabbitMQ topology, production deployment, release 또는 issue closure
+14 -4
View File
@@ -88,7 +88,11 @@
{ "slot_id": "str (UUID)",
"status": "inactive | pending_buy | pending_sell | holding | error",
"symbol": "str | null",
"qty": "float | null", // slot mirror (REQ-062)
"qty": "float | null", // holding: broker authority / unavailable=null
"qty_source": "broker | unavailable | null", // non-holding=null
"broker_quantity_available": "bool", // non-holding=false
"broker_quantity_captured_at": "ISO-8601 | null", // non-holding=null
"slot_quantity": "float | null", // core daily_slot mirror (diagnostic)
"asset_quantity": "float | null", // hub authority (REQ-062)
"avg_price": "float | null",
"entry_price": "float | null",
@@ -97,7 +101,7 @@
"universe_detached": "bool",
"cash_balance": "float | null", // hub authority
"cash_estimated": "bool", // true = pending ledger outbox delta 반영
"warned": "bool", // qty != asset_quantity 시 true (REQ-062)
"warned": "bool", // slot_quantity != asset_quantity 시 true
"watchdog": { // null if watchdog store unavailable
"expired": "bool",
"last_seen": "ISO-8601 | null"
@@ -119,12 +123,16 @@
| 필드 | 권위 소스 | 비고 |
|---|---|---|
| `slot_id`, `status`, `avg_price`, `entry_price`, `forced_close_at`, `universe_detached` | slot (core) | 슬롯 row 직접 |
| `qty` | slot (core) | 슬롯 mirror — broker position |
| `qty` | broker account snapshot | holding 슬롯에서만 authoritative. unavailable/stale 시 null |
| `qty_source` | 파생(metadata) | holding 슬롯: `broker` \| `unavailable`; non-holding은 null |
| `broker_quantity_available` | 파생(metadata) | holding 슬롯 broker qty freshness 통과 여부; non-holding=false |
| `broker_quantity_captured_at` | broker account snapshot | holding 슬롯 broker qty capture 시각; non-holding=null |
| `slot_quantity` | slot (core) | 기존 daily_slot.qty mirror(진단용) |
| `symbol`, `portfolio_id` | slot (core) | 자산 바인딩 |
| `asset_quantity` | hub (SnapshotStore) | portfolio-hub 권위값 (REQ-062) |
| `cash_balance` | hub (SnapshotStore) | portfolio-hub 권위값; `cash_estimated=true` 시 ledger outbox delta 추정값 |
| `cash_estimated` | ledger outbox | pending delta 있을 때 true (REQ-029) |
| `warned` | 파생 | `qty != asset_quantity` (REQ-074 AC-2) |
| `warned` | 파생 | `slot_quantity != asset_quantity` (REQ-074 AC-2) |
| `watchdog` | WatchdogStore | 만료 여부 + last_seen; null=store 미연결 |
| `paused` | PauseStore | durable pause overlay (REQ-032) |
| `strategy_name` | hub (PortfolioItem via SnapshotStore) | REQ-116 additive. portfolio-hub 정본. hub 미제공 시 null |
@@ -132,6 +140,7 @@
| `markets` | hub (PortfolioItem via SnapshotStore) | REQ-116 additive. list[str]; hub 미제공 시 [] |
- **slot-origin strategy 필드 없음** (REQ-074 AC-5, REQ-116 additive 재도입 후에도 유지): `strategy_id`, `sell_strategy_id`, `active_strategy_id` 는 응답에 포함되지 않는다. `strategy_name`/`strategy_params`/`markets` 는 PortfolioItem(hub) 출처이므로 AC-5 제외 대상과 구분된다 — REQ-116 Notes 참조.
- **Breaking change (holding `qty`) rollout**: 기존 전략 클라이언트가 `qty=slot mirror`로 가정했다면 `slot_quantity`를 읽도록 전환해야 한다. 권장 경로는 (1) 클라이언트에 `qty_source`/`broker_quantity_available` 체크 추가 → (2) fail-closed (`qty is null`) 처리 배포 → (3) `qty`를 broker 정본으로 사용하도록 cutover 순서다.
- `spy=false`(기본): holding 슬롯 응답 후 `WatchdogStore.update(slot_id)` 호출 — 전략 폴링 = 생존 신호.
- `spy=true`: watchdog 갱신 건너뜀 — 운영자 CLI(`universe`/`slots`) 가 항상 사용.
@@ -645,6 +654,7 @@ trex-core가 `source="trex-core"`, `severity="critical"`로 발행하는 초기
결과불명 주문 탐색을 KIS 신형 일별주문체결조회
(`TTTC0081R`/`VTTC0081R`, `EXCG_ID_DVSN_CD=ALL`)로 단일화했다.
- 2026-07-06 :: BAT-056 (dcy-002 FR-6, REQ-117/118) — `OrderRequest.venue: str = "KRX"` additive 추가(기존 위치 인수 순서 불변 — AMQP wire 계약 영향 없음, 브로커 어댑터 계층 DTO까지만). KIS 어댑터: submit_order/cancel_order body에 `EXCG_ID_DVSN_CD` 항상 명시 전송(기존 미전송 → KRX 명시화 견고성 향상), cancel_order/get_current_price 시그니처 venue/market 파라미터 추가(기본값 KRX/"J" — 기존 호출자 무영향). paper broker 시그니처 정합(venue/market 무시). 실측 근거: 2026-07-06 NXT 주문/취소 rt_cd=0 확인.
- 2026-08-24 :: issue #62 — `trex.slots.req` holding row `qty`를 broker holdings snapshot 정본으로 전환. `qty_source`/`broker_quantity_available`/`broker_quantity_captured_at`/`slot_quantity`를 추가하고, broker qty unavailable 또는 stale이면 fail-closed(`qty=null`, no core/hub fallback)로 응답.
## REQ-123 Contract Addendum: cost-basis canonical parsing & sync diagnostics
+6 -5
View File
@@ -141,10 +141,11 @@
1. 요청 payload 디코드: `{filter ∈ {all, inactive, pending, holding, error}, spy: bool}`.
2. `slot_repo.list_slots(instance_key)` → status 기준 서버 필터 적용(`inactive`/`pending`/`holding`/`error`; 미인식 = `all`).
3. `spy=false` 면 `slot_repo.list_holding_slots()` 의 모든 holding 슬롯에 `WatchdogStore.update(slot_id)`.
4. 각 슬롯에 `slot_to_dict(s, snapshot_store=..., watchdog_store=..., pause_store=..., watchdog_timeout_sec=...)` 적용 → 병합 row 산출:
- slot 코어 필드: `slot_id/status/symbol/qty/avg_price/entry_price/portfolio_id/forced_close_at/universe_detached`
4. 각 슬롯에 `slot_to_dict(s, snapshot_store=..., watchdog_store=..., pause_store=..., watchdog_timeout_sec=...)` 적용 후 broker holdings snapshot 오버레이를 적용해 병합 row 산출:
- slot 코어 필드: `slot_id/status/symbol/slot_quantity/avg_price/entry_price/portfolio_id/forced_close_at/universe_detached`
- holding authoritative 수량: `qty`(broker), `qty_source`, `broker_quantity_available`, `broker_quantity_captured_at` (`qty_source=unavailable`이면 `qty=null`, mirror fallback 금지)
- hub 권위 필드: `asset_quantity`/`cash_balance` (SnapshotStore)
- 파생: `warned = (qty != asset_quantity)` (REQ-074 AC-2)
- 파생: `warned = (slot_quantity != asset_quantity)` (REQ-074 AC-2)
- 추정 현금(REQ-029 OQ-4): `cash_estimated=True` + `cash_balance += pending ledger delta`(best-effort)
- 워치독: `watchdog: {expired, last_seen}` inline (WatchdogStore)
- 오버레이: `paused` (PauseStore, REQ-032)
@@ -286,11 +287,11 @@ timeout 미지정·취소 실패로 잔여 고착된 pending 슬롯을 처분:
집행 루프 fill 전이 직후 `ExecutionService` 가 ledger 기록을 outbox 에 적재해 슬롯 전이 critical path 와 ledger write 를 분리한다(원장 기록 단일 주체 = 집행 경로, NFR-3).
- ledger skip 가시화(dcy-011/REQ-018, issue #11): `_record_ledger` 가 `portfolio_id` 또는 `execution_id` 누락으로 skip 하면 무성 반환하지 않고 **WARN 로그 + `ExecutionResult.ledger_skipped` 카운터**로 노출한다(슬롯 전이는 그대로 진행). portfolio_id 필수화(REQ-007)로 주원인은 제거됐고, 잔여 누락은 운영 점검 쿼리(REQ-055: closed 슬롯 ↔ `ledger_outbox` anti-join, `find_ledger_gap_slots`)로 영구 탐지한다.
- ledger skip 가시화(dcy-011/REQ-018, issue #11, issue #68): `_record_ledger` 가 completed fill을 정상 `ledger_outbox` 경로에 올릴 수 없으면 무성 반환하지 않고 **WARN 로그 + `ExecutionResult.ledger_skipped` 카운터 + durable `ledger_skip` row + best-effort `trex.alerts` alarm** 으로 남긴다(슬롯 전이는 그대로 진행). unresolved row는 `trex-core ledger skips list` 로 조회하고, manual backfill 또는 operator resolve 후 resolved 상태와 별도 resolved alarm을 남긴다.
1. fill 감지: `BrokerPort.poll_fill` 이 `state="filled"` 와 결정적 `execution_id`(paper: `paper:{broker_order_id}:{filled_qty}` / KIS: `kis-daily-ccld:{ord_dt}:{ord_gno_brno}:{odno}:{tot_ccld_qty}`), 계산된 `fee`(`calculate_fee(side, amount)`)·`tax`(`calculate_tax(side, amount)`)를 운반. fee/tax 는 단일 정책 지점(`broker_port.py`)에서 env 로 관리한다(dcy-026/REQ-018 CHG-20260612-01): `TREX_KIS_FEE_BPS`(기본 1.5 = 0.015%, 매수·매도 공통)·`TREX_KRX_TAX_SELL_BPS`(기본 20.0 = 0.2%, **매도 증권거래세**; 매수 tax=0). paper·KIS 가 동일 정책을 공유하며, KIS daily-ccld 응답은 per-fill fee/tax 필드를 제공하지 않아 정책 추정값을 사용한다(향후 노출 endpoint 도입 시 브로커 정본 우선).
2. 슬롯 전이: `pending_buy→holding`(avg_price/qty) 또는 `pending_sell→closed` 를 advisory lock 안에서 먼저 commit. 이 단계가 critical path.
3. ledger enqueue: 슬롯의 `portfolio_id`(== ledger `asset_id`, OQ-1 결정) 가 있으면 `LedgerTrade(asset_id, order_id, execution_id, side, qty, price, fee, tax, executed_at)` 을 `LedgerOutbox.enqueue` 로 적재. 행 식별자는 `correlation_id = {asset_id}:{order_id}:{execution_id}`(로컬 dedup). `portfolio_id` 부재면 ledger 기록 skip 후 슬롯 전이는 정상 유지.
3. ledger enqueue: 슬롯의 `portfolio_id`(== ledger `asset_id`, OQ-1 결정) 와 결정적 `execution_id`가 있으면 `LedgerTrade(asset_id, order_id, execution_id, side, qty, price, fee, tax, executed_at)` 을 `LedgerOutbox.enqueue` 로 적재한다. 행 식별자는 `correlation_id = {asset_id}:{order_id}:{execution_id}`(로컬 dedup)이다. completed fill이 `portfolio_id`/`execution_id` 누락 또는 unsupported actual venue 등 genuinely non-recordable 상태이면 `ledger_outbox` 대신 `ledger_skip` 에 unresolved 감사 row를 남기고 알람을 best-effort 발행한다. unknown actual venue alone(`order_venue=SOR`, `execution_venue=NULL`)는 계속 정상 outbox 경로다.
4. `clear_dispatch` 로 `broker_order_id` 해제.
## Flow: `ledger writer` (백그라운드, BAT-006)
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "trex-core"
version = "0.1.1"
version = "0.4.0"
description = "Shared core for the trex product family; first domain: slot lifecycle (storage-agnostic pure domain)"
requires-python = ">=3.12"
dependencies = [
+20
View File
@@ -175,6 +175,26 @@ def test_kis_fill_status_carries_sor_actual_execution_venue():
assert status.execution_venue == "NXT"
def test_kis_sor_fill_without_actual_exchange_keeps_execution_venue_unknown():
status = KISBroker._fill_status_from_row(
{
"ord_qty": "10",
"tot_ccld_qty": "10",
"avg_prvs": "70000",
"ord_dt": "20260824",
"ord_gno_brno": "12345",
"odno": "0000000002",
"excg_id_dvsn_cd": "SOR",
},
side="buy",
today="20260824",
fallback_odno="0000000002",
)
assert status.is_filled
assert status.execution_venue is None
# ---------------------------------------------------------------------------
# REQ-117 AC-4: 기존 위치 인수 순서 불변
# ---------------------------------------------------------------------------
@@ -0,0 +1,163 @@
from __future__ import annotations
import threading
from datetime import UTC, date, datetime
from uuid import uuid4
from trex_core.adapters.persistence.ledger_skip_store import PostgreSQLLedgerSkipStore
def test_ledger_skip_store_round_trips_and_dedupes(pg_clean):
store = PostgreSQLLedgerSkipStore(pg_clean)
slot_id = uuid4()
record, notify = store.record_skip(
instance_key="inst-1",
slot_id=slot_id,
portfolio_id="PF-1",
symbol="005930",
order_id="O-1",
execution_id=None,
side="buy",
qty=10.0,
price=1000.0,
executed_at=datetime(2026, 8, 24, 0, 0, tzinfo=UTC),
order_venue="SOR",
execution_venue="unknown",
skip_reason="missing_execution_id",
)
again, notify_again = store.record_skip(
instance_key="inst-1",
slot_id=slot_id,
portfolio_id="PF-1",
symbol="005930",
order_id="O-1",
execution_id=None,
side="buy",
qty=10.0,
price=1000.0,
executed_at=datetime(2026, 8, 24, 0, 0, tzinfo=UTC),
order_venue="SOR",
execution_venue="unknown",
skip_reason="missing_execution_id",
)
assert notify is True
assert notify_again is False
assert again.skip_key == record.skip_key
listed = store.list_records(instance_key="inst-1", status="unresolved")
assert len(listed) == 1
assert listed[0].skip_reason == "missing_execution_id"
def test_ledger_skip_store_filters_and_resolves(pg_clean):
store = PostgreSQLLedgerSkipStore(pg_clean)
slot_id = uuid4()
record, _ = store.record_skip(
instance_key="inst-1",
slot_id=slot_id,
portfolio_id="PF-2",
symbol="005930",
order_id="O-2",
execution_id="E-2",
side="sell",
qty=5.0,
price=2000.0,
executed_at=datetime(2026, 8, 24, 2, 0, tzinfo=UTC),
order_venue="KRX",
execution_venue="NXTX",
skip_reason="unsupported_execution_venue",
)
rows = store.list_records(
instance_key="inst-1",
portfolio_id="PF-2",
on_date=date(2026, 8, 24),
status="unresolved",
)
assert [row.skip_key for row in rows] == [record.skip_key]
resolved = store.resolve_for_fill(
instance_key="inst-1",
slot_id=slot_id,
side="sell",
order_id="O-2",
execution_id="E-2",
resolution_reason="manual_ledger_backfill",
)
assert len(resolved) == 1
assert resolved[0].status == "resolved"
assert resolved[0].resolution_reason == "manual_ledger_backfill"
def test_ledger_skip_store_resolves_null_execution_id_on_backfill(pg_clean):
store = PostgreSQLLedgerSkipStore(pg_clean)
slot_id = uuid4()
record, _ = store.record_skip(
instance_key="inst-1",
slot_id=slot_id,
portfolio_id="PF-1",
symbol="005930",
order_id="O-3",
execution_id=None,
side="buy",
qty=1.0,
price=1010.0,
executed_at=datetime(2026, 8, 24, 3, 0, tzinfo=UTC),
order_venue="SOR",
execution_venue="unknown",
skip_reason="missing_execution_id",
)
resolved = store.resolve_for_fill(
instance_key="inst-1",
slot_id=slot_id,
side="buy",
order_id="O-3",
execution_id="E-3",
resolution_reason="manual_ledger_backfill",
)
assert [row.skip_key for row in resolved] == [record.skip_key]
assert resolved[0].execution_id == "E-3"
assert resolved[0].status == "resolved"
def test_ledger_skip_store_retries_unique_violation_during_concurrent_dedupe(pg_clean):
slot_id = uuid4()
start = threading.Barrier(2)
results = []
def worker():
store = PostgreSQLLedgerSkipStore(pg_clean)
start.wait()
results.append(
store.record_skip(
instance_key="inst-1",
slot_id=slot_id,
portfolio_id="PF-1",
symbol="005930",
order_id="O-4",
execution_id=None,
side="buy",
qty=1.0,
price=1111.0,
executed_at=datetime(2026, 8, 24, 4, 0, tzinfo=UTC),
order_venue="SOR",
execution_venue="unknown",
skip_reason="missing_execution_id",
)
)
t1 = threading.Thread(target=worker)
t2 = threading.Thread(target=worker)
t1.start()
t2.start()
t1.join()
t2.join()
assert len(results) == 2
assert len({record.skip_key for record, _notify in results}) == 1
assert sorted(notify for _record, notify in results) == [False, True]
listed = PostgreSQLLedgerSkipStore(pg_clean).list_records(instance_key="inst-1", status="unresolved")
assert len(listed) == 1
@@ -26,6 +26,8 @@ _EXPECTED_TABLES = {
"ledger_outbox", # BAT-006 / REQ-019
"snapshot_pause", # BAT-008 / REQ-032
"broker_account_snapshot", # dcy-041 / GitHub #45
"ledger_skip", # GitHub #68 skipped-ledger durable audit
"reconcile_notification_state", # req-0007 / ac-0024
}
@@ -0,0 +1,213 @@
"""PostgreSQLReconcileNotificationStore integration tests (req-0007, ac-0024).
Verifies migration 026 schema creation (idempotent replay), full CRUD
round-trip, and lifecycle semantics against a real PostgreSQL 16 instance.
Gated on TREX_POSTGRESQL_TEST_DSN — skipped in unit-test-only CI.
"""
from __future__ import annotations
import pytest
from trex_core.application.reconciliation import _c_identity_key
@pytest.fixture
def pg_notify_store(pg_clean):
from trex_core.adapters.portfolio.reconcile_notification_store import (
PostgreSQLReconcileNotificationStore,
)
return PostgreSQLReconcileNotificationStore(pg_clean)
# ---------------------------------------------------------------------------
# Migration 026 schema
# ---------------------------------------------------------------------------
class TestMigration026:
def test_table_exists_after_migration(self, pg_clean):
"""Migration 026 creates reconcile_notification_state (IF NOT EXISTS)."""
import psycopg
with psycopg.connect(pg_clean) as conn, conn.cursor() as cur:
cur.execute(
"SELECT table_name FROM information_schema.tables "
"WHERE table_schema = 'public' AND table_name = 'reconcile_notification_state'"
)
assert cur.fetchone() is not None, "reconcile_notification_state table not found"
def test_table_has_expected_columns(self, pg_clean):
"""Migration 026 creates all required columns."""
import psycopg
with psycopg.connect(pg_clean) as conn, conn.cursor() as cur:
cur.execute(
"SELECT column_name FROM information_schema.columns "
"WHERE table_schema = 'public' AND table_name = 'reconcile_notification_state'"
)
cols = {r[0] for r in cur.fetchall()}
expected = {
"identity_key", "case_code", "instance_key", "symbol",
"quantity_a", "quantity_b", "lifecycle_state", "created_at", "updated_at",
}
assert expected <= cols, f"Missing columns: {expected - cols}"
def test_migration_is_idempotent(self, pg_clean):
"""Running migration 026 twice does not fail (IF NOT EXISTS guard)."""
from trex_core.adapters.persistence.migrate import apply_migrations
# pg_clean already ran apply_migrations once; run again to verify idempotency.
apply_migrations(pg_clean) # must not raise
# ---------------------------------------------------------------------------
# CRUD round-trip
# ---------------------------------------------------------------------------
class TestCRUDRoundTrip:
def test_set_and_get_state_b_case(self, pg_notify_store):
"""set_state → get_state round-trip for B-case (slot_id identity)."""
pg_notify_store.set_state(
"slot-uuid-001",
case_code="B",
instance_key="inst-A",
symbol="005930",
quantity_a=10,
quantity_b=7,
lifecycle_state="notified",
)
state = pg_notify_store.get_state("slot-uuid-001")
assert state is not None
assert state["case_code"] == "B"
assert state["instance_key"] == "inst-A"
assert state["symbol"] == "005930"
assert state["quantity_a"] == 10
assert state["quantity_b"] == 7
assert state["lifecycle_state"] == "notified"
def test_set_and_get_state_c_case(self, pg_notify_store):
"""set_state → get_state round-trip for C-case (SHA-256 digest identity)."""
c_key = _c_identity_key("inst-A", "005930")
pg_notify_store.set_state(
c_key,
case_code="C",
instance_key="inst-A",
symbol="005930",
quantity_a=50,
quantity_b=42,
lifecycle_state="notified",
)
state = pg_notify_store.get_state(c_key)
assert state is not None
assert state["case_code"] == "C"
assert state["symbol"] == "005930"
assert state["quantity_a"] == 50
def test_set_state_upsert(self, pg_notify_store):
"""set_state is an upsert — second call with same key updates the row."""
pg_notify_store.set_state(
"slot-uuid-002",
case_code="B",
instance_key="inst-A",
symbol="005930",
quantity_a=10,
quantity_b=7,
lifecycle_state="notified",
)
pg_notify_store.set_state(
"slot-uuid-002",
case_code="B",
instance_key="inst-A",
symbol="005930",
quantity_a=10,
quantity_b=9, # quantity changed
lifecycle_state="notified",
)
state = pg_notify_store.get_state("slot-uuid-002")
assert state["quantity_b"] == 9
def test_get_state_returns_none_for_missing_key(self, pg_notify_store):
assert pg_notify_store.get_state("does-not-exist") is None
def test_clear_state_removes_row(self, pg_notify_store):
"""clear_state deletes the row; subsequent get_state returns None."""
pg_notify_store.set_state(
"slot-uuid-003",
case_code="B",
instance_key="inst-A",
symbol="005930",
quantity_a=10,
quantity_b=7,
lifecycle_state="notified",
)
pg_notify_store.clear_state("slot-uuid-003")
assert pg_notify_store.get_state("slot-uuid-003") is None
def test_list_states_by_lifecycle(self, pg_notify_store):
"""list_states_by_lifecycle returns only rows with the given state."""
pg_notify_store.set_state(
"slot-A",
case_code="B", instance_key="inst-A", symbol="005930",
quantity_a=10, quantity_b=7, lifecycle_state="notified",
)
pg_notify_store.set_state(
"slot-B",
case_code="B", instance_key="inst-A", symbol="005930",
quantity_a=20, quantity_b=15, lifecycle_state="pending_resolved",
)
notified = pg_notify_store.list_states_by_lifecycle("notified")
pending = pg_notify_store.list_states_by_lifecycle("pending_resolved")
assert len(notified) == 1
assert notified[0]["identity_key"] == "slot-A"
assert len(pending) == 1
assert pending[0]["identity_key"] == "slot-B"
def test_durability_across_store_instances(self, pg_clean):
"""State survives a simulated process restart (new store instance, same DSN)."""
from trex_core.adapters.portfolio.reconcile_notification_store import (
PostgreSQLReconcileNotificationStore,
)
a = PostgreSQLReconcileNotificationStore(pg_clean)
a.set_state(
"slot-restart",
case_code="B", instance_key="inst-A", symbol="005930",
quantity_a=10, quantity_b=7, lifecycle_state="notified",
)
# "restart" — fresh store instance
b = PostgreSQLReconcileNotificationStore(pg_clean)
state = b.get_state("slot-restart")
assert state is not None
assert state["lifecycle_state"] == "notified"
def test_c_identity_keys_for_distinct_inputs_are_distinct(self, pg_notify_store):
"""Two distinct (instance_key, symbol) pairs must yield distinct DB rows."""
key_a = _c_identity_key("inst-A", "005930")
key_b = _c_identity_key("inst-A", "005931")
assert key_a != key_b
pg_notify_store.set_state(
key_a, case_code="C", instance_key="inst-A", symbol="005930",
quantity_a=10, quantity_b=8, lifecycle_state="notified",
)
pg_notify_store.set_state(
key_b, case_code="C", instance_key="inst-A", symbol="005931",
quantity_a=20, quantity_b=15, lifecycle_state="notified",
)
state_a = pg_notify_store.get_state(key_a)
state_b = pg_notify_store.get_state(key_b)
assert state_a["symbol"] == "005930"
assert state_b["symbol"] == "005931"
def test_crafted_separator_collision_absent(self, pg_notify_store):
"""Crafted-separator inputs ('a\\x1fC','b') vs ('a','C\\x1fb') map to distinct rows."""
key_a = _c_identity_key("a\x1fC", "b")
key_b = _c_identity_key("a", "C\x1fb")
assert key_a != key_b
pg_notify_store.set_state(
key_a, case_code="C", instance_key="a\x1fC", symbol="b",
quantity_a=1, quantity_b=2, lifecycle_state="notified",
)
pg_notify_store.set_state(
key_b, case_code="C", instance_key="a", symbol="C\x1fb",
quantity_a=3, quantity_b=4, lifecycle_state="notified",
)
state_a = pg_notify_store.get_state(key_a)
state_b = pg_notify_store.get_state(key_b)
assert state_a["instance_key"] == "a\x1fC"
assert state_b["instance_key"] == "a"
+38 -2
View File
@@ -1,7 +1,7 @@
"""Bounded account-read recovery (dcy-041 / GitHub #45)."""
from __future__ import annotations
from datetime import datetime, timedelta, timezone
from datetime import UTC, datetime, timedelta
import pytest
@@ -36,8 +36,13 @@ class FakeAccountBroker:
return self.cash
class BrokenLoadStore(InMemoryAccountSnapshotStore):
def load(self, account_hash: str) -> AccountSnapshot | None:
raise RuntimeError("db read failed")
def _wire():
now = [datetime(2026, 7, 29, tzinfo=timezone.utc)]
now = [datetime(2026, 7, 29, tzinfo=UTC)]
broker = FakeAccountBroker()
store = InMemoryAccountSnapshotStore()
service = AccountSnapshotService(
@@ -162,3 +167,34 @@ def test_invalid_or_missing_buy_notional_fails_closed():
assert service.buy_guard(qty=1, limit_price=None) == (
"account_snapshot_price_required"
)
def test_slots_rpc_uses_snapshot_only_with_sell_ttl_freshness():
broker, _, service, now = _wire()
assert service.refresh()
broker.balance = [BalanceItem("005930", 999)]
holdings, captured_at = service.holdings_for_slots_rpc()
assert holdings == [BalanceItem("005930", 2)]
assert captured_at == now[0]
now[0] += timedelta(seconds=3601)
with pytest.raises(AccountSnapshotUnavailable) as raised:
service.holdings_for_slots_rpc()
assert raised.value.code == "expired"
def test_slots_rpc_normalizes_snapshot_store_read_failure():
broker = FakeAccountBroker()
service = AccountSnapshotService(
broker=broker,
store=BrokenLoadStore(),
buy_max_age_sec=600,
sell_max_age_sec=3600,
clock=lambda: datetime(2026, 7, 29, tzinfo=UTC),
rng=lambda: 0.5,
)
with pytest.raises(AccountSnapshotUnavailable) as raised:
service.holdings_for_slots_rpc()
assert raised.value.code == "read_failed"
+38
View File
@@ -15,6 +15,7 @@ import pytest
from trex_core.adapters.messaging.in_memory import InMemoryTransport
from trex_core.application.alarm_emitter import AlarmEmitter
from trex_core.application.messaging import TransportUnavailableError
# ------------------------------------------------------------------ helpers
@@ -154,3 +155,40 @@ def test_alarm_emitter_no_telegram_reference():
source = inspect.getsource(ae_mod)
assert "TELEGRAM" not in source, "alarm_emitter must not reference TELEGRAM_*"
assert "telegram" not in source.lower(), "alarm_emitter must not reference telegram"
def test_publish_alarm_transport_unavailable_log_is_sanitized(
caplog: pytest.LogCaptureFixture,
):
class UnavailableTransport:
def publish_alarm_event(self, routing_key, payload):
raise TransportUnavailableError("secret-token-value")
def health(self):
from trex_core.application.messaging import TransportHealth
return TransportHealth.DISCONNECTED
emitter = AlarmEmitter(UnavailableTransport()) # type: ignore[arg-type]
with caplog.at_level(logging.ERROR, logger="trex_core.application.alarm_emitter"):
emitter.publish_alarm("strategy_intent_rejected", "alarm title")
assert any("transport unavailable" in r.message for r in caplog.records)
assert "secret-token-value" not in caplog.text
def test_publish_alarm_generic_exception_log_is_sanitized(caplog: pytest.LogCaptureFixture):
class BrokenTransport:
def publish_alarm_event(self, routing_key, payload):
raise RuntimeError("******")
def health(self):
from trex_core.application.messaging import TransportHealth
return TransportHealth.CONNECTED
emitter = AlarmEmitter(BrokenTransport()) # type: ignore[arg-type]
with caplog.at_level(logging.ERROR, logger="trex_core.application.alarm_emitter"):
emitter.publish_alarm("strategy_intent_rejected", "alarm title")
assert any("alarm publish failed" in r.message for r in caplog.records)
assert "RuntimeError" in caplog.text
assert "******" not in caplog.text
+328 -26
View File
@@ -6,9 +6,15 @@ critical path while the ledger write is eventually-consistent.
"""
from __future__ import annotations
from datetime import UTC, datetime
from trex_core.adapters.broker.kis import KISBroker
from trex_core.adapters.broker.paper import PaperBroker
from trex_core.adapters.ledger.in_memory import InMemoryLedger
from trex_core.adapters.messaging.in_memory import InMemoryTransport
from trex_core.adapters.persistence.in_memory_intent_store import InMemoryIntentStore
from trex_core.application.alarm_emitter import AlarmEmitter
from trex_core.application.broker_port import FillStatus
from trex_core.application.execution import ExecutionService
from trex_core.application.intents import BuyIntent, IntentService, SellIntent
from trex_core.application.ledger_outbox import (
@@ -17,21 +23,118 @@ from trex_core.application.ledger_outbox import (
OutboxOutcome,
WorkItemStatus,
)
from trex_core.application.ledger_skip import LedgerSkipRecord
from trex_core.domain.slot import SlotService, SlotState
from trex_core.domain.slot.in_memory_repository import InMemorySlotRepository
class _SkipStore:
def __init__(self):
self.records = []
self._by_key = {}
def _wire(broker, *, ledger=None, clock=None):
def record_skip(self, **kwargs):
record = LedgerSkipRecord(
skip_key=f"{kwargs['slot_id']}:{kwargs['side']}:{kwargs['skip_reason']}",
instance_key=kwargs["instance_key"],
slot_id=kwargs["slot_id"],
portfolio_id=kwargs["portfolio_id"],
symbol=kwargs["symbol"],
order_id=kwargs["order_id"],
execution_id=kwargs["execution_id"],
side=kwargs["side"],
qty=kwargs["qty"],
price=kwargs["price"],
executed_at=kwargs["executed_at"],
order_venue=kwargs["order_venue"],
execution_venue=kwargs["execution_venue"],
skip_reason=kwargs["skip_reason"],
status="unresolved",
resolution_reason=None,
resolved_at=None,
created_at=kwargs["executed_at"],
updated_at=kwargs["executed_at"],
)
should_notify = record.skip_key not in self._by_key
self._by_key[record.skip_key] = record
self.records = list(self._by_key.values())
return record, should_notify
def resolve_for_fill(self, **_kwargs):
return []
def resolve_for_delivery(
self,
*,
instance_key,
portfolio_id,
side,
order_id,
execution_id,
resolution_reason,
):
resolved = []
updated = {}
for key, record in self._by_key.items():
if (
record.instance_key == instance_key
and record.portfolio_id == portfolio_id
and record.side == side
and (
record.execution_id == execution_id
or (record.execution_id is None and record.order_id == order_id)
)
):
record = LedgerSkipRecord(
skip_key=record.skip_key,
instance_key=record.instance_key,
slot_id=record.slot_id,
portfolio_id=record.portfolio_id,
symbol=record.symbol,
order_id=record.order_id,
execution_id=record.execution_id or execution_id,
side=record.side,
qty=record.qty,
price=record.price,
executed_at=record.executed_at,
order_venue=record.order_venue,
execution_venue=record.execution_venue,
skip_reason=record.skip_reason,
status="resolved",
resolution_reason=resolution_reason,
resolved_at=record.executed_at,
created_at=record.created_at,
updated_at=record.executed_at,
)
resolved.append(record)
updated[key] = record
self._by_key = updated
self.records = list(self._by_key.values())
return resolved
def _wire(broker, *, ledger=None, clock=None, alarm_emitter=None, ledger_skip_store=None):
repo = InMemorySlotRepository()
svc = SlotService(repo)
intents = IntentService(svc, repo, InMemoryIntentStore(),
resolve_symbol=lambda pid: "005930")
outbox = InMemoryLedgerOutbox(clock=clock) if clock else InMemoryLedgerOutbox()
execution = ExecutionService(svc, repo, broker, ledger_outbox=outbox)
writer = (
LedgerWriter(outbox, ledger or InMemoryLedger(), clock=clock)
if clock else LedgerWriter(outbox, ledger or InMemoryLedger())
execution = ExecutionService(
svc,
repo,
broker,
ledger_outbox=outbox,
alarm_emitter=alarm_emitter,
ledger_skip_store=ledger_skip_store,
)
writer = LedgerWriter(
outbox,
ledger or InMemoryLedger(),
clock=clock,
alarm_emitter=alarm_emitter,
ledger_skip_store=ledger_skip_store,
instance_key="default",
)
repo.reconcile_universe("default", {"PF-1": "SYM"})
return repo, svc, intents, execution, outbox, writer
@@ -44,10 +147,42 @@ def _buy(intents, *, iid="b1", portfolio_id="PF-1"):
))
class KisSorUnknownVenueBroker(PaperBroker):
"""KIS daily-ccld fixture with SOR routing but no actual exchange evidence."""
def poll_fill(self, broker_order_id, *, side, symbol, **kwargs):
return KISBroker._fill_status_from_row(
{
"ord_qty": "10",
"tot_ccld_qty": "10",
"avg_prvs": "1000",
"ord_dt": "20260824",
"ord_gno_brno": "12345678",
"odno": broker_order_id,
"excg_id_dvsn_cd": "SOR",
},
side=side,
today="20260824",
fallback_odno=broker_order_id,
)
class MissingExecutionIdBroker(PaperBroker):
def poll_fill(self, broker_order_id, *, side, symbol, **kwargs):
return FillStatus(
state="filled",
filled_qty=10,
avg_price=1000,
execution_venue="KRX",
)
def test_buy_fill_enqueues_one_ledger_item_and_writer_records_it():
ledger = InMemoryLedger()
repo, svc, intents, execution, outbox, writer = _wire(PaperBroker(fill_after_polls=0),
ledger=ledger)
repo, _, intents, execution, _, writer = _wire(
PaperBroker(fill_after_polls=0),
ledger=ledger,
)
sid = _buy(intents).slot_id
r = execution.run_once()
assert r.filled == 1 and r.ledger_enqueued == 1
@@ -63,10 +198,42 @@ def test_buy_fill_enqueues_one_ledger_item_and_writer_records_it():
assert t.venue == "KRX" # paper's deterministic SOR execution exchange
def test_kis_sor_fill_without_actual_venue_enqueues_and_writes_once() -> None:
"""Issues #65/#66: SOR provenance must not suppress an unknown actual venue fill."""
ledger = InMemoryLedger()
repo, _, intents, execution, outbox, writer = _wire(KisSorUnknownVenueBroker(), ledger=ledger)
sid = _buy(intents, iid="sor-unknown").slot_id
result = execution.run_once()
assert result.filled == 1
assert result.ledger_enqueued == 1
assert result.ledger_skipped == 0
assert repo.fetch_slot(sid).status is SlotState.HOLDING
[item] = outbox.all_items()
assert item.trade.venue is None
assert item.trade.order_venue == "SOR"
assert writer.run_once().completed == 1
assert ledger.count() == 1
[trade] = ledger.trades()
assert trade.execution_id == item.trade.execution_id
assert trade.venue is None
assert trade.order_venue == "SOR"
# Replay remains idempotent at the durable outbox and portfolio ledger boundaries.
outbox.enqueue(item.trade)
assert len(outbox.all_items()) == 1
assert writer.run_once().completed == 0
assert ledger.count() == 1
def test_sell_fill_enqueues_sell_ledger_item():
ledger = InMemoryLedger()
repo, svc, intents, execution, outbox, writer = _wire(PaperBroker(fill_after_polls=0),
ledger=ledger)
repo, _, intents, execution, _, writer = _wire(
PaperBroker(fill_after_polls=0),
ledger=ledger,
)
sid = _buy(intents).slot_id
execution.run_once() # -> holding + buy ledger item
writer.run_once() # drain buy
@@ -81,13 +248,16 @@ def test_sell_fill_enqueues_sell_ledger_item():
def test_portfolio_id_absent_skips_ledger_but_transitions_slot():
ledger = InMemoryLedger()
repo, svc, intents, execution, outbox, writer = _wire(PaperBroker(fill_after_polls=0),
ledger=ledger)
repo, svc, _, execution, outbox, _ = _wire(
PaperBroker(fill_after_polls=0),
ledger=ledger,
)
# dcy-011: buy admission now requires portfolio_id, so create the
# pending_buy slot directly (bypass admission) to exercise the
# execution-loop ledger skip + visibility (REQ-018 FR-4).
from trex_core.domain.slot.models import Slot
from uuid import uuid4
from trex_core.domain.slot.models import Slot
anon = Slot(slot_id=uuid4(), status=SlotState.INACTIVE)
repo.add_slot(anon)
svc.transit_slot(anon.slot_id, SlotState.INACTIVE, SlotState.PENDING_BUY,
@@ -100,14 +270,138 @@ def test_portfolio_id_absent_skips_ledger_but_transitions_slot():
assert ledger.count() == 0
def test_missing_execution_id_still_skips_malformed_ledger_work_item() -> None:
repo, _, intents, execution, outbox, _ = _wire(MissingExecutionIdBroker())
sid = _buy(intents, iid="missing-execution-id").slot_id
assert sid is not None
result = execution.run_once()
assert result.filled == 1
assert result.ledger_enqueued == 0
assert result.ledger_skipped == 1
assert repo.fetch_slot(sid).status is SlotState.HOLDING
assert outbox.all_items() == []
def test_missing_portfolio_id_persists_skip_and_emits_alert() -> None:
transport = InMemoryTransport()
skip_store = _SkipStore()
repo, svc, _, execution, outbox, _ = _wire(
PaperBroker(fill_after_polls=0),
alarm_emitter=AlarmEmitter(transport),
ledger_skip_store=skip_store,
)
from uuid import uuid4
from trex_core.domain.slot.models import Slot
anon = Slot(slot_id=uuid4(), status=SlotState.INACTIVE)
repo.add_slot(anon)
svc.transit_slot(
anon.slot_id,
SlotState.INACTIVE,
SlotState.PENDING_BUY,
reason="buy",
actor="test",
symbol="005930",
portfolio_id=None,
)
result = execution.run_once()
assert result.ledger_skipped == 1
assert outbox.all_items() == []
assert len(skip_store.records) == 1
record = skip_store.records[0]
assert record.skip_reason == "missing_portfolio_id"
assert record.order_venue == "KRX"
alarms = transport.alarm_events()
assert len(alarms) == 1
_, payload = alarms[0]
assert payload["code"] == "ledger_skip_unresolved"
assert payload["fields"]["skip_reason"] == "missing_portfolio_id"
def test_missing_execution_id_persists_skip_record() -> None:
transport = InMemoryTransport()
skip_store = _SkipStore()
repo, _, intents, execution, outbox, _ = _wire(
MissingExecutionIdBroker(),
alarm_emitter=AlarmEmitter(transport),
ledger_skip_store=skip_store,
)
sid = _buy(intents, iid="missing-execution-id").slot_id
assert sid is not None
result = execution.run_once()
assert result.ledger_skipped == 1
assert result.ledger_enqueued == 0
assert repo.fetch_slot(sid).status is SlotState.HOLDING
assert outbox.all_items() == []
assert len(skip_store.records) == 1
record = skip_store.records[0]
assert record.skip_reason == "missing_execution_id"
assert record.portfolio_id == "PF-1"
assert record.execution_id is None
def test_skip_resolves_only_after_writer_completes_delivery() -> None:
transport = InMemoryTransport()
skip_store = _SkipStore()
ledger = InMemoryLedger()
_repo, _, intents, execution, outbox, writer = _wire(
PaperBroker(fill_after_polls=0),
ledger=ledger,
alarm_emitter=AlarmEmitter(transport),
ledger_skip_store=skip_store,
)
sid = _buy(intents, iid="delayed-skip-resolution").slot_id
record, _ = skip_store.record_skip(
instance_key="default",
slot_id=sid,
portfolio_id="PF-1",
symbol="005930",
order_id="PAPER-000001",
execution_id=None,
side="buy",
qty=10.0,
price=1000.0,
executed_at=datetime(2026, 8, 24, 0, 0, tzinfo=UTC),
order_venue="SOR",
execution_venue="unknown",
skip_reason="missing_execution_id",
)
result = execution.run_once()
assert result.ledger_enqueued == 1
assert outbox.all_items()
assert skip_store._by_key[record.skip_key].status == "unresolved"
assert transport.alarm_events() == []
stats = writer.run_once()
assert stats.completed == 1
resolved = skip_store._by_key[record.skip_key]
assert resolved.status == "resolved"
assert resolved.execution_id is not None
assert resolved.resolution_reason == "ledger_recorded"
assert [payload["code"] for _, payload in transport.alarm_events()] == ["ledger_skip_resolved"]
def test_ledger_skip_emits_warning(caplog):
"""dcy-011 / REQ-018 FR-4 (S-3): ledger skip is logged, not silent."""
import logging
ledger = InMemoryLedger()
repo, svc, intents, execution, outbox, writer = _wire(PaperBroker(fill_after_polls=0),
ledger=ledger)
from trex_core.domain.slot.models import Slot
repo, svc, _, execution, _, _ = _wire(
PaperBroker(fill_after_polls=0),
ledger=ledger,
)
from uuid import uuid4
from trex_core.domain.slot.models import Slot
anon = Slot(slot_id=uuid4(), status=SlotState.INACTIVE)
repo.add_slot(anon)
svc.transit_slot(anon.slot_id, SlotState.INACTIVE, SlotState.PENDING_BUY,
@@ -121,8 +415,10 @@ def test_ledger_skip_emits_warning(caplog):
def test_duplicate_execution_id_results_in_zero_duplicate_in_ledger():
"""REQ-019 AC-1: same execution_id processed twice = 1 ledger entry."""
ledger = InMemoryLedger()
repo, svc, intents, execution, outbox, writer = _wire(PaperBroker(fill_after_polls=0),
ledger=ledger)
_, _, intents, execution, outbox, writer = _wire(
PaperBroker(fill_after_polls=0),
ledger=ledger,
)
_buy(intents)
execution.run_once()
writer.run_once()
@@ -148,10 +444,10 @@ def test_retryable_failure_then_eventually_completes():
dcy-015: retryable rows back off, so advance a shared clock past each
next_retry_at between writer passes.
"""
from datetime import datetime, timedelta, timezone
now = {"t": datetime(2026, 6, 4, tzinfo=timezone.utc)}
from datetime import datetime, timedelta
now = {"t": datetime(2026, 6, 4, tzinfo=UTC)}
ledger = InMemoryLedger(fail_n_then_succeed=2)
repo, svc, intents, execution, outbox, writer = _wire(
repo, _, intents, execution, outbox, writer = _wire(
PaperBroker(fill_after_polls=0), ledger=ledger, clock=lambda: now["t"],
)
sid = _buy(intents).slot_id
@@ -177,8 +473,10 @@ def test_conflict_blocks_without_auto_retry():
# order id is PAPER-000001 / qty=10 => execution_id = paper:PAPER-000001:10.
conflict_id = "paper:PAPER-000001:10"
ledger = InMemoryLedger(conflict_executions={conflict_id})
repo, svc, intents, execution, outbox, writer = _wire(PaperBroker(fill_after_polls=0),
ledger=ledger)
_, _, intents, execution, outbox, writer = _wire(
PaperBroker(fill_after_polls=0),
ledger=ledger,
)
_buy(intents)
execution.run_once()
stats = writer.run_once()
@@ -195,8 +493,10 @@ def test_conflict_blocks_without_auto_retry():
def test_duplicate_completed_when_reprocessing_terminal_item():
"""REQ-019 AC-4: a completed item reports duplicate_completed if re-listed."""
ledger = InMemoryLedger()
repo, svc, intents, execution, outbox, writer = _wire(PaperBroker(fill_after_polls=0),
ledger=ledger)
_, _, intents, execution, outbox, writer = _wire(
PaperBroker(fill_after_polls=0),
ledger=ledger,
)
_buy(intents)
execution.run_once()
writer.run_once()
@@ -214,8 +514,10 @@ def test_duplicate_completed_when_reprocessing_terminal_item():
def test_slot_transition_proceeds_when_ledger_is_down():
"""REQ-019 AC-5: portfolio-hub outage does not block the fill transition."""
ledger = InMemoryLedger(fail_n_then_succeed=100)
repo, svc, intents, execution, outbox, writer = _wire(PaperBroker(fill_after_polls=0),
ledger=ledger)
repo, _, intents, execution, outbox, writer = _wire(
PaperBroker(fill_after_polls=0),
ledger=ledger,
)
sid = _buy(intents).slot_id
execution.run_once()
assert repo.fetch_slot(sid).status is SlotState.HOLDING
+18 -3
View File
@@ -1,7 +1,7 @@
"""LedgerOutbox + LedgerWriter direct unit tests (REQ-019)."""
from __future__ import annotations
from datetime import datetime, timedelta, timezone
from datetime import UTC, datetime, timedelta
from trex_core.adapters.ledger.in_memory import InMemoryLedger
from trex_core.application.ledger_outbox import (
@@ -17,7 +17,7 @@ def _trade(execution_id="paper:PAPER-1:10") -> LedgerTrade:
return LedgerTrade(
asset_id="PF-1", order_id="PAPER-1", execution_id=execution_id,
side="buy", qty=10, price=1000.0,
executed_at=datetime(2026, 6, 4, tzinfo=timezone.utc),
executed_at=datetime(2026, 6, 4, tzinfo=UTC),
)
@@ -57,7 +57,7 @@ def test_writer_marks_conflict_blocked_and_does_not_retry():
def test_writer_retries_until_success():
# dcy-015: retryable rows now back off; share an injected clock between the
# outbox (list_pending "now") and the writer (next_retry_at) to advance past it.
now = {"t": datetime(2026, 6, 4, tzinfo=timezone.utc)}
now = {"t": datetime(2026, 6, 4, tzinfo=UTC)}
outbox = InMemoryLedgerOutbox(clock=lambda: now["t"])
ledger = InMemoryLedger(fail_n_then_succeed=1)
writer = LedgerWriter(outbox, ledger, clock=lambda: now["t"])
@@ -68,3 +68,18 @@ def test_writer_retries_until_success():
now["t"] += timedelta(seconds=600) # past the backoff window
s2 = writer.run_once()
assert s2.completed == 1 and ledger.count() == 1
def test_process_correlation_id_only_drains_requested_row():
outbox = InMemoryLedgerOutbox()
ledger = InMemoryLedger()
writer = LedgerWriter(outbox, ledger)
first = outbox.enqueue(_trade(execution_id="e-1"))
second = outbox.enqueue(_trade(execution_id="e-2"))
outcome = writer.process_correlation_id(first.correlation_id)
assert outcome is OutboxOutcome.COMPLETED
assert ledger.count() == 1
assert outbox.get(first.correlation_id).status is WorkItemStatus.COMPLETED
assert outbox.get(second.correlation_id).status is WorkItemStatus.PENDING
@@ -0,0 +1,932 @@
"""Reconciliation mismatch notification lifecycle tests (req-0007, ac-0024).
Verifies B-case and C-case detected/changed/resolved alarm publication with
durable deduplication via ReconcileNotificationStore, including:
- first detection
- unchanged suppression (within and across simulated restarts)
- quantity change (changed event)
- multi-portfolio C-case aggregation (one event per symbol)
- B-case slot identity isolation
- B sync resolution
- C verify resolution
- publish failure: fingerprint not advanced, retry next detection
- resolved publish failure: warning clear preserved, pending_resolved set
- notification-state persistence failure: business path not aborted
- duplicate resolved: idempotent
- sensitive-field absence
"""
from __future__ import annotations
from unittest.mock import MagicMock
from uuid import uuid4
from trex_core.adapters.messaging.in_memory import InMemoryTransport
from trex_core.application.alarm_emitter import AlarmEmitter
from trex_core.application.broker_port import BalanceItem
from trex_core.application.portfolio import (
InMemoryReconcileNotificationStore,
InMemoryReconcileWarningStore,
PortfolioItem,
SnapshotStore,
)
from trex_core.application.reconciliation import (
_c_identity_key,
reconcile_core_holdings,
reconcile_positions,
sync_slot_to_broker,
verify_slot_reconcile,
)
from trex_core.domain.slot import SlotService, SlotState
from trex_core.domain.slot.in_memory_repository import InMemorySlotRepository
from trex_core.domain.slot.models import Slot
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
INSTANCE = "test-instance"
def _make_emitter() -> tuple[AlarmEmitter, InMemoryTransport]:
transport = InMemoryTransport()
return AlarmEmitter(transport), transport
def _make_failing_emitter() -> AlarmEmitter:
"""AlarmEmitter backed by a transport that always raises."""
transport = InMemoryTransport()
transport.set_available(False)
return AlarmEmitter(transport)
def _alarm_codes(transport: InMemoryTransport) -> list[str]:
return [payload["code"] for _, payload in transport.alarm_events()]
def _alarm_fields_for(transport: InMemoryTransport, code: str) -> list[dict]:
return [
payload["fields"]
for _, payload in transport.alarm_events()
if payload["code"] == code
]
def _make_slot(
status: SlotState = SlotState.HOLDING,
qty: float = 10.0,
symbol: str = "005930",
pid: str = "pf-A",
) -> Slot:
return Slot(
slot_id=uuid4(),
status=status,
symbol=symbol,
qty=qty,
portfolio_id=pid,
instance_key=INSTANCE,
)
def _snapshot(portfolio_id: str, asset_qty: float, symbol: str = "005930") -> SnapshotStore:
store = SnapshotStore()
store.swap([PortfolioItem(
portfolio_id=portfolio_id,
symbol=symbol,
cash_balance=0.0,
asset_quantity=asset_qty,
)])
return store
class FakeBroker:
def __init__(self, balances: dict[str, int]):
self._b = balances
def name(self) -> str:
return "fake"
def submit_order(self, req): ...
def poll_fill(self, *args, **kw): ...
def cancel_order(self, *args, **kw): ...
def get_current_price(self, sym): return None
def get_balance(self) -> list[BalanceItem]:
return [BalanceItem(symbol=sym, hldg_qty=qty)
for sym, qty in self._b.items() if qty > 0]
def _repo_with_slot(slot: Slot) -> InMemorySlotRepository:
repo = InMemorySlotRepository()
repo.add_slot(slot)
return repo
def _run_reconcile_positions(
hub_qty: int,
broker_qty: int,
symbol: str = "005930",
pid: str = "pf-A",
emitter: AlarmEmitter | None = None,
notification_store=None,
warn_store=None,
slot_repo=None,
) -> dict:
store = _snapshot(pid, float(hub_qty), symbol=symbol)
broker = FakeBroker({symbol: broker_qty})
return reconcile_positions(
store=store,
broker=broker,
instance_key=INSTANCE,
alarm_emitter=emitter,
notification_store=notification_store,
warn_store=warn_store,
slot_repo=slot_repo,
)
# ---------------------------------------------------------------------------
# B-case: reconcile_core_holdings
# ---------------------------------------------------------------------------
def test_b_case_first_detection_publishes_detected():
"""New B-case slot mismatch → detected event published once."""
emitter, transport = _make_emitter()
ns = InMemoryReconcileNotificationStore()
slot = _make_slot(qty=10.0)
repo = _repo_with_slot(slot)
slot_service = SlotService(repo)
reconcile_core_holdings(
slot_repo=repo,
slot_service=slot_service,
broker=FakeBroker({"005930": 7}),
instance_key=INSTANCE,
alarm_emitter=emitter,
notification_store=ns,
)
codes = _alarm_codes(transport)
assert codes == ["reconcile_mismatch_detected"]
state = ns.get_state(str(slot.slot_id))
assert state is not None
assert state["lifecycle_state"] == "notified"
assert state["quantity_a"] == 10
assert state["quantity_b"] == 7
def test_b_case_unchanged_suppression():
"""Repeated tick with same qty tuple → no second event."""
emitter, transport = _make_emitter()
ns = InMemoryReconcileNotificationStore()
slot = _make_slot(qty=10.0)
repo = _repo_with_slot(slot)
slot_service = SlotService(repo)
for _ in range(3):
reconcile_core_holdings(
slot_repo=repo,
slot_service=slot_service,
broker=FakeBroker({"005930": 7}),
instance_key=INSTANCE,
alarm_emitter=emitter,
notification_store=ns,
)
assert len(transport.alarm_events()) == 1
def test_b_case_process_restart_suppression():
"""After restart (new store seeded with old fingerprint), unchanged tick is suppressed."""
emitter1, transport1 = _make_emitter()
ns1 = InMemoryReconcileNotificationStore()
slot = _make_slot(qty=10.0)
slot_id_str = str(slot.slot_id)
repo = _repo_with_slot(slot)
ss = SlotService(repo)
reconcile_core_holdings(
slot_repo=repo, slot_service=ss,
broker=FakeBroker({"005930": 7}),
instance_key=INSTANCE,
alarm_emitter=emitter1, notification_store=ns1,
)
assert len(transport1.alarm_events()) == 1
# Simulate restart: new emitter + new in-memory store, but seed it from ns1 state.
emitter2, transport2 = _make_emitter()
ns2 = InMemoryReconcileNotificationStore()
state = ns1.get_state(slot_id_str)
ns2.set_state(slot_id_str, **{k: v for k, v in state.items() if k != "identity_key"})
reconcile_core_holdings(
slot_repo=repo, slot_service=ss,
broker=FakeBroker({"005930": 7}),
instance_key=INSTANCE,
alarm_emitter=emitter2, notification_store=ns2,
)
assert len(transport2.alarm_events()) == 0 # suppressed
def test_b_case_quantity_change_publishes_changed():
"""B-case same slot, different qty tuple → changed event."""
emitter, transport = _make_emitter()
ns = InMemoryReconcileNotificationStore()
slot = _make_slot(qty=10.0)
repo = _repo_with_slot(slot)
ss = SlotService(repo)
reconcile_core_holdings(
slot_repo=repo, slot_service=ss,
broker=FakeBroker({"005930": 7}),
instance_key=INSTANCE,
alarm_emitter=emitter, notification_store=ns,
)
# Now broker qty changes → different fingerprint
# We need to update the slot qty in-place to simulate a changed qty scenario.
# Manually update notification state to simulate a different prior fingerprint.
ns.set_state(str(slot.slot_id), case_code="B", instance_key=INSTANCE,
symbol="005930", quantity_a=10, quantity_b=7, lifecycle_state="notified")
# New broker qty: 5 instead of 7
reconcile_core_holdings(
slot_repo=repo, slot_service=ss,
broker=FakeBroker({"005930": 5}),
instance_key=INSTANCE,
alarm_emitter=emitter, notification_store=ns,
)
codes = _alarm_codes(transport)
assert "reconcile_mismatch_changed" in codes
state = ns.get_state(str(slot.slot_id))
assert state["quantity_b"] == 5
def test_b_case_separate_slots_no_fingerprint_mixing():
"""Different B-case slot identities have independent fingerprints."""
emitter, transport = _make_emitter()
ns = InMemoryReconcileNotificationStore()
slot_a = _make_slot(qty=10.0, symbol="005930", pid="pf-A")
slot_b = _make_slot(qty=20.0, symbol="000660", pid="pf-B")
repo = InMemorySlotRepository()
repo.add_slot(slot_a)
repo.add_slot(slot_b)
ss = SlotService(repo)
reconcile_core_holdings(
slot_repo=repo, slot_service=ss,
broker=FakeBroker({"005930": 7, "000660": 15}),
instance_key=INSTANCE,
alarm_emitter=emitter, notification_store=ns,
)
# Both should get detected events
codes = _alarm_codes(transport)
assert codes.count("reconcile_mismatch_detected") == 2
state_a = ns.get_state(str(slot_a.slot_id))
state_b = ns.get_state(str(slot_b.slot_id))
assert state_a is not None
assert state_b is not None
# Fingerprints are independent
assert state_a["quantity_a"] == 10
assert state_b["quantity_a"] == 20
assert state_a["identity_key"] != state_b["identity_key"]
def test_b_case_publish_failure_fingerprint_not_advanced():
"""Transport failure → fingerprint not advanced → retry on next detection."""
failing_emitter = _make_failing_emitter()
ns = InMemoryReconcileNotificationStore()
slot = _make_slot(qty=10.0)
repo = _repo_with_slot(slot)
ss = SlotService(repo)
reconcile_core_holdings(
slot_repo=repo, slot_service=ss,
broker=FakeBroker({"005930": 7}),
instance_key=INSTANCE,
alarm_emitter=failing_emitter, notification_store=ns,
)
# Fingerprint must not be advanced
assert ns.get_state(str(slot.slot_id)) is None
# Next detection should try again (detected, not changed)
emitter2, transport2 = _make_emitter()
reconcile_core_holdings(
slot_repo=repo, slot_service=ss,
broker=FakeBroker({"005930": 7}),
instance_key=INSTANCE,
alarm_emitter=emitter2, notification_store=ns,
)
assert _alarm_codes(transport2) == ["reconcile_mismatch_detected"]
def test_b_case_no_notification_without_store():
"""Backward compat: no notification_store → no crash, no notification."""
slot = _make_slot(qty=10.0)
repo = _repo_with_slot(slot)
ss = SlotService(repo)
emitter, transport = _make_emitter()
decisions = reconcile_core_holdings(
slot_repo=repo, slot_service=ss,
broker=FakeBroker({"005930": 7}),
instance_key=INSTANCE,
alarm_emitter=emitter,
# no notification_store
)
assert str(slot.slot_id) in decisions
assert len(transport.alarm_events()) == 0
# ---------------------------------------------------------------------------
# B-case resolved: sync_slot_to_broker
# ---------------------------------------------------------------------------
def test_b_sync_resolution_publishes_resolved():
"""operator sync clears warning → resolved event published."""
emitter, transport = _make_emitter()
ns = InMemoryReconcileNotificationStore()
slot = _make_slot(qty=10.0)
slot_id_str = str(slot.slot_id)
repo = _repo_with_slot(slot)
ss = SlotService(repo)
ws = InMemoryReconcileWarningStore()
ws.set_warned(slot_id_str, portfolio_id="pf-A", case_code="B")
# Seed notification state (simulating prior detection)
ns.set_state(slot_id_str, case_code="B", instance_key=INSTANCE,
symbol="005930", quantity_a=10, quantity_b=7, lifecycle_state="notified")
sync_slot_to_broker(
slot_id=slot_id_str,
slot_repo=repo,
slot_service=ss,
broker=FakeBroker({"005930": 7}),
instance_key=INSTANCE,
warn_store=ws,
alarm_emitter=emitter,
notification_store=ns,
)
codes = _alarm_codes(transport)
assert "reconcile_mismatch_resolved" in codes
# State cleared after resolved
assert ns.get_state(slot_id_str) is None
def test_b_sync_resolved_publish_failure_warning_clear_preserved():
"""Resolved publish fails → warning clear is preserved, pending_resolved set."""
failing_emitter = _make_failing_emitter()
ns = InMemoryReconcileNotificationStore()
slot = _make_slot(qty=10.0)
slot_id_str = str(slot.slot_id)
repo = _repo_with_slot(slot)
ss = SlotService(repo)
ws = InMemoryReconcileWarningStore()
ws.set_warned(slot_id_str, portfolio_id="pf-A", case_code="B")
ns.set_state(slot_id_str, case_code="B", instance_key=INSTANCE,
symbol="005930", quantity_a=10, quantity_b=7, lifecycle_state="notified")
result = sync_slot_to_broker(
slot_id=slot_id_str,
slot_repo=repo,
slot_service=ss,
broker=FakeBroker({"005930": 7}),
instance_key=INSTANCE,
warn_store=ws,
alarm_emitter=failing_emitter,
notification_store=ns,
)
# Warning clear preserved
assert result["warning_cleared"] is True
assert not ws.is_warned(slot_id_str)
# pending_resolved set for retry
state = ns.get_state(slot_id_str)
assert state is not None
assert state["lifecycle_state"] == "pending_resolved"
def test_b_sync_resolved_retry_on_next_reconcile():
"""After pending_resolved set, next reconcile pass retries resolved publish."""
ns = InMemoryReconcileNotificationStore()
slot_id_str = "some-slot-id"
# Manually set pending_resolved state (warning already cleared)
ns.set_state(slot_id_str, case_code="B", instance_key=INSTANCE,
symbol="005930", quantity_a=10, quantity_b=7, lifecycle_state="pending_resolved")
emitter, transport = _make_emitter()
slot = _make_slot(qty=7.0) # qty matches broker now — not in active mismatch set
repo = _repo_with_slot(slot)
ss = SlotService(repo)
reconcile_core_holdings(
slot_repo=repo, slot_service=ss,
broker=FakeBroker({"005930": 7}),
instance_key=INSTANCE,
alarm_emitter=emitter, notification_store=ns,
)
codes = _alarm_codes(transport)
assert "reconcile_mismatch_resolved" in codes
assert ns.get_state(slot_id_str) is None
def test_b_sync_duplicate_resolved_idempotent():
"""No notification state → resolved is a no-op (idempotent)."""
emitter, transport = _make_emitter()
ns = InMemoryReconcileNotificationStore()
slot = _make_slot(qty=10.0)
slot_id_str = str(slot.slot_id)
repo = _repo_with_slot(slot)
ss = SlotService(repo)
ws = InMemoryReconcileWarningStore()
ws.set_warned(slot_id_str, portfolio_id="pf-A", case_code="B")
# No notification state seeded — simulate never-detected
sync_slot_to_broker(
slot_id=slot_id_str,
slot_repo=repo,
slot_service=ss,
broker=FakeBroker({"005930": 7}), # qty 10 != 7 so sync updates
instance_key=INSTANCE,
warn_store=ws,
alarm_emitter=emitter,
notification_store=ns,
)
# resolved alarm must not be published when no prior state
resolved = [c for c in _alarm_codes(transport) if c == "reconcile_mismatch_resolved"]
assert len(resolved) == 0
# ---------------------------------------------------------------------------
# C-case: reconcile_positions
# ---------------------------------------------------------------------------
def test_c_case_first_detection_publishes_detected():
"""New C-case symbol aggregate mismatch → detected event published once."""
emitter, transport = _make_emitter()
ns = InMemoryReconcileNotificationStore()
_run_reconcile_positions(
hub_qty=10, broker_qty=7,
emitter=emitter, notification_store=ns,
)
assert _alarm_codes(transport) == ["reconcile_mismatch_detected"]
c_key = _c_identity_key(INSTANCE, "005930")
state = ns.get_state(c_key)
assert state is not None
assert state["case_code"] == "C"
assert state["quantity_a"] == 10
assert state["quantity_b"] == 7
def test_c_case_multi_portfolio_single_event():
"""C-case multiple portfolios for same symbol → exactly one detected event."""
emitter, transport = _make_emitter()
ns = InMemoryReconcileNotificationStore()
# pf-A has 6, pf-B has 4 → hub total 10 ≠ broker 7
store = SnapshotStore()
store.swap([
PortfolioItem(portfolio_id="pf-A", symbol="005930", cash_balance=0, asset_quantity=6.0),
PortfolioItem(portfolio_id="pf-B", symbol="005930", cash_balance=0, asset_quantity=4.0),
])
broker = FakeBroker({"005930": 7})
reconcile_positions(
store=store,
broker=broker,
instance_key=INSTANCE,
alarm_emitter=emitter,
notification_store=ns,
)
assert len(transport.alarm_events()) == 1
assert _alarm_codes(transport) == ["reconcile_mismatch_detected"]
# Only one state entry for the symbol
c_key = _c_identity_key(INSTANCE, "005930")
state = ns.get_state(c_key)
assert state is not None
assert state["quantity_a"] == 10 # 6 + 4
def test_c_case_unchanged_suppression():
"""Repeated C-case ticks with same qty tuple → no second event."""
emitter, transport = _make_emitter()
ns = InMemoryReconcileNotificationStore()
for _ in range(3):
_run_reconcile_positions(hub_qty=10, broker_qty=7, emitter=emitter, notification_store=ns)
assert len(transport.alarm_events()) == 1
def test_c_case_quantity_change_publishes_changed():
"""C-case qty tuple changes → changed event."""
emitter, transport = _make_emitter()
ns = InMemoryReconcileNotificationStore()
_run_reconcile_positions(hub_qty=10, broker_qty=7, emitter=emitter, notification_store=ns)
assert _alarm_codes(transport) == ["reconcile_mismatch_detected"]
# Broker qty changes
_run_reconcile_positions(hub_qty=10, broker_qty=5, emitter=emitter, notification_store=ns)
codes = _alarm_codes(transport)
assert "reconcile_mismatch_changed" in codes
def test_c_case_publish_failure_fingerprint_not_advanced():
"""Transport failure → fingerprint not advanced → retry on next detection."""
failing_emitter = _make_failing_emitter()
ns = InMemoryReconcileNotificationStore()
_run_reconcile_positions(
hub_qty=10, broker_qty=7, emitter=failing_emitter, notification_store=ns,
)
c_key = _c_identity_key(INSTANCE, "005930")
assert ns.get_state(c_key) is None
# Retry with working emitter
emitter2, transport2 = _make_emitter()
_run_reconcile_positions(hub_qty=10, broker_qty=7, emitter=emitter2, notification_store=ns)
assert _alarm_codes(transport2) == ["reconcile_mismatch_detected"]
def test_c_case_no_notification_without_store():
"""Backward compat: no alarm_emitter/notification_store → no crash."""
decisions = _run_reconcile_positions(hub_qty=10, broker_qty=7)
assert "pf-A" in decisions
# ---------------------------------------------------------------------------
# C-case resolved: verify_slot_reconcile
# ---------------------------------------------------------------------------
def test_c_verify_resolution_publishes_resolved():
"""operator verify clears C-case warning → resolved event published."""
emitter, transport = _make_emitter()
ns = InMemoryReconcileNotificationStore()
slot = _make_slot(qty=10.0)
slot_id_str = str(slot.slot_id)
repo = _repo_with_slot(slot)
ws = InMemoryReconcileWarningStore()
ws.set_warned(slot_id_str, portfolio_id="pf-A", case_code="C")
# Seed notification state
c_key = _c_identity_key(INSTANCE, "005930")
ns.set_state(c_key, case_code="C", instance_key=INSTANCE,
symbol="005930", quantity_a=8, quantity_b=10, lifecycle_state="notified")
store = _snapshot("pf-A", 10.0) # hub 10 == broker 10 → match
verify_slot_reconcile(
portfolio_id="pf-A",
store=store,
broker=FakeBroker({"005930": 10}),
slot_repo=repo,
instance_key=INSTANCE,
warn_store=ws,
alarm_emitter=emitter,
notification_store=ns,
)
assert "reconcile_mismatch_resolved" in _alarm_codes(transport)
assert ns.get_state(c_key) is None
def test_c_verify_resolved_publish_failure_warning_clear_preserved():
"""Resolved publish fails → warning clear preserved, pending_resolved set."""
failing_emitter = _make_failing_emitter()
ns = InMemoryReconcileNotificationStore()
slot = _make_slot(qty=10.0)
slot_id_str = str(slot.slot_id)
repo = _repo_with_slot(slot)
ws = InMemoryReconcileWarningStore()
ws.set_warned(slot_id_str, portfolio_id="pf-A", case_code="C")
c_key = _c_identity_key(INSTANCE, "005930")
ns.set_state(c_key, case_code="C", instance_key=INSTANCE,
symbol="005930", quantity_a=8, quantity_b=10, lifecycle_state="notified")
store = _snapshot("pf-A", 10.0)
result = verify_slot_reconcile(
portfolio_id="pf-A",
store=store,
broker=FakeBroker({"005930": 10}),
slot_repo=repo,
instance_key=INSTANCE,
warn_store=ws,
alarm_emitter=failing_emitter,
notification_store=ns,
)
# Warning cleared
assert result["result"] == "cleared"
assert not ws.is_warned(slot_id_str)
# pending_resolved set
state = ns.get_state(c_key)
assert state is not None
assert state["lifecycle_state"] == "pending_resolved"
def test_c_verify_duplicate_resolved_idempotent():
"""No prior notification state → verify resolved is a no-op."""
emitter, transport = _make_emitter()
ns = InMemoryReconcileNotificationStore()
slot = _make_slot(qty=10.0)
slot_id_str = str(slot.slot_id)
repo = _repo_with_slot(slot)
ws = InMemoryReconcileWarningStore()
ws.set_warned(slot_id_str, portfolio_id="pf-A", case_code="C")
store = _snapshot("pf-A", 10.0)
verify_slot_reconcile(
portfolio_id="pf-A",
store=store,
broker=FakeBroker({"005930": 10}),
slot_repo=repo,
instance_key=INSTANCE,
warn_store=ws,
alarm_emitter=emitter,
notification_store=ns,
)
resolved = [c for c in _alarm_codes(transport) if c == "reconcile_mismatch_resolved"]
assert len(resolved) == 0
def test_c_verify_resolved_retry_on_next_reconcile():
"""pending_resolved → next reconcile_positions pass retries resolved publish."""
ns = InMemoryReconcileNotificationStore()
c_key = _c_identity_key(INSTANCE, "005930")
ns.set_state(c_key, case_code="C", instance_key=INSTANCE,
symbol="005930", quantity_a=8, quantity_b=10, lifecycle_state="pending_resolved")
emitter, transport = _make_emitter()
# Simulate match (hub == broker) so no active mismatch
_run_reconcile_positions(hub_qty=10, broker_qty=10, emitter=emitter, notification_store=ns)
assert "reconcile_mismatch_resolved" in _alarm_codes(transport)
assert ns.get_state(c_key) is None
# ---------------------------------------------------------------------------
# Notification-state persistence failure: business path not aborted
# ---------------------------------------------------------------------------
def test_notification_state_persistence_failure_does_not_abort_reconcile():
"""notification_store raises → reconciliation completes normally."""
bad_ns = MagicMock()
bad_ns.get_state.side_effect = RuntimeError("db down")
bad_ns.set_state.side_effect = RuntimeError("db down")
emitter, _transport = _make_emitter()
slot = _make_slot(qty=10.0)
repo = _repo_with_slot(slot)
ss = SlotService(repo)
decisions = reconcile_core_holdings(
slot_repo=repo, slot_service=ss,
broker=FakeBroker({"005930": 7}),
instance_key=INSTANCE,
alarm_emitter=emitter,
notification_store=bad_ns,
)
# Business path: decisions produced, no crash
assert str(slot.slot_id) in decisions
assert decisions[str(slot.slot_id)] == "qty_warning"
def test_notification_state_persistence_failure_c_case_does_not_abort_reconcile():
"""C-case notification_store raises → reconcile_positions completes normally."""
bad_ns = MagicMock()
bad_ns.get_state.side_effect = RuntimeError("db down")
bad_ns.list_states_by_lifecycle.side_effect = RuntimeError("db down")
emitter, _transport = _make_emitter()
decisions = _run_reconcile_positions(
hub_qty=10, broker_qty=7, emitter=emitter, notification_store=bad_ns,
)
assert "pf-A" in decisions
# ---------------------------------------------------------------------------
# Sensitive-field absence
# ---------------------------------------------------------------------------
def test_b_case_alarm_fields_no_sensitive_data():
"""B-case alarm fields contain only allowlist values — no account/credential/DSN."""
emitter, transport = _make_emitter()
ns = InMemoryReconcileNotificationStore()
slot = _make_slot(qty=10.0)
repo = _repo_with_slot(slot)
ss = SlotService(repo)
reconcile_core_holdings(
slot_repo=repo, slot_service=ss,
broker=FakeBroker({"005930": 7}),
instance_key=INSTANCE,
alarm_emitter=emitter, notification_store=ns,
)
events = transport.alarm_events()
assert len(events) == 1
_, payload = events[0]
fields = payload["fields"]
# Allowlist: instance_key, symbol, case_code, broker_quantity, slot_quantity, slot_id
allowed = {"instance_key", "symbol", "case_code", "broker_quantity", "slot_quantity", "slot_id"}
extra = set(fields.keys()) - allowed
assert extra == set(), f"Unexpected fields: {extra}"
# No sensitive values
all_values = " ".join(str(v) for v in fields.values())
for forbidden in ("account", "credential", "token", "dsn", "postgresql", "amqp", "secret"):
assert forbidden not in all_values.lower(), f"Sensitive token '{forbidden}' found in fields"
def test_c_case_alarm_fields_no_sensitive_data():
"""C-case alarm fields contain only allowlist values."""
emitter, transport = _make_emitter()
ns = InMemoryReconcileNotificationStore()
_run_reconcile_positions(hub_qty=10, broker_qty=7, emitter=emitter, notification_store=ns)
_, payload = transport.alarm_events()[0]
fields = payload["fields"]
allowed = {"instance_key", "symbol", "case_code", "broker_quantity", "hub_quantity"}
extra = set(fields.keys()) - allowed
assert extra == set(), f"Unexpected fields: {extra}"
# hub_quantity present, slot_quantity absent
assert "hub_quantity" in fields
assert "slot_quantity" not in fields
def test_resolved_alarm_fields_b_case():
"""Resolved B-case alarm fields include allowlist values only."""
emitter, transport = _make_emitter()
ns = InMemoryReconcileNotificationStore()
slot = _make_slot(qty=10.0)
slot_id_str = str(slot.slot_id)
repo = _repo_with_slot(slot)
ss = SlotService(repo)
ws = InMemoryReconcileWarningStore()
ws.set_warned(slot_id_str, portfolio_id="pf-A", case_code="B")
ns.set_state(slot_id_str, case_code="B", instance_key=INSTANCE,
symbol="005930", quantity_a=10, quantity_b=7, lifecycle_state="notified")
sync_slot_to_broker(
slot_id=slot_id_str,
slot_repo=repo, slot_service=ss,
broker=FakeBroker({"005930": 7}),
instance_key=INSTANCE,
warn_store=ws,
alarm_emitter=emitter, notification_store=ns,
)
resolved_events = [
(rk, p) for rk, p in transport.alarm_events()
if p["code"] == "reconcile_mismatch_resolved"
]
assert len(resolved_events) == 1
_, payload = resolved_events[0]
fields = payload["fields"]
allowed = {"instance_key", "symbol", "case_code", "broker_quantity", "slot_quantity", "slot_id"}
extra = set(fields.keys()) - allowed
assert extra == set(), f"Unexpected fields: {extra}"
# ---------------------------------------------------------------------------
# Existing behavior preservation
# ---------------------------------------------------------------------------
def test_b_case_existing_alarm_emitter_still_works_without_notification_store():
"""AlarmEmitter existing callers: no notification_store → no regression."""
emitter, transport = _make_emitter()
slot = _make_slot(qty=10.0)
repo = _repo_with_slot(slot)
ss = SlotService(repo)
ws = InMemoryReconcileWarningStore()
reconcile_core_holdings(
slot_repo=repo, slot_service=ss,
broker=FakeBroker({"005930": 7}),
instance_key=INSTANCE,
alarm_emitter=emitter,
warn_store=ws,
)
# No alarm from B-case without notification_store (only existing A-case alarm path unchanged)
assert len(transport.alarm_events()) == 0
assert ws.is_warned(str(slot.slot_id))
def test_publish_alarm_returns_bool():
"""AlarmEmitter.publish_alarm returns True on success, False on failure."""
emitter, _ = _make_emitter()
result = emitter.publish_alarm("test_code", "Test title")
assert result is True
failing = _make_failing_emitter()
result2 = failing.publish_alarm("test_code", "Test title")
assert result2 is False
# ---------------------------------------------------------------------------
# Regression tests for ac-0024 blocker fixes
# ---------------------------------------------------------------------------
def test_c_identity_key_is_injective_and_postgres_safe():
"""_c_identity_key must be injective (no collisions) and PostgreSQL TEXT-safe.
Specifically verifies the crafted-separator collision that bit the \\x1f
implementation: ('a\\x1fC', 'b') and ('a', 'C\\x1fb') previously produced
the same key and suppressed the second detected event.
"""
key = _c_identity_key("my-instance", "005930")
assert "\x00" not in key, f"identity key contains NUL: {key!r}"
# Different symbols → different keys
assert key != _c_identity_key("my-instance", "005931")
# Different instances → different keys
assert key != _c_identity_key("other-instance", "005930")
# Crafted-separator collision must not exist
crafted_a = _c_identity_key("a\x1fC", "b")
crafted_b = _c_identity_key("a", "C\x1fb")
assert crafted_a != crafted_b, (
f"Separator-crafted collision: both produce {crafted_a!r}"
)
def test_retry_pending_resolved_ignores_other_instance_states():
"""_retry_pending_resolved must not touch pending states belonging to a different instance."""
from trex_core.application.reconciliation import _retry_pending_resolved
ns = InMemoryReconcileNotificationStore()
other_slot_id = "slot-other-instance"
# pending_resolved belonging to *another* instance
ns.set_state(other_slot_id, case_code="B", instance_key="other-instance",
symbol="005930", quantity_a=10, quantity_b=7, lifecycle_state="pending_resolved")
emitter, transport = _make_emitter()
_retry_pending_resolved(
emitter, ns,
case_code="B",
instance_key=INSTANCE, # current instance — should NOT see other-instance states
active_identity_keys=set(),
)
# No resolved alarm published for other instance's state
codes = _alarm_codes(transport)
assert "reconcile_mismatch_resolved" not in codes
# State preserved (not consumed by wrong instance)
assert ns.get_state(other_slot_id) is not None
def test_b_sync_does_not_publish_resolved_when_clear_warned_fails():
"""Resolved alarm must NOT be published when clear_warned() raises (warning still active)."""
from trex_core.application.reconciliation import sync_slot_to_broker
class FailingWarnStore(InMemoryReconcileWarningStore):
def clear_warned(self, slot_id):
raise RuntimeError("DB unavailable")
ns = InMemoryReconcileNotificationStore()
slot = _make_slot(qty=10.0)
slot_id_str = str(slot.slot_id)
repo = _repo_with_slot(slot)
ss = SlotService(repo)
ws = FailingWarnStore()
ws.set_warned(slot_id_str, portfolio_id="pf-A", case_code="B")
ns.set_state(slot_id_str, case_code="B", instance_key=INSTANCE,
symbol="005930", quantity_a=10, quantity_b=7, lifecycle_state="notified")
emitter, transport = _make_emitter()
result = sync_slot_to_broker(
slot_id=slot_id_str,
slot_repo=repo,
slot_service=ss,
broker=FakeBroker({"005930": 7}), # qty matches → clear attempt made
instance_key=INSTANCE,
warn_store=ws,
alarm_emitter=emitter,
notification_store=ns,
)
# warning_cleared is False because clear_warned raised
assert result["warning_cleared"] is False
# No resolved alarm should have been published
codes = _alarm_codes(transport)
assert "reconcile_mismatch_resolved" not in codes
# notification state untouched
state = ns.get_state(slot_id_str)
assert state is not None
assert state["lifecycle_state"] == "notified"
@@ -3,6 +3,8 @@ from __future__ import annotations
import logging
import pytest
from trex_core.application.broker_port import BalanceItem
from trex_core.application.intents import BuyIntent, IntentService, SellIntent
from trex_core.application.messaging import RpcRequest
@@ -58,6 +60,72 @@ class InMemoryIntentStore:
self._db[intent_id] = {"status": status, "slot_id": slot_id, "reason": reason}
class RecordingIntentStore(InMemoryIntentStore):
def __init__(self, trace=None, fail_finalize=False):
super().__init__()
self._trace = trace
self._fail_finalize = fail_finalize
def finalize(self, intent_id, status, slot_id, reason):
if self._trace is not None:
self._trace.append("finalize")
if self._fail_finalize:
raise RuntimeError("finalize failed")
super().finalize(intent_id, status, slot_id, reason)
class RecordingAlarmEmitter:
def __init__(self, *, trace=None, return_value=True, raise_error=False):
self.calls = []
self._trace = trace
self._return_value = return_value
self._raise_error = raise_error
def publish_alarm(self, code, title, *, fields=None, severity="critical"):
if self._trace is not None:
self._trace.append("publish")
self.calls.append((code, title, severity, fields or {}))
if self._raise_error:
raise RuntimeError("transport down")
return self._return_value
class DuplicateInFlightIntentStore:
def try_insert(self, intent_id, intent_type, payload):
return False
def get(self, intent_id):
return None
def finalize(self, intent_id, status, slot_id, reason):
raise AssertionError("finalize must not be called for duplicate_in_flight")
class PostFinalizeContextFailingSlotRepository(InMemorySlotRepository):
"""Raise fetch_slot only after explicitly armed by finalize."""
def __init__(self):
super().__init__()
self._raise_on_fetch = False
def fetch_slot(self, slot_id):
if self._raise_on_fetch:
raise RuntimeError("context lookup failed")
return super().fetch_slot(slot_id)
class FinalizeArmingIntentStore(InMemoryIntentStore):
"""Arm repo fetch failure immediately after finalize persistence."""
def __init__(self, repo):
super().__init__()
self._repo = repo
def finalize(self, intent_id, status, slot_id, reason):
super().finalize(intent_id, status, slot_id, reason)
self._repo._raise_on_fetch = True
def _make_holding_slot(repo, svc, instance_key="default", portfolio_id="pf-A", symbol="005930", qty=10):
"""Helper: creates universe, puts slot in HOLDING state with given qty."""
repo.reconcile_universe(instance_key, {portfolio_id: symbol})
@@ -272,6 +340,256 @@ def test_intent_service_sell_rejected_when_reconcile_warned():
assert d.reason == "reconcile_warned"
def test_reconcile_warned_buy_emits_alarm_with_allowlisted_fields_after_finalize():
repo = InMemorySlotRepository()
svc = SlotService(repo)
warn_store = InMemoryReconcileWarningStore()
trace = []
intent_store = RecordingIntentStore(trace=trace)
alarms = RecordingAlarmEmitter(trace=trace)
repo.reconcile_universe("default", {"pf-A": "005930"})
slot = repo.find_slot_by_portfolio("default", "pf-A")
warn_store.set_warned(str(slot.slot_id), portfolio_id="pf-A", case_code="B")
intent_svc = IntentService(
svc, repo, intent_store,
instance_key="default",
warn_store=warn_store,
alarm_emitter=alarms,
)
d = intent_svc.submit_buy(BuyIntent(
intent_id="b-alert-001",
slot_id=slot.slot_id,
entry_price=100.0,
qty=10.0,
strategy_id="stg-buy",
))
assert d.status == "rejected"
assert d.reason == "reconcile_warned"
assert trace == ["finalize", "publish"]
assert len(alarms.calls) == 1
code, title, severity, fields = alarms.calls[0]
assert code == "strategy_intent_rejected"
assert severity == "warn"
assert title
assert fields["instance_key"] == "default"
assert fields["intent_id"] == "b-alert-001"
assert fields["intent_type"] == "buy"
assert fields["reason"] == "reconcile_warned"
assert fields["broker_submitted"] is False
assert fields["slot_id"] == str(slot.slot_id)
assert fields["portfolio_id"] == "pf-A"
assert fields["symbol"] == "005930"
assert fields["strategy_id"] == "stg-buy"
for blocked in ("qty", "entry_price", "limit_price", "venue", "extra", "sell_strategy_id"):
assert blocked not in fields
text = repr((title, fields))
for forbidden in ("account", "token", "password", "secret", "dsn", "connection", "raw_payload"):
assert forbidden not in text.lower()
def test_reconcile_warned_sell_emits_alarm_with_normalized_strategy_id():
repo = InMemorySlotRepository()
svc = SlotService(repo)
warn_store = InMemoryReconcileWarningStore()
intent_store = InMemoryIntentStore()
alarms = RecordingAlarmEmitter()
slot = _make_holding_slot(repo, svc, qty=10, portfolio_id="pf-A", symbol="005930")
warn_store.set_warned(str(slot.slot_id), portfolio_id="pf-A", case_code="C")
intent_svc = IntentService(
svc, repo, intent_store,
instance_key="default",
warn_store=warn_store,
alarm_emitter=alarms,
)
d = intent_svc.submit_sell(SellIntent(
intent_id="s-alert-001",
slot_id=slot.slot_id,
reason="strategy_exit",
sell_strategy_id="stg-sell",
))
assert d.status == "rejected"
assert d.reason == "reconcile_warned"
assert len(alarms.calls) == 1
_, _, severity, fields = alarms.calls[0]
assert severity == "warn"
assert fields["intent_type"] == "sell"
assert fields["strategy_id"] == "stg-sell"
assert fields["slot_id"] == str(slot.slot_id)
def test_reconcile_warned_alarm_dedup_by_intent_id_across_service_recreation():
repo = InMemorySlotRepository()
svc = SlotService(repo)
warn_store = InMemoryReconcileWarningStore()
intent_store = InMemoryIntentStore()
alarms = RecordingAlarmEmitter()
repo.reconcile_universe("default", {"pf-A": "005930"})
slot = repo.find_slot_by_portfolio("default", "pf-A")
warn_store.set_warned(str(slot.slot_id), portfolio_id="pf-A", case_code="B")
s1 = IntentService(
svc, repo, intent_store, instance_key="default", warn_store=warn_store, alarm_emitter=alarms
)
s2 = IntentService(
svc, repo, intent_store, instance_key="default", warn_store=warn_store, alarm_emitter=alarms
)
first = s1.submit_buy(BuyIntent(intent_id="b-dedup", slot_id=slot.slot_id))
second = s1.submit_buy(BuyIntent(intent_id="b-dedup", slot_id=slot.slot_id))
third = s2.submit_buy(BuyIntent(intent_id="b-dedup", slot_id=slot.slot_id))
assert first.reason == "reconcile_warned"
assert second.reason == "reconcile_warned"
assert third.reason == "reconcile_warned"
assert len(alarms.calls) == 1
def test_reconcile_warned_alarm_emits_per_distinct_intent_id_on_same_slot():
repo = InMemorySlotRepository()
svc = SlotService(repo)
warn_store = InMemoryReconcileWarningStore()
intent_store = InMemoryIntentStore()
alarms = RecordingAlarmEmitter()
repo.reconcile_universe("default", {"pf-A": "005930"})
slot = repo.find_slot_by_portfolio("default", "pf-A")
warn_store.set_warned(str(slot.slot_id), portfolio_id="pf-A", case_code="B")
intent_svc = IntentService(
svc, repo, intent_store, instance_key="default", warn_store=warn_store, alarm_emitter=alarms
)
d1 = intent_svc.submit_buy(BuyIntent(intent_id="b-uniq-1", slot_id=slot.slot_id))
d2 = intent_svc.submit_buy(BuyIntent(intent_id="b-uniq-2", slot_id=slot.slot_id))
assert d1.reason == "reconcile_warned"
assert d2.reason == "reconcile_warned"
assert len(alarms.calls) == 2
def test_reconcile_warned_alarm_not_published_for_duplicate_in_flight_or_other_decisions():
repo = InMemorySlotRepository()
svc = SlotService(repo)
alarms = RecordingAlarmEmitter()
duplicate = IntentService(
svc, repo, DuplicateInFlightIntentStore(), instance_key="default", alarm_emitter=alarms
)
d_dup = duplicate.submit_buy(BuyIntent(intent_id="dup", portfolio_id="pf-A"))
assert d_dup.reason == "duplicate_in_flight"
assert alarms.calls == []
repo.reconcile_universe("default", {"pf-A": "005930"})
slot = repo.find_slot_by_portfolio("default", "pf-A")
normal = IntentService(
svc, repo, InMemoryIntentStore(), instance_key="default", alarm_emitter=alarms
)
admitted = normal.submit_buy(BuyIntent(intent_id="admitted", slot_id=slot.slot_id, qty=1, entry_price=100))
assert admitted.status == "admitted"
invalid = normal.submit_buy(BuyIntent(intent_id="bad-venue", slot_id=slot.slot_id, venue="BAD"))
assert invalid.reason == "invalid_venue"
repo.reconcile_universe("default", {"pf-A": "005930", "pf-B": "000660"})
slot_b = repo.find_slot_by_portfolio("default", "pf-B")
capped = IntentService(
svc, repo, InMemoryIntentStore(), instance_key="default", max_active_slots=0, alarm_emitter=alarms
)
full = capped.submit_buy(BuyIntent(intent_id="full", slot_id=slot_b.slot_id))
assert full.reason == "capacity_full"
assert alarms.calls == []
def test_reconcile_warned_alarm_finalize_failure_and_publish_failure_are_isolated():
repo = InMemorySlotRepository()
svc = SlotService(repo)
warn_store = InMemoryReconcileWarningStore()
repo.reconcile_universe("default", {"pf-A": "005930"})
slot = repo.find_slot_by_portfolio("default", "pf-A")
warn_store.set_warned(str(slot.slot_id), portfolio_id="pf-A", case_code="B")
alarms = RecordingAlarmEmitter()
failing_store = RecordingIntentStore(fail_finalize=True)
failing_finalize = IntentService(
svc, repo, failing_store,
instance_key="default",
warn_store=warn_store,
alarm_emitter=alarms,
)
with pytest.raises(RuntimeError, match="finalize failed"):
failing_finalize.submit_buy(BuyIntent(intent_id="b-finalize-fail", slot_id=slot.slot_id))
assert alarms.calls == []
soft_fail_alarm = RecordingAlarmEmitter(return_value=False)
raising_alarm = RecordingAlarmEmitter(raise_error=True)
non_blocking_1 = IntentService(
svc, repo, InMemoryIntentStore(), instance_key="default", warn_store=warn_store, alarm_emitter=soft_fail_alarm
)
non_blocking_2 = IntentService(
svc, repo, InMemoryIntentStore(), instance_key="default", warn_store=warn_store, alarm_emitter=raising_alarm
)
d1 = non_blocking_1.submit_buy(BuyIntent(intent_id="b-alarm-false", slot_id=slot.slot_id))
d2 = non_blocking_2.submit_buy(BuyIntent(intent_id="b-alarm-exc", slot_id=slot.slot_id))
d3 = IntentService(
svc, repo, InMemoryIntentStore(), instance_key="default", warn_store=warn_store, alarm_emitter=None
).submit_buy(BuyIntent(intent_id="b-no-emitter", slot_id=slot.slot_id))
assert d1.reason == "reconcile_warned"
assert d2.reason == "reconcile_warned"
assert d3.reason == "reconcile_warned"
def test_reconcile_warned_buy_context_lookup_failure_does_not_change_response():
repo = PostFinalizeContextFailingSlotRepository()
svc = SlotService(repo)
warn_store = InMemoryReconcileWarningStore()
intent_store = FinalizeArmingIntentStore(repo)
alarms = RecordingAlarmEmitter()
repo.reconcile_universe("default", {"pf-A": "005930"})
slot = repo.find_slot_by_portfolio("default", "pf-A")
warn_store.set_warned(str(slot.slot_id), portfolio_id="pf-A", case_code="B")
intent_svc = IntentService(
svc, repo, intent_store, instance_key="default", warn_store=warn_store, alarm_emitter=alarms
)
decision = intent_svc.submit_buy(
BuyIntent(intent_id="buy-context-fail", slot_id=slot.slot_id, strategy_id="stg-buy")
)
assert decision.status == "rejected"
assert decision.reason == "reconcile_warned"
assert len(alarms.calls) == 1
_, _, _, fields = alarms.calls[0]
assert fields["intent_id"] == "buy-context-fail"
assert fields["strategy_id"] == "stg-buy"
assert "slot_id" not in fields
assert "portfolio_id" not in fields
assert "symbol" not in fields
def test_reconcile_warned_sell_context_lookup_failure_does_not_change_response():
repo = PostFinalizeContextFailingSlotRepository()
svc = SlotService(repo)
warn_store = InMemoryReconcileWarningStore()
intent_store = FinalizeArmingIntentStore(repo)
alarms = RecordingAlarmEmitter()
slot = _make_holding_slot(repo, svc, qty=10, portfolio_id="pf-A", symbol="005930")
warn_store.set_warned(str(slot.slot_id), portfolio_id="pf-A", case_code="C")
intent_svc = IntentService(
svc, repo, intent_store, instance_key="default", warn_store=warn_store, alarm_emitter=alarms
)
decision = intent_svc.submit_sell(
SellIntent(intent_id="sell-context-fail", slot_id=slot.slot_id, sell_strategy_id="stg-sell")
)
assert decision.status == "rejected"
assert decision.reason == "reconcile_warned"
assert len(alarms.calls) == 1
_, _, _, fields = alarms.calls[0]
assert fields["intent_id"] == "sell-context-fail"
assert fields["strategy_id"] == "stg-sell"
assert "slot_id" not in fields
assert "portfolio_id" not in fields
assert "symbol" not in fields
# REQ-104 AC-3: no warn, no pause => buy/sell admitted normally
def test_intent_service_no_warn_admitted_normally():
repo = InMemorySlotRepository()
@@ -598,3 +916,61 @@ def test_slot_to_dict_paused_and_warned_independent():
row = slot_to_dict(slot, pause_store=pause_store, warn_store=warn_store)
assert row["paused"] is True
assert row["reconcile_warned"] is True
# ===========================================================================
# AC-0023 review: warn_store.is_warned() raises → intent fail-closed
# ===========================================================================
class _RaisingWarnStore:
"""Stub warn_store whose is_warned always raises (simulates DB unavailable)."""
def is_warned(self, slot_id: str) -> bool:
raise RuntimeError("DB unavailable")
def list_c_case_warned(self) -> set:
raise RuntimeError("DB unavailable")
def test_buy_intent_fail_closed_when_warn_store_raises():
"""If warn_store.is_warned() raises, buy intent must be rejected (fail-closed)."""
repo = InMemorySlotRepository()
svc = SlotService(repo)
intent_store = InMemoryIntentStore()
repo.reconcile_universe("default", {"pf-A": "005930"})
slot = repo.find_slot_by_portfolio("default", "pf-A")
intent_svc = IntentService(
svc, repo, intent_store,
instance_key="default",
warn_store=_RaisingWarnStore(),
)
d = intent_svc.submit_buy(BuyIntent(
intent_id="b-fc-001",
slot_id=slot.slot_id,
entry_price=100.0,
qty=10.0,
))
assert d.status == "rejected"
assert d.reason == "reconcile_warning_unavailable"
def test_sell_intent_fail_closed_when_warn_store_raises():
"""If warn_store.is_warned() raises, sell intent must be rejected (fail-closed)."""
repo = InMemorySlotRepository()
svc = SlotService(repo)
intent_store = InMemoryIntentStore()
slot = _make_holding_slot(repo, svc, qty=10)
intent_svc = IntentService(
svc, repo, intent_store,
instance_key="default",
warn_store=_RaisingWarnStore(),
)
d = intent_svc.submit_sell(SellIntent(
intent_id="s-fc-001",
slot_id=slot.slot_id,
reason="strategy_exit",
))
assert d.status == "rejected"
assert d.reason == "reconcile_warning_unavailable"
+2 -1
View File
@@ -155,7 +155,8 @@ def test_warned_true_when_qty_diverges():
resp = handler(_req(filter="holding"))
s = resp.payload["slots"][0]
# slot.qty=5 vs hub asset_quantity=3 → divergence
assert s["qty"] == 5.0
assert s["qty"] is None
assert s["slot_quantity"] == 5.0
assert s["asset_quantity"] == 3.0
assert s["warned"] is True
@@ -0,0 +1,231 @@
"""Broker-authoritative qty projection in `trex.slots.req` holding rows."""
from __future__ import annotations
from datetime import UTC, datetime
from typing import Protocol
from trex_core.application.account_snapshot import (
AccountSnapshot,
AccountSnapshotService,
AccountSnapshotUnavailable,
InMemoryAccountSnapshotStore,
)
from trex_core.application.broker_port import BalanceItem
from trex_core.application.messaging import RpcRequest
from trex_core.application.messaging_handlers import make_slots_rpc_handler
from trex_core.application.portfolio import PortfolioItem, SnapshotStore
from trex_core.domain.slot import SlotService, SlotState
from trex_core.domain.slot.in_memory_repository import InMemorySlotRepository
class _Reader:
def __init__(
self,
*,
holdings: list[BalanceItem] | None = None,
captured_at: datetime | None = None,
unavailable_code: str | None = None,
) -> None:
self._holdings = holdings or []
self._captured_at = captured_at
self._unavailable_code = unavailable_code
def holdings_for_slots_rpc(self) -> tuple[list[BalanceItem], datetime]:
if self._unavailable_code is not None:
raise AccountSnapshotUnavailable(self._unavailable_code)
if self._captured_at is None:
raise AccountSnapshotUnavailable("not_ready")
return list(self._holdings), self._captured_at
class _SlotsReader(Protocol):
def holdings_for_slots_rpc(self) -> tuple[list[BalanceItem], datetime]: ...
class _BrokenLoadStore(InMemoryAccountSnapshotStore):
def load(self, account_hash: str) -> AccountSnapshot | None:
raise RuntimeError("db read failed")
class _Broker:
def account_scope_hash(self) -> str:
return "a" * 64
def get_balance(self) -> list[BalanceItem]:
return []
def get_available_cash(self, *, price_check_code: str = "005930") -> int:
return 0
def _holding_handler(
*,
broker_reader: _SlotsReader,
instance_key: str = "default",
portfolio_id: str = "pf-A",
symbol: str = "104620",
):
repo = InMemorySlotRepository()
svc = SlotService(repo)
repo.reconcile_universe(instance_key, {portfolio_id: symbol})
slot = repo.find_slot_by_portfolio(instance_key, portfolio_id)
pending = svc.transit_slot(
slot.slot_id,
SlotState.INACTIVE,
SlotState.PENDING_BUY,
reason="buy",
actor="test",
symbol=symbol,
)
svc.transit_slot(
pending.slot_id,
SlotState.PENDING_BUY,
SlotState.HOLDING,
reason="filled",
actor="test",
avg_price=100.0,
qty=435.0,
)
store = SnapshotStore()
store.swap(
[PortfolioItem(portfolio_id=portfolio_id, symbol=symbol, cash_balance=0, asset_quantity=435)]
)
handler = make_slots_rpc_handler(
repo,
instance_key=instance_key,
snapshot_store=store,
broker_quantity_reader=broker_reader,
)
return repo, handler
def _req_holding():
return RpcRequest(payload={"filter": "holding", "spy": True}, correlation_id="corr")
def test_holding_qty_uses_broker_authority_with_diagnostic_fields():
_, handler = _holding_handler(
broker_reader=_Reader(
holdings=[BalanceItem(symbol="104620", hldg_qty=440)],
captured_at=datetime(2026, 8, 24, 12, 0, tzinfo=UTC),
)
)
row = handler(_req_holding()).payload["slots"][0]
assert row["qty"] == 440.0
assert row["qty_source"] == "broker"
assert row["broker_quantity_available"] is True
assert row["broker_quantity_captured_at"] == "2026-08-24T12:00:00+00:00"
assert row["slot_quantity"] == 435.0
assert row["asset_quantity"] == 435.0
def test_holding_qty_distinguishes_confirmed_broker_zero_from_unavailable():
_, handler = _holding_handler(
broker_reader=_Reader(
holdings=[],
captured_at=datetime(2026, 8, 24, 12, 0, tzinfo=UTC),
)
)
row = handler(_req_holding()).payload["slots"][0]
assert row["qty"] == 0.0
assert row["qty_source"] == "broker"
assert row["broker_quantity_available"] is True
assert row["slot_quantity"] == 435.0
def test_holding_qty_unavailable_does_not_fallback_to_slot_or_hub():
_, handler = _holding_handler(
broker_reader=_Reader(unavailable_code="not_ready"),
)
row = handler(_req_holding()).payload["slots"][0]
assert row["qty"] is None
assert row["qty_source"] == "unavailable"
assert row["broker_quantity_available"] is False
assert row["broker_quantity_captured_at"] is None
assert row["slot_quantity"] == 435.0
assert row["asset_quantity"] == 435.0
def test_holding_qty_stale_snapshot_is_unavailable():
_, handler = _holding_handler(
broker_reader=_Reader(unavailable_code="expired"),
)
row = handler(_req_holding()).payload["slots"][0]
assert row["qty"] is None
assert row["qty_source"] == "unavailable"
assert row["broker_quantity_available"] is False
def test_holding_qty_store_read_failure_is_unavailable():
_, handler = _holding_handler(
broker_reader=AccountSnapshotService(
broker=_Broker(),
store=_BrokenLoadStore(),
buy_max_age_sec=600,
sell_max_age_sec=3600,
clock=lambda: datetime(2026, 8, 24, 12, 0, tzinfo=UTC),
rng=lambda: 0.5,
),
)
row = handler(_req_holding()).payload["slots"][0]
assert row["qty"] is None
assert row["qty_source"] == "unavailable"
assert row["broker_quantity_available"] is False
assert row["broker_quantity_captured_at"] is None
def test_broker_quantity_reader_is_instance_bound_per_handler():
repo = InMemorySlotRepository()
svc = SlotService(repo)
repo.reconcile_universe("inst-A", {"pf-A": "104620"})
repo.reconcile_universe("inst-B", {"pf-B": "104620"})
slot_a = repo.find_slot_by_portfolio("inst-A", "pf-A")
slot_b = repo.find_slot_by_portfolio("inst-B", "pf-B")
pending_a = svc.transit_slot(
slot_a.slot_id, SlotState.INACTIVE, SlotState.PENDING_BUY,
reason="buy", actor="test", symbol="104620",
)
pending_b = svc.transit_slot(
slot_b.slot_id, SlotState.INACTIVE, SlotState.PENDING_BUY,
reason="buy", actor="test", symbol="104620",
)
svc.transit_slot(
pending_a.slot_id, SlotState.PENDING_BUY, SlotState.HOLDING,
reason="filled", actor="test", avg_price=100.0, qty=435.0,
)
svc.transit_slot(
pending_b.slot_id, SlotState.PENDING_BUY, SlotState.HOLDING,
reason="filled", actor="test", avg_price=100.0, qty=435.0,
)
store = SnapshotStore()
store.swap(
[
PortfolioItem(portfolio_id="pf-A", symbol="104620", cash_balance=0, asset_quantity=435),
PortfolioItem(portfolio_id="pf-B", symbol="104620", cash_balance=0, asset_quantity=435),
]
)
captured_at = datetime(2026, 8, 24, 12, 0, tzinfo=UTC)
handler_a = make_slots_rpc_handler(
repo,
instance_key="inst-A",
snapshot_store=store,
broker_quantity_reader=_Reader(
holdings=[BalanceItem(symbol="104620", hldg_qty=440)],
captured_at=captured_at,
),
)
handler_b = make_slots_rpc_handler(
repo,
instance_key="inst-B",
snapshot_store=store,
broker_quantity_reader=_Reader(
holdings=[BalanceItem(symbol="104620", hldg_qty=111)],
captured_at=captured_at,
),
)
row_a = handler_a(_req_holding()).payload["slots"][0]
row_b = handler_b(_req_holding()).payload["slots"][0]
assert row_a["qty"] == 440.0
assert row_b["qty"] == 111.0
@@ -0,0 +1,560 @@
"""AC-0023: C-case quantity mismatch — strategy RPC exclusion, fail-closed, and clearing.
Coverage:
AC-1: single portfolio mismatch → slot excluded
AC-2: multi-portfolio same symbol: aggregate matches → both slots included
AC-3: multi-portfolio same symbol: aggregate mismatches → both slots excluded
AC-4: different symbol slots unaffected by C-case warning on another symbol
AC-5: excluded_slots.reconcile_quantity_mismatch count in response
AC-6: warn_store=None → no filtering (backward compat), no excluded_slots key
AC-7: warn_store raises → fail-closed: slots=[], degraded=true, warn_store_unavailable
AC-8: intent admitted before warning is set → rejected at intent time (race)
AC-9: in-flight (terminal) slot transitions not blocked
AC-10: verify_slot_reconcile uses symbol aggregate for multi-portfolio clear
AC-11: verify_slot_reconcile clears all C-case warnings for symbol when resolved
AC-12: C-case warning not auto-cleared by time; only cleared on quantity match
"""
from __future__ import annotations
from trex_core.application.broker_port import BalanceItem
from trex_core.application.intents import IntentService, SellIntent
from trex_core.application.messaging import RpcRequest
from trex_core.application.messaging_handlers import make_slots_rpc_handler
from trex_core.application.portfolio import (
InMemoryPauseStore,
InMemoryReconcileWarningStore,
PortfolioItem,
SnapshotStore,
)
from trex_core.application.reconciliation import (
reconcile_positions,
verify_slot_reconcile,
)
from trex_core.application.watchdog import WatchdogStore
from trex_core.domain.slot import SlotService, SlotState
from trex_core.domain.slot.in_memory_repository import InMemorySlotRepository
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
INSTANCE = "default"
SYM_A = "104620"
SYM_B = "005930"
def _make_holding_slot(repo, svc, portfolio_id="pf-A", symbol=SYM_A, qty=10, instance_key=INSTANCE):
repo.reconcile_universe(instance_key, {portfolio_id: symbol})
slot = repo.find_slot_by_portfolio(instance_key, portfolio_id)
pending = svc.transit_slot(
slot.slot_id, SlotState.INACTIVE, SlotState.PENDING_BUY,
reason="buy", actor="test", symbol=symbol,
)
holding = svc.transit_slot(
pending.slot_id, SlotState.PENDING_BUY, SlotState.HOLDING,
reason="filled", actor="test", avg_price=100.0, qty=float(qty),
)
return holding
def _make_handler(repo, warn_store, snapshot_items=None):
store = SnapshotStore()
if snapshot_items:
store.swap(snapshot_items)
return make_slots_rpc_handler(
repo,
instance_key=INSTANCE,
snapshot_store=store,
pause_store=InMemoryPauseStore(),
warn_store=warn_store,
watchdog_store=WatchdogStore(),
)
def _req(filter=None):
payload = {}
if filter:
payload["filter"] = filter
return RpcRequest(payload=payload, correlation_id="test")
class FakeBroker:
def __init__(self, balances):
self._b = balances
def name(self): return "fake"
def submit_order(self, req): ...
def poll_fill(self, *args, **kw): ...
def cancel_order(self, *args, **kw): ...
def get_balance(self): return list(self._b)
class InMemoryIntentStore:
def __init__(self):
self._db = {}
def try_insert(self, intent_id, intent_type, payload):
if intent_id in self._db:
return False
self._db[intent_id] = {"status": None, "slot_id": None, "reason": None}
return True
def get(self, intent_id):
rec = self._db.get(intent_id)
if rec is None:
return None
class R:
def __init__(self, d): self.__dict__.update(d)
r = R(rec)
r.intent_id = intent_id
return r
def finalize(self, intent_id, status, slot_id, reason):
self._db[intent_id] = {"status": status, "slot_id": slot_id, "reason": reason}
# ---------------------------------------------------------------------------
# AC-1: single portfolio C-case → slot excluded from strategy RPC
# ---------------------------------------------------------------------------
def test_ac1_single_portfolio_c_case_excluded():
repo = InMemorySlotRepository()
svc = SlotService(repo)
warn_store = InMemoryReconcileWarningStore()
slot = _make_holding_slot(repo, svc, portfolio_id="pf-A", symbol=SYM_A, qty=10)
warn_store.set_warned(str(slot.slot_id), portfolio_id="pf-A", case_code="C")
handler = _make_handler(repo, warn_store)
resp = handler(_req(filter="holding"))
assert resp.payload["slots"] == []
assert resp.payload["excluded_slots"]["reconcile_quantity_mismatch"] == 1
# ---------------------------------------------------------------------------
# AC-2: multi-portfolio same symbol: KIS 440, A=200, B=240 → aggregate 440 → both included
# ---------------------------------------------------------------------------
def test_ac2_multi_portfolio_aggregate_matches_both_included():
repo = InMemorySlotRepository()
svc = SlotService(repo)
warn_store = InMemoryReconcileWarningStore()
# Build two holding slots for same symbol
repo.reconcile_universe(INSTANCE, {"pf-A": SYM_A, "pf-B": SYM_A})
slot_a_base = repo.find_slot_by_portfolio(INSTANCE, "pf-A")
slot_b_base = repo.find_slot_by_portfolio(INSTANCE, "pf-B")
p = svc.transit_slot(slot_a_base.slot_id, SlotState.INACTIVE, SlotState.PENDING_BUY,
reason="buy", actor="test", symbol=SYM_A)
svc.transit_slot(p.slot_id, SlotState.PENDING_BUY, SlotState.HOLDING,
reason="filled", actor="test", avg_price=100.0, qty=200.0)
p = svc.transit_slot(slot_b_base.slot_id, SlotState.INACTIVE, SlotState.PENDING_BUY,
reason="buy", actor="test", symbol=SYM_A)
svc.transit_slot(p.slot_id, SlotState.PENDING_BUY, SlotState.HOLDING,
reason="filled", actor="test", avg_price=100.0, qty=240.0)
# No C-case warnings set (aggregate matches KIS 440)
handler = _make_handler(repo, warn_store)
resp = handler(_req(filter="holding"))
portfolio_ids = {r["portfolio_id"] for r in resp.payload["slots"]}
assert "pf-A" in portfolio_ids
assert "pf-B" in portfolio_ids
assert resp.payload["excluded_slots"]["reconcile_quantity_mismatch"] == 0
def test_ac2_multi_portfolio_individual_qty_vs_kis_total_not_compared():
"""AC-2: each portfolio qty (200, 240) != KIS 440, but aggregate 440 == KIS 440 → no block."""
repo = InMemorySlotRepository()
svc = SlotService(repo)
warn_store = InMemoryReconcileWarningStore()
repo.reconcile_universe(INSTANCE, {"pf-A": SYM_A, "pf-B": SYM_A})
for pid, qty in [("pf-A", 200.0), ("pf-B", 240.0)]:
base = repo.find_slot_by_portfolio(INSTANCE, pid)
p = svc.transit_slot(base.slot_id, SlotState.INACTIVE, SlotState.PENDING_BUY,
reason="buy", actor="test", symbol=SYM_A)
svc.transit_slot(p.slot_id, SlotState.PENDING_BUY, SlotState.HOLDING,
reason="filled", actor="test", avg_price=100.0, qty=qty)
# simulate reconcile_positions with broker=440 → aggregate 440 == 440 → no warning set
snapshot_items = [
PortfolioItem(portfolio_id="pf-A", symbol=SYM_A, cash_balance=0, asset_quantity=200.0),
PortfolioItem(portfolio_id="pf-B", symbol=SYM_A, cash_balance=0, asset_quantity=240.0),
]
broker = FakeBroker([BalanceItem(symbol=SYM_A, hldg_qty=440)])
store = SnapshotStore()
store.swap(snapshot_items)
reconcile_positions(store=store, broker=broker, slot_repo=repo,
instance_key=INSTANCE, warn_store=warn_store)
# No warnings should be set
assert warn_store.list_c_case_warned() == set()
handler = _make_handler(repo, warn_store, snapshot_items)
resp = handler(_req(filter="holding"))
assert len(resp.payload["slots"]) == 2
assert resp.payload["excluded_slots"]["reconcile_quantity_mismatch"] == 0
# ---------------------------------------------------------------------------
# AC-3: multi-portfolio same symbol: aggregate mismatches → all excluded
# ---------------------------------------------------------------------------
def test_ac3_multi_portfolio_aggregate_mismatch_all_excluded():
"""KIS 440, hub A=200 + B=235 → aggregate 435 ≠ 440 → both slots excluded."""
repo = InMemorySlotRepository()
svc = SlotService(repo)
warn_store = InMemoryReconcileWarningStore()
repo.reconcile_universe(INSTANCE, {"pf-A": SYM_A, "pf-B": SYM_A})
for pid, qty in [("pf-A", 200.0), ("pf-B", 235.0)]:
base = repo.find_slot_by_portfolio(INSTANCE, pid)
p = svc.transit_slot(base.slot_id, SlotState.INACTIVE, SlotState.PENDING_BUY,
reason="buy", actor="test", symbol=SYM_A)
svc.transit_slot(p.slot_id, SlotState.PENDING_BUY, SlotState.HOLDING,
reason="filled", actor="test", avg_price=100.0, qty=qty)
# Set C-case warning for both
snapshot_items = [
PortfolioItem(portfolio_id="pf-A", symbol=SYM_A, cash_balance=0, asset_quantity=200.0),
PortfolioItem(portfolio_id="pf-B", symbol=SYM_A, cash_balance=0, asset_quantity=235.0),
]
broker = FakeBroker([BalanceItem(symbol=SYM_A, hldg_qty=440)])
store = SnapshotStore()
store.swap(snapshot_items)
reconcile_positions(store=store, broker=broker, slot_repo=repo,
instance_key=INSTANCE, warn_store=warn_store)
# Both should be C-case warned
assert len(warn_store.list_c_case_warned()) == 2
handler = _make_handler(repo, warn_store, snapshot_items)
resp = handler(_req(filter="holding"))
assert resp.payload["slots"] == []
assert resp.payload["excluded_slots"]["reconcile_quantity_mismatch"] == 2
# ---------------------------------------------------------------------------
# AC-4: other-symbol slots not affected by C-case warning on a different symbol
# ---------------------------------------------------------------------------
def test_ac4_other_symbol_slots_unaffected():
repo = InMemorySlotRepository()
svc = SlotService(repo)
warn_store = InMemoryReconcileWarningStore()
repo.reconcile_universe(INSTANCE, {"pf-A": SYM_A, "pf-B": SYM_B})
# pf-A: SYM_A, holding, C-case warned
base_a = repo.find_slot_by_portfolio(INSTANCE, "pf-A")
p = svc.transit_slot(base_a.slot_id, SlotState.INACTIVE, SlotState.PENDING_BUY,
reason="buy", actor="test", symbol=SYM_A)
slot_a = svc.transit_slot(p.slot_id, SlotState.PENDING_BUY, SlotState.HOLDING,
reason="filled", actor="test", avg_price=100.0, qty=10.0)
warn_store.set_warned(str(slot_a.slot_id), portfolio_id="pf-A", case_code="C")
# pf-B: SYM_B, holding, no warning
base_b = repo.find_slot_by_portfolio(INSTANCE, "pf-B")
p = svc.transit_slot(base_b.slot_id, SlotState.INACTIVE, SlotState.PENDING_BUY,
reason="buy", actor="test", symbol=SYM_B)
svc.transit_slot(p.slot_id, SlotState.PENDING_BUY, SlotState.HOLDING,
reason="filled", actor="test", avg_price=200.0, qty=5.0)
handler = _make_handler(repo, warn_store)
resp = handler(_req(filter="holding"))
portfolio_ids = {r["portfolio_id"] for r in resp.payload["slots"]}
assert "pf-A" not in portfolio_ids, "C-case warned slot should be excluded"
assert "pf-B" in portfolio_ids, "unwarned symbol slot should be included"
assert resp.payload["excluded_slots"]["reconcile_quantity_mismatch"] == 1
# ---------------------------------------------------------------------------
# AC-5: excluded_slots present and count is correct
# ---------------------------------------------------------------------------
def test_ac5_excluded_slots_count_zero_when_no_warnings():
repo = InMemorySlotRepository()
svc = SlotService(repo)
warn_store = InMemoryReconcileWarningStore()
_make_holding_slot(repo, svc, portfolio_id="pf-A", symbol=SYM_A, qty=10)
handler = _make_handler(repo, warn_store)
resp = handler(_req(filter="holding"))
assert "excluded_slots" in resp.payload
assert resp.payload["excluded_slots"] == {"reconcile_quantity_mismatch": 0}
# ---------------------------------------------------------------------------
# AC-6: warn_store=None → no filtering, no excluded_slots key (backward compat)
# ---------------------------------------------------------------------------
def test_ac6_no_warn_store_no_filtering_no_excluded_slots_key():
repo = InMemorySlotRepository()
svc = SlotService(repo)
_make_holding_slot(repo, svc, portfolio_id="pf-A", symbol=SYM_A, qty=10)
handler = make_slots_rpc_handler(repo, instance_key=INSTANCE) # no warn_store
resp = handler(_req(filter="holding"))
# No warn_store → no filtering (slot included), no excluded_slots key
assert len(resp.payload["slots"]) == 1
assert "excluded_slots" not in resp.payload
# ---------------------------------------------------------------------------
# AC-7: warn_store raises → fail-closed
# ---------------------------------------------------------------------------
class BrokenWarnStore(InMemoryReconcileWarningStore):
"""Simulates a warn_store that fails on list_c_case_warned."""
def list_c_case_warned(self):
raise RuntimeError("db connection lost")
def test_ac7_warn_store_failure_fail_closed():
repo = InMemorySlotRepository()
svc = SlotService(repo)
_make_holding_slot(repo, svc, portfolio_id="pf-A", symbol=SYM_A, qty=10)
broken_warn_store = BrokenWarnStore()
handler = _make_handler(repo, broken_warn_store)
resp = handler(_req())
assert resp.payload["slots"] == []
assert resp.payload["degraded"] is True
assert resp.payload["reason"] == "warn_store_unavailable"
assert "excluded_slots" in resp.payload
# ---------------------------------------------------------------------------
# AC-8: intent rejected if C-case warning set after last slots RPC (race)
# ---------------------------------------------------------------------------
def test_ac8_intent_rejected_when_c_case_warning_set_after_slots_rpc():
"""Even if strategy saw the slot in a previous RPC, intent is rejected at processing time."""
repo = InMemorySlotRepository()
svc = SlotService(repo)
warn_store = InMemoryReconcileWarningStore()
slot = _make_holding_slot(repo, svc, portfolio_id="pf-A", symbol=SYM_A, qty=10)
intent_store = InMemoryIntentStore()
intent_svc = IntentService(
svc, repo, intent_store,
instance_key=INSTANCE,
warn_store=warn_store,
)
# Warning set after strategy "saw" the slot
warn_store.set_warned(str(slot.slot_id), portfolio_id="pf-A", case_code="C")
sell = SellIntent(intent_id="sell-001", slot_id=slot.slot_id, reason="test")
d = intent_svc.submit_sell(sell)
assert d.status == "rejected"
assert d.reason == "reconcile_warned"
# ---------------------------------------------------------------------------
# AC-9: in-flight (holding→pending_sell) transition not blocked
# ---------------------------------------------------------------------------
def test_ac9_in_flight_order_terminal_transition_not_blocked():
"""C-case warning does not block slot state convergence for in-flight orders."""
repo = InMemorySlotRepository()
svc = SlotService(repo)
warn_store = InMemoryReconcileWarningStore()
slot = _make_holding_slot(repo, svc, portfolio_id="pf-A", symbol=SYM_A, qty=10)
warn_store.set_warned(str(slot.slot_id), portfolio_id="pf-A", case_code="C")
# Transition to PENDING_SELL (simulating in-flight cancel) — should not raise
pending_sell = svc.transit_slot(
slot.slot_id, SlotState.HOLDING, SlotState.PENDING_SELL,
reason="sell", actor="test",
)
assert pending_sell.status == SlotState.PENDING_SELL
# Further transition to INACTIVE (fill/cancel reconciliation) — should not raise
terminal = svc.transit_slot(
pending_sell.slot_id, SlotState.PENDING_SELL, SlotState.INACTIVE,
reason="filled_sell", actor="test",
)
assert terminal.status == SlotState.INACTIVE
# ---------------------------------------------------------------------------
# AC-10: verify_slot_reconcile uses symbol aggregate for multi-portfolio clear
# ---------------------------------------------------------------------------
def test_ac10_verify_uses_symbol_aggregate_for_match():
"""verify uses symbol-level hub aggregate (not single portfolio qty) for comparison."""
from uuid import uuid4
from trex_core.domain.slot.models import Slot
repo = InMemorySlotRepository()
ws = InMemoryReconcileWarningStore()
# Two portfolios for same symbol
slot_a = Slot(slot_id=uuid4(), status=SlotState.HOLDING, symbol=SYM_A,
qty=200.0, portfolio_id="pf-A", instance_key=INSTANCE)
slot_b = Slot(slot_id=uuid4(), status=SlotState.HOLDING, symbol=SYM_A,
qty=240.0, portfolio_id="pf-B", instance_key=INSTANCE)
repo.add_slot(slot_a)
repo.add_slot(slot_b)
# Both warned
ws.set_warned(str(slot_a.slot_id), portfolio_id="pf-A", case_code="C")
ws.set_warned(str(slot_b.slot_id), portfolio_id="pf-B", case_code="C")
store = SnapshotStore()
store.swap([
PortfolioItem(portfolio_id="pf-A", symbol=SYM_A, cash_balance=0, asset_quantity=200.0),
PortfolioItem(portfolio_id="pf-B", symbol=SYM_A, cash_balance=0, asset_quantity=240.0),
])
broker = FakeBroker([BalanceItem(symbol=SYM_A, hldg_qty=440)]) # 200+240=440 == 440 ✓
result = verify_slot_reconcile(
portfolio_id="pf-A",
store=store,
broker=broker,
slot_repo=repo,
instance_key=INSTANCE,
warn_store=ws,
)
assert result["result"] == "cleared"
assert result["hub_qty"] == 440 # aggregate, not just pf-A's 200
# ---------------------------------------------------------------------------
# AC-11: verify clears ALL C-case warnings for the symbol
# ---------------------------------------------------------------------------
def test_ac11_verify_clears_all_symbol_c_case_warnings():
from uuid import uuid4
from trex_core.domain.slot.models import Slot
repo = InMemorySlotRepository()
ws = InMemoryReconcileWarningStore()
slot_a = Slot(slot_id=uuid4(), status=SlotState.HOLDING, symbol=SYM_A,
qty=200.0, portfolio_id="pf-A", instance_key=INSTANCE)
slot_b = Slot(slot_id=uuid4(), status=SlotState.HOLDING, symbol=SYM_A,
qty=240.0, portfolio_id="pf-B", instance_key=INSTANCE)
repo.add_slot(slot_a)
repo.add_slot(slot_b)
ws.set_warned(str(slot_a.slot_id), portfolio_id="pf-A", case_code="C")
ws.set_warned(str(slot_b.slot_id), portfolio_id="pf-B", case_code="C")
store = SnapshotStore()
store.swap([
PortfolioItem(portfolio_id="pf-A", symbol=SYM_A, cash_balance=0, asset_quantity=200.0),
PortfolioItem(portfolio_id="pf-B", symbol=SYM_A, cash_balance=0, asset_quantity=240.0),
])
broker = FakeBroker([BalanceItem(symbol=SYM_A, hldg_qty=440)])
verify_slot_reconcile(
portfolio_id="pf-A",
store=store,
broker=broker,
slot_repo=repo,
instance_key=INSTANCE,
warn_store=ws,
)
# Both pf-A and pf-B warnings cleared
assert not ws.is_warned(str(slot_a.slot_id))
assert not ws.is_warned(str(slot_b.slot_id))
def test_ac11_verify_mismatch_keeps_all_warnings():
"""When verify still mismatches, all warnings are kept."""
from uuid import uuid4
from trex_core.domain.slot.models import Slot
repo = InMemorySlotRepository()
ws = InMemoryReconcileWarningStore()
slot_a = Slot(slot_id=uuid4(), status=SlotState.HOLDING, symbol=SYM_A,
qty=200.0, portfolio_id="pf-A", instance_key=INSTANCE)
slot_b = Slot(slot_id=uuid4(), status=SlotState.HOLDING, symbol=SYM_A,
qty=235.0, portfolio_id="pf-B", instance_key=INSTANCE)
repo.add_slot(slot_a)
repo.add_slot(slot_b)
ws.set_warned(str(slot_a.slot_id), portfolio_id="pf-A", case_code="C")
ws.set_warned(str(slot_b.slot_id), portfolio_id="pf-B", case_code="C")
store = SnapshotStore()
store.swap([
PortfolioItem(portfolio_id="pf-A", symbol=SYM_A, cash_balance=0, asset_quantity=200.0),
PortfolioItem(portfolio_id="pf-B", symbol=SYM_A, cash_balance=0, asset_quantity=235.0),
])
broker = FakeBroker([BalanceItem(symbol=SYM_A, hldg_qty=440)]) # 435 != 440
result = verify_slot_reconcile(
portfolio_id="pf-A",
store=store,
broker=broker,
slot_repo=repo,
instance_key=INSTANCE,
warn_store=ws,
)
assert result["result"] == "rejected"
# Both warnings still present
assert ws.is_warned(str(slot_a.slot_id))
assert ws.is_warned(str(slot_b.slot_id))
# ---------------------------------------------------------------------------
# AC-12: C-case warning NOT auto-cleared by time; only cleared on quantity match
# ---------------------------------------------------------------------------
def test_ac12_c_case_warning_not_auto_cleared():
"""C-case warning persists across multiple slots RPC calls without manual verify."""
repo = InMemorySlotRepository()
svc = SlotService(repo)
warn_store = InMemoryReconcileWarningStore()
slot = _make_holding_slot(repo, svc, portfolio_id="pf-A", symbol=SYM_A, qty=10)
warn_store.set_warned(str(slot.slot_id), portfolio_id="pf-A", case_code="C")
handler = _make_handler(repo, warn_store)
# Call slots RPC multiple times — slot stays excluded (no auto-clear)
for _ in range(3):
resp = handler(_req(filter="holding"))
assert resp.payload["slots"] == []
assert resp.payload["excluded_slots"]["reconcile_quantity_mismatch"] == 1
# Warning still present
assert warn_store.is_warned(str(slot.slot_id))
# ---------------------------------------------------------------------------
# Non-invasive: slots RPC does not mutate broker, ledger, or slot state
# ---------------------------------------------------------------------------
def test_non_invasive_slots_rpc_no_side_effects():
"""Slots RPC with C-case exclusion must not change slot status or broker state."""
repo = InMemorySlotRepository()
svc = SlotService(repo)
warn_store = InMemoryReconcileWarningStore()
slot = _make_holding_slot(repo, svc, portfolio_id="pf-A", symbol=SYM_A, qty=10)
warn_store.set_warned(str(slot.slot_id), portfolio_id="pf-A", case_code="C")
handler = _make_handler(repo, warn_store)
handler(_req())
# Slot still in HOLDING state (no transition)
refreshed = repo.fetch_slot(slot.slot_id)
assert refreshed.status == SlotState.HOLDING
assert float(refreshed.qty) == 10.0
# Warning still present (not cleared by RPC)
assert warn_store.is_warned(str(slot.slot_id))
+137
View File
@@ -265,3 +265,140 @@ def test_verify_empty_snapshot_treats_hub_qty_as_zero():
assert result["result"] == "rejected"
assert result["hub_qty"] == 0
assert result["broker_qty"] == 10
# ---------------------------------------------------------------------------
# AC-0023 comment 3853379970: B-case warning preserved; non-holding C-case cleared
# ---------------------------------------------------------------------------
def test_verify_c_case_clear_preserves_b_case_warning():
"""B-case warning for same symbol must NOT be cleared by a C-case verify clear."""
sym = "005930"
# Slot A: HOLDING, C-case warning (should be cleared)
slot_c = _make_slot(status=SlotState.HOLDING, symbol=sym, pid="pf-c")
# Slot B: HOLDING, B-case warning (must be preserved)
slot_b = _make_slot(status=SlotState.HOLDING, symbol=sym, pid="pf-b")
repo = InMemorySlotRepository()
repo.add_slot(slot_c)
repo.add_slot(slot_b)
ws = InMemoryReconcileWarningStore()
ws.set_warned(str(slot_c.slot_id), portfolio_id="pf-c", case_code="C")
ws.set_warned(str(slot_b.slot_id), portfolio_id="pf-b", case_code="B")
# Aggregate hub qty across both portfolios = 10
store = SnapshotStore()
store.swap([
PortfolioItem(portfolio_id="pf-c", symbol=sym, cash_balance=0.0, asset_quantity=5.0),
PortfolioItem(portfolio_id="pf-b", symbol=sym, cash_balance=0.0, asset_quantity=5.0),
])
broker = FakeBroker({sym: 10})
result = verify_slot_reconcile(
portfolio_id="pf-c",
store=store,
broker=broker,
slot_repo=repo,
instance_key=INSTANCE,
warn_store=ws,
)
assert result["result"] == "cleared"
# C-case slot cleared
assert not ws.is_warned(str(slot_c.slot_id))
# B-case slot untouched
assert ws.is_warned(str(slot_b.slot_id))
b_info = ws.get_warning_info(str(slot_b.slot_id))
assert b_info is not None and b_info["case_code"] == "B"
def test_verify_c_case_clear_non_holding_slot():
"""A non-HOLDING slot with C-case warning for the same symbol must also be cleared."""
sym = "005930"
slot_holding = _make_slot(status=SlotState.HOLDING, symbol=sym, pid="pf-hold")
# Simulate a slot that is INACTIVE (e.g. after sell filled) but still has a C-case warning
slot_inactive = _make_slot(status=SlotState.INACTIVE, symbol=sym, pid="pf-inact")
repo = InMemorySlotRepository()
repo.add_slot(slot_holding)
repo.add_slot(slot_inactive)
ws = InMemoryReconcileWarningStore()
ws.set_warned(str(slot_holding.slot_id), portfolio_id="pf-hold", case_code="C")
ws.set_warned(str(slot_inactive.slot_id), portfolio_id="pf-inact", case_code="C")
store = SnapshotStore()
store.swap([
PortfolioItem(portfolio_id="pf-hold", symbol=sym, cash_balance=0.0, asset_quantity=10.0),
])
broker = FakeBroker({sym: 10})
result = verify_slot_reconcile(
portfolio_id="pf-hold",
store=store,
broker=broker,
slot_repo=repo,
instance_key=INSTANCE,
warn_store=ws,
)
assert result["result"] == "cleared"
assert not ws.is_warned(str(slot_holding.slot_id))
assert not ws.is_warned(str(slot_inactive.slot_id))
# ---------------------------------------------------------------------------
# AC-0023 comment 3853379979: partial clear failure → clear_failed, not cleared
# ---------------------------------------------------------------------------
class _FailAfterNStore(InMemoryReconcileWarningStore):
"""Raises on the Nth+ call to clear_warned (1-indexed)."""
def __init__(self, fail_after: int = 1) -> None:
super().__init__()
self._calls = 0
self._fail_after = fail_after
def clear_warned(self, slot_id: str) -> None:
self._calls += 1
if self._calls > self._fail_after:
raise RuntimeError("simulated DB error")
super().clear_warned(slot_id)
def test_verify_partial_clear_failure_returns_clear_failed():
"""If clear_warned raises mid-loop, result must be 'clear_failed', not 'cleared'."""
sym = "005930"
slot1 = _make_slot(status=SlotState.HOLDING, symbol=sym, pid="pf-1")
slot2 = _make_slot(status=SlotState.HOLDING, symbol=sym, pid="pf-2")
repo = InMemorySlotRepository()
repo.add_slot(slot1)
repo.add_slot(slot2)
# Fails on 2nd clear_warned call
ws = _FailAfterNStore(fail_after=1)
ws.set_warned(str(slot1.slot_id), portfolio_id="pf-1", case_code="C")
ws.set_warned(str(slot2.slot_id), portfolio_id="pf-2", case_code="C")
store = SnapshotStore()
store.swap([
PortfolioItem(portfolio_id="pf-1", symbol=sym, cash_balance=0.0, asset_quantity=10.0),
PortfolioItem(portfolio_id="pf-2", symbol=sym, cash_balance=0.0, asset_quantity=0.0),
])
broker = FakeBroker({sym: 10})
result = verify_slot_reconcile(
portfolio_id="pf-1",
store=store,
broker=broker,
slot_repo=repo,
instance_key=INSTANCE,
warn_store=ws,
)
assert result["result"] == "clear_failed"
assert result["hub_qty"] == 10
assert result["broker_qty"] == 10
assert result["reason"] is not None and "warn_store" in result["reason"]
+2
View File
@@ -18,6 +18,7 @@ from trex_core.adapters.broker.kis import KISBroker
_PG_TABLES = (
"broker_account_snapshot",
"ledger_skip",
"killswitch_exclusion",
"snapshot_pause",
"ledger_outbox",
@@ -25,6 +26,7 @@ _PG_TABLES = (
"slot_transition_log",
"daily_symbol_entries",
"daily_slot",
"reconcile_notification_state",
)
+328
View File
@@ -0,0 +1,328 @@
from __future__ import annotations
import json
from datetime import datetime
from types import SimpleNamespace
from uuid import UUID
from trex_core.application.ledger_outbox import OutboxOutcome
from trex_core.application.ledger_port import LedgerTrade
from trex_core.application.ledger_skip import LedgerSkipRecord
from trex_core.runtime import cli
def _cfg(*, live: bool = False):
return SimpleNamespace(
dsn="postgresql://x/y",
broker="kis",
kis=SimpleNamespace(allow_live=live),
portfolio_hub=SimpleNamespace(base_url="http://pf.local", token="tok", timeout=5.0),
)
def _args(tmp_path, **overrides):
data = {
"date": "2026-08-24",
"execution_id": "kis-daily-ccld:20260824:12345678:0001234567:10",
"dry_run": False,
"yes": True,
"live": True,
"json": True,
"audit_dir": str(tmp_path),
"instance": "inst-1",
"config": None,
"config_path": None,
"env_file": None,
"timeout": 10.0,
"ledger_cmd": "backfill",
"cmd": "ledger",
}
data.update(overrides)
return SimpleNamespace(**data)
def _evidence():
return {
"symbol": "005930",
"side": "sell",
"qty": 10.0,
"price": 70000.0,
"fee": 105.0,
"tax": 1400.0,
"order_id": "12345678:0001234567",
"order_venue": "SOR",
"execution_venue": None,
"executed_at": "2026-08-24T00:01:02+00:00",
"raw": {
"pdno": "005930",
"sll_buy_dvsn_cd": "01",
"tot_ccld_qty": "10",
"avg_prvs": "70000",
"ord_dt": "20260824",
"ord_gno_brno": "12345678",
"odno": "0001234567",
"excg_id_dvsn_cd": "SOR",
},
}
def _skip_record():
return LedgerSkipRecord(
skip_key="ledger-skip:inst-1:slot-1:sell:12345678:0001234567:kis-daily-ccld:20260824:12345678:0001234567:10:missing_execution_id",
instance_key="inst-1",
slot_id=UUID(_SLOT_ID),
portfolio_id="PF-1",
symbol="005930",
order_id="12345678:0001234567",
execution_id="kis-daily-ccld:20260824:12345678:0001234567:10",
side="sell",
qty=10.0,
price=70000.0,
executed_at=datetime.fromisoformat("2026-08-24T00:01:02+00:00"),
order_venue="SOR",
execution_venue="unknown",
skip_reason="missing_execution_id",
status="resolved",
resolution_reason="manual_ledger_backfill",
resolved_at=datetime.fromisoformat("2026-08-24T00:03:02+00:00"),
created_at=datetime.fromisoformat("2026-08-24T00:01:02+00:00"),
updated_at=datetime.fromisoformat("2026-08-24T00:03:02+00:00"),
)
_SLOT_ID = "00000000-0000-0000-0000-000000000001"
def test_ledger_backfill_parser_accepts_required_flags():
parser = cli._build_parser()
args = parser.parse_args([
"ledger", "backfill",
"--date", "2026-08-24",
"--execution-id", "kis-daily-ccld:20260824:12345678:0001234567:10",
"--dry-run",
])
assert args.cmd == "ledger"
assert args.ledger_cmd == "backfill"
assert args.dry_run is True
def test_ledger_backfill_dry_run_emits_trade_and_writes_audit(monkeypatch, tmp_path, capsys):
monkeypatch.setattr(cli, "_lookup_ledger_backfill_evidence",
lambda cfg, **kwargs: (_evidence(), SimpleNamespace(mode="env", identity_label="kis-config")))
monkeypatch.setattr(cli, "_resolve_ledger_backfill_slot",
lambda dsn, **kwargs: {"slot_id": _SLOT_ID, "portfolio_id": "PF-1", "status": "inactive", "symbol": "005930"})
monkeypatch.setattr(cli, "_lookup_outbox_by_execution_id", lambda dsn, **kwargs: [])
rc = cli._cmd_ledger_backfill(_args(tmp_path, dry_run=True), _cfg(), lambda _: True)
assert rc == 0
out = json.loads(capsys.readouterr().out)
assert out["result"]["status"] == "dry_run"
assert out["trade"]["correlation_id"] == "PF-1:12345678:0001234567:kis-daily-ccld:20260824:12345678:0001234567:10"
assert out["trade"]["order_venue"] == "SOR"
assert out["trade"]["execution_venue"] is None
audit = json.loads(next(tmp_path.glob("*.json")).read_text(encoding="utf-8"))
assert audit["request"]["execution_id"] == _args(tmp_path).execution_id
def test_ledger_backfill_live_requires_live_flag(monkeypatch, tmp_path, capsys):
monkeypatch.setattr(cli, "_lookup_ledger_backfill_evidence",
lambda cfg, **kwargs: (_evidence(), SimpleNamespace(mode="env", identity_label="kis-config")))
monkeypatch.setattr(cli, "_resolve_ledger_backfill_slot",
lambda dsn, **kwargs: {"slot_id": _SLOT_ID, "portfolio_id": "PF-1", "status": "inactive", "symbol": "005930"})
monkeypatch.setattr(cli, "_lookup_outbox_by_execution_id", lambda dsn, **kwargs: [])
rc = cli._cmd_ledger_backfill(_args(tmp_path, live=False), _cfg(live=True), lambda _: True)
assert rc == 2
err = capsys.readouterr().err
assert "--live" in err
audit = json.loads(next(tmp_path.glob("*.json")).read_text(encoding="utf-8"))
assert audit["result"]["status"] == "rejected"
def test_ledger_backfill_existing_completed_row_is_idempotent(monkeypatch, tmp_path, capsys):
monkeypatch.setattr(cli, "_lookup_ledger_backfill_evidence",
lambda cfg, **kwargs: (_evidence(), SimpleNamespace(mode="env", identity_label="kis-config")))
monkeypatch.setattr(cli, "_resolve_ledger_backfill_slot",
lambda dsn, **kwargs: {"slot_id": _SLOT_ID, "portfolio_id": "PF-1", "status": "inactive", "symbol": "005930"})
existing = [{
"correlation_id": "PF-1:12345678:0001234567:kis-daily-ccld:20260824:12345678:0001234567:10",
"asset_id": "PF-1",
"order_id": "12345678:0001234567",
"execution_id": "kis-daily-ccld:20260824:12345678:0001234567:10",
"status": "completed",
"attempts": 1,
"last_outcome": "completed",
"last_error": None,
}]
monkeypatch.setattr(cli, "_lookup_outbox_by_execution_id", lambda dsn, **kwargs: existing)
class _Outbox:
def enqueue(self, trade):
raise AssertionError("completed execution must not be re-enqueued")
def get(self, correlation_id):
return SimpleNamespace(
status=SimpleNamespace(value="completed"),
attempts=1,
last_outcome=SimpleNamespace(value="completed"),
last_error=None,
)
class _Writer:
def __init__(self, *_args, **_kwargs):
pass
def process_correlation_id(self, correlation_id):
raise AssertionError("completed execution must not be re-processed")
class _Store:
def __init__(self, *_args, **_kwargs):
pass
def resolve_for_fill(self, **_kwargs):
return []
import trex_core.adapters.ledger.portfolio_hub as hub_mod
import trex_core.adapters.persistence.postgresql as pg_mod
monkeypatch.setattr(pg_mod, "PostgreSQLLedgerOutbox", lambda dsn: _Outbox())
monkeypatch.setattr(hub_mod, "PortfolioHubLedger", lambda *args, **kwargs: object())
monkeypatch.setattr(cli, "LedgerWriter", _Writer)
monkeypatch.setattr(
"trex_core.adapters.persistence.ledger_skip_store.PostgreSQLLedgerSkipStore",
_Store,
)
rc = cli._cmd_ledger_backfill(_args(tmp_path), _cfg(), lambda _: True)
assert rc == 0
out = json.loads(capsys.readouterr().out)
assert out["result"]["status"] == "already_recorded"
def test_ledger_backfill_live_enqueue_and_processes_exact_trade(monkeypatch, tmp_path, capsys):
monkeypatch.setattr(cli, "_lookup_ledger_backfill_evidence",
lambda cfg, **kwargs: (_evidence(), SimpleNamespace(mode="env", identity_label="kis-config")))
monkeypatch.setattr(cli, "_resolve_ledger_backfill_slot",
lambda dsn, **kwargs: {"slot_id": _SLOT_ID, "portfolio_id": "PF-1", "status": "inactive", "symbol": "005930"})
monkeypatch.setattr(cli, "_lookup_outbox_by_execution_id", lambda dsn, **kwargs: [])
class _Outbox:
def __init__(self):
self.trade = None
def enqueue(self, trade: LedgerTrade):
self.trade = trade
def get(self, correlation_id):
assert self.trade is not None
return SimpleNamespace(
status=SimpleNamespace(value="completed"),
attempts=1,
last_outcome=SimpleNamespace(value="completed"),
last_error=None,
)
outbox = _Outbox()
class _Writer:
def __init__(self, seen_outbox, *_args, **_kwargs):
self.seen_outbox = seen_outbox
def process_correlation_id(self, correlation_id):
assert self.seen_outbox.trade is not None
assert correlation_id == self.seen_outbox.trade.correlation_id
return OutboxOutcome.COMPLETED
class _Store:
def __init__(self, *_args, **_kwargs):
pass
def resolve_for_fill(self, **_kwargs):
return []
import trex_core.adapters.ledger.portfolio_hub as hub_mod
import trex_core.adapters.persistence.postgresql as pg_mod
monkeypatch.setattr(pg_mod, "PostgreSQLLedgerOutbox", lambda dsn: outbox)
monkeypatch.setattr(hub_mod, "PortfolioHubLedger", lambda *args, **kwargs: object())
monkeypatch.setattr(cli, "LedgerWriter", _Writer)
monkeypatch.setattr(
"trex_core.adapters.persistence.ledger_skip_store.PostgreSQLLedgerSkipStore",
_Store,
)
rc = cli._cmd_ledger_backfill(_args(tmp_path), _cfg(), lambda _: True)
assert rc == 0
out = json.loads(capsys.readouterr().out)
assert out["result"]["status"] == "completed"
assert outbox.trade is not None
assert outbox.trade.order_venue == "SOR"
assert outbox.trade.venue is None
def test_ledger_backfill_resolves_matching_skip(monkeypatch, tmp_path, capsys):
monkeypatch.setattr(cli, "_lookup_ledger_backfill_evidence",
lambda cfg, **kwargs: (_evidence(), SimpleNamespace(mode="env", identity_label="kis-config")))
monkeypatch.setattr(cli, "_resolve_ledger_backfill_slot",
lambda dsn, **kwargs: {"slot_id": _SLOT_ID, "portfolio_id": "PF-1", "status": "inactive", "symbol": "005930"})
monkeypatch.setattr(cli, "_lookup_outbox_by_execution_id", lambda dsn, **kwargs: [])
class _Outbox:
def enqueue(self, trade: LedgerTrade):
self.trade = trade
def get(self, correlation_id):
return SimpleNamespace(
status=SimpleNamespace(value="completed"),
attempts=1,
last_outcome=SimpleNamespace(value="completed"),
last_error=None,
)
class _Writer:
def __init__(self, *_args, **_kwargs):
pass
def process_correlation_id(self, _correlation_id):
return OutboxOutcome.COMPLETED
class _Store:
def __init__(self, *_args, **_kwargs):
pass
def resolve_for_fill(self, **_kwargs):
return [_skip_record()]
published = []
class _Emitter:
def publish_alarm(self, code, title, *, fields=None, severity="critical"):
published.append((code, severity, fields))
class _Transport:
def close(self):
return None
import trex_core.adapters.ledger.portfolio_hub as hub_mod
import trex_core.adapters.persistence.postgresql as pg_mod
monkeypatch.setattr(pg_mod, "PostgreSQLLedgerOutbox", lambda dsn: _Outbox())
monkeypatch.setattr(hub_mod, "PortfolioHubLedger", lambda *args, **kwargs: object())
monkeypatch.setattr(cli, "LedgerWriter", _Writer)
monkeypatch.setattr(
"trex_core.adapters.persistence.ledger_skip_store.PostgreSQLLedgerSkipStore",
_Store,
)
monkeypatch.setattr(cli, "_default_alarm_emitter", lambda cfg: (_Emitter(), _Transport()))
rc = cli._cmd_ledger_backfill(_args(tmp_path), _cfg(), lambda _: True)
assert rc == 0
out = json.loads(capsys.readouterr().out)
assert len(out["resolved_skips"]) == 1
assert out["resolved_skips"][0]["resolution_reason"] == "manual_ledger_backfill"
assert published[0][0] == "ledger_skip_resolved"
+245
View File
@@ -0,0 +1,245 @@
from __future__ import annotations
import json
from datetime import UTC, datetime
from types import SimpleNamespace
from uuid import UUID
from trex_core.application.ledger_skip import LedgerSkipRecord
from trex_core.runtime import cli
def _cfg(*, live: bool = False):
return SimpleNamespace(
dsn="postgresql://x/y",
rabbitmq=None,
broker="kis",
kis=SimpleNamespace(allow_live=live),
)
def _record(*, status="unresolved", resolution_reason=None):
return LedgerSkipRecord(
skip_key="ledger-skip:inst-1:slot-1:sell:O-1:E-1:missing_execution_id",
instance_key="inst-1",
slot_id=UUID("00000000-0000-0000-0000-000000000001"),
portfolio_id="PF-1",
symbol="005930",
order_id="O-1",
execution_id="E-1",
side="sell",
qty=10.0,
price=70000.0,
executed_at=datetime(2026, 8, 24, 0, 1, 2, tzinfo=UTC),
order_venue="SOR",
execution_venue="unknown",
skip_reason="missing_execution_id",
status=status,
resolution_reason=resolution_reason,
resolved_at=(
datetime(2026, 8, 24, 0, 5, 0, tzinfo=UTC)
if status == "resolved"
else None
),
created_at=datetime(2026, 8, 24, 0, 1, 2, tzinfo=UTC),
updated_at=datetime(2026, 8, 24, 0, 5, 0, tzinfo=UTC),
)
def test_ledger_skips_parser_accepts_list_and_resolve():
parser = cli._build_parser()
args = parser.parse_args(["ledger", "skips", "list", "--status", "all"])
assert args.ledger_cmd == "skips"
assert args.ledger_skips_cmd == "list"
args = parser.parse_args(["ledger", "skips", "resolve", "--skip-key", "abc", "--yes"])
assert args.ledger_skips_cmd == "resolve"
def test_ledger_skips_list_outputs_records(monkeypatch, capsys):
class _Store:
def __init__(self, *_args, **_kwargs):
pass
def list_records(self, **_kwargs):
return [_record()]
monkeypatch.setattr(
"trex_core.adapters.persistence.ledger_skip_store.PostgreSQLLedgerSkipStore",
_Store,
)
args = SimpleNamespace(
cmd="ledger",
ledger_cmd="skips",
ledger_skips_cmd="list",
date="2026-08-24",
portfolio_id="PF-1",
status="unresolved",
limit=10,
instance="inst-1",
config=None,
config_path=None,
env_file=None,
json=True,
timeout=10.0,
)
rc = cli._cmd_ledger_skips_list(args, _cfg())
assert rc == 0
out = json.loads(capsys.readouterr().out)
assert out["count"] == 1
assert out["records"][0]["skip_reason"] == "missing_execution_id"
def test_ledger_skips_resolve_marks_record_and_emits_alert(monkeypatch, tmp_path, capsys):
class _Store:
def __init__(self, *_args, **_kwargs):
pass
def resolve_by_key(self, skip_key, *, instance_key, resolution_reason):
assert skip_key == "abc"
assert instance_key == "inst-1"
return _record(status="resolved", resolution_reason=resolution_reason)
published = []
class _Emitter:
def publish_alarm(self, code, title, *, fields=None, severity="critical"):
published.append((code, severity, fields))
class _Transport:
def close(self):
return None
monkeypatch.setattr(
"trex_core.adapters.persistence.ledger_skip_store.PostgreSQLLedgerSkipStore",
_Store,
)
monkeypatch.setattr(cli, "_default_alarm_emitter", lambda cfg: (_Emitter(), _Transport()))
args = SimpleNamespace(
cmd="ledger",
ledger_cmd="skips",
ledger_skips_cmd="resolve",
skip_key="abc",
reason="operator_reconciled",
instance="inst-1",
yes=True,
live=False,
audit_dir=str(tmp_path),
config=None,
config_path=None,
env_file=None,
json=True,
timeout=10.0,
)
rc = cli._cmd_ledger_skips_resolve(args, _cfg(), lambda _: True)
assert rc == 0
out = json.loads(capsys.readouterr().out)
assert out["result"] == "resolved"
assert out["audit_path"].endswith(".json")
assert published[0][0] == "ledger_skip_resolved"
def test_ledger_skips_resolve_live_requires_live_flag(monkeypatch, tmp_path, capsys):
monkeypatch.setattr(
"trex_core.adapters.persistence.ledger_skip_store.PostgreSQLLedgerSkipStore",
lambda *_args, **_kwargs: object(),
)
args = SimpleNamespace(
cmd="ledger",
ledger_cmd="skips",
ledger_skips_cmd="resolve",
skip_key="abc",
reason="operator_reconciled",
instance="inst-1",
yes=True,
live=False,
audit_dir=str(tmp_path),
config=None,
config_path=None,
env_file=None,
json=True,
timeout=10.0,
)
rc = cli._cmd_ledger_skips_resolve(args, _cfg(live=True), lambda _: True)
assert rc == 2
err = capsys.readouterr().err
assert "--live" in err
audit = json.loads(next(tmp_path.glob("*.json")).read_text(encoding="utf-8"))
assert audit["request"]["instance"] == "inst-1"
assert audit["result"]["status"] == "rejected"
assert audit["audit_path"].endswith(".json")
def test_ledger_skips_resolve_live_requires_yes_and_live(monkeypatch, tmp_path, capsys):
monkeypatch.setattr(
"trex_core.adapters.persistence.ledger_skip_store.PostgreSQLLedgerSkipStore",
lambda *_args, **_kwargs: object(),
)
args = SimpleNamespace(
cmd="ledger",
ledger_cmd="skips",
ledger_skips_cmd="resolve",
skip_key="abc",
reason="operator_reconciled",
instance="inst-1",
yes=False,
live=True,
audit_dir=str(tmp_path),
config=None,
config_path=None,
env_file=None,
json=True,
timeout=10.0,
)
rc = cli._cmd_ledger_skips_resolve(args, _cfg(live=True), lambda _: True)
assert rc == 2
err = capsys.readouterr().err
assert "--yes --live" in err
audit = json.loads(next(tmp_path.glob("*.json")).read_text(encoding="utf-8"))
assert audit["result"]["reason"] == "live account requires --yes --live"
def test_ledger_skips_resolve_not_found_emits_audit_report(monkeypatch, tmp_path, capsys):
class _Store:
def __init__(self, *_args, **_kwargs):
pass
def resolve_by_key(self, *_args, **_kwargs):
return None
monkeypatch.setattr(
"trex_core.adapters.persistence.ledger_skip_store.PostgreSQLLedgerSkipStore",
_Store,
)
args = SimpleNamespace(
cmd="ledger",
ledger_cmd="skips",
ledger_skips_cmd="resolve",
skip_key="missing",
reason="operator_reconciled",
instance="inst-1",
yes=True,
live=False,
audit_dir=str(tmp_path),
config=None,
config_path=None,
env_file=None,
json=False,
timeout=10.0,
)
rc = cli._cmd_ledger_skips_resolve(args, _cfg(), lambda _: True)
assert rc == 1
out = json.loads(capsys.readouterr().out)
assert out["result"]["status"] == "not_found"
assert out["audit_path"].endswith(".json")
audit = json.loads(next(tmp_path.glob("*.json")).read_text(encoding="utf-8"))
assert audit["audit_path"].endswith(".json")
@@ -7,6 +7,9 @@ daemon wiring was never run until the REL-004 live smoke timed out every buy.
"""
from __future__ import annotations
import ast
from pathlib import Path
from trex_core.application.portfolio import PortfolioItem, SnapshotStore
from trex_core.runtime.daemon import Daemon
@@ -35,3 +38,38 @@ def test_resolve_symbol_reads_item_symbol():
def test_resolve_symbol_unknown_returns_none():
holder = _StoreHolder(_store())
assert Daemon._resolve_symbol(holder, "pf-missing") is None
def test_daemon_wires_shared_alarm_emitter_into_intent_service():
src = (
Path(__file__).resolve().parents[2]
/ "trex_core"
/ "runtime"
/ "daemon.py"
).read_text(encoding="utf-8")
tree = ast.parse(src)
found = False
for node in ast.walk(tree):
if not isinstance(node, ast.Assign):
continue
if not any(isinstance(t, ast.Attribute) and t.attr == "intent_service" for t in node.targets):
continue
call = node.value
if not (
isinstance(call, ast.Call)
and isinstance(call.func, ast.Name)
and call.func.id == "IntentService"
):
continue
for kw in call.keywords:
if kw.arg != "alarm_emitter":
continue
if (
isinstance(kw.value, ast.Attribute)
and kw.value.attr == "alarm_emitter"
and isinstance(kw.value.value, ast.Name)
and kw.value.value.id == "self"
):
found = True
break
assert found, "Daemon must pass self.alarm_emitter into IntentService(alarm_emitter=...)"
+143 -17
View File
@@ -14,11 +14,11 @@ from __future__ import annotations
import base64
import hashlib
import json
from collections.abc import Callable
from dataclasses import replace
from datetime import datetime, timedelta, timezone
from datetime import UTC, datetime, timedelta, timezone
from decimal import Decimal, InvalidOperation
from time import monotonic
from typing import Callable, Optional
import httpx
@@ -43,7 +43,11 @@ _SUBMISSION_LOOKUP_GRACE_SECONDS = 90.0
_SUPPORTED_ORDER_VENUES = frozenset({"KRX", "NXT", "SOR"})
def _balance_http_error_code(status: Optional[int]) -> str:
class KISEvidenceLookupError(RuntimeError):
"""Read-only daily-fill evidence lookup failed."""
def _balance_http_error_code(status: int | None) -> str:
if status == 401:
return "authentication_failed"
if status == 403:
@@ -65,7 +69,7 @@ def resolve_order_venue(venue: str, at: datetime) -> str:
return normalized
def _safe_int(v: object) -> Optional[int]:
def _safe_int(v: object) -> int | None:
try:
return int(str(v).strip())
except (TypeError, ValueError):
@@ -88,7 +92,7 @@ def _encode_submission_unknown(req: OrderRequest, submitted_at: datetime) -> str
return f"{_SUBMISSION_UNKNOWN_PREFIX}{encoded}"
def _decode_submission_unknown(broker_order_id: str) -> Optional[dict]:
def _decode_submission_unknown(broker_order_id: str) -> dict | None:
if not broker_order_id.startswith(_SUBMISSION_UNKNOWN_PREFIX):
return None
encoded = broker_order_id[len(_SUBMISSION_UNKNOWN_PREFIX) :]
@@ -134,15 +138,15 @@ class KISBroker:
timeout: float = 5.0,
allow_live: bool = False,
token_cache=None,
clock: Optional[Callable[[], datetime]] = None,
clock: Callable[[], datetime] | None = None,
) -> None:
self._account_ref = account_ref
self._app_key = app_key
self._app_secret = app_secret
self._base_url = base_url.rstrip("/")
self._timeout = timeout
self._token: Optional[str] = None
self._token_exp: datetime = datetime.now(tz=timezone.utc)
self._token: str | None = None
self._token_exp: datetime = datetime.now(tz=UTC)
self._token_cache = token_cache # optional TokenCache (L2, REQ-016)
self._clock = clock or (lambda: datetime.now(tz=_KST))
self._fill_poll_not_before: dict[str, float] = {}
@@ -193,12 +197,12 @@ class KISBroker:
self._app_secret = credential.app_secret
# Invalidate L1 in-memory token so next call re-issues.
self._token = None
self._token_exp = datetime.min.replace(tzinfo=timezone.utc)
self._token_exp = datetime.min.replace(tzinfo=UTC)
# --- auth ---
def _ensure_token(self) -> str:
# L1: in-memory
if self._token and datetime.now(tz=timezone.utc) < self._token_exp:
if self._token and datetime.now(tz=UTC) < self._token_exp:
return self._token
# L2: shared/persistent cache (Redis) — survives restart / multi-process (REQ-016)
if self._token_cache is not None:
@@ -220,7 +224,7 @@ class KISBroker:
data = resp.json()
self._token = data["access_token"]
ttl = int(data.get("expires_in", 3600))
self._token_exp = datetime.now(tz=timezone.utc) + timedelta(seconds=max(60, ttl - 300))
self._token_exp = datetime.now(tz=UTC) + timedelta(seconds=max(60, ttl - 300))
if self._token_cache is not None:
self._token_cache.store(self._account_ref, self._app_key, self._token, self._token_exp)
return self._token
@@ -360,8 +364,8 @@ class KISBroker:
*,
side: str,
symbol: str,
order_date: Optional[str] = None,
submitted_at: Optional[datetime] = None,
order_date: str | None = None,
submitted_at: datetime | None = None,
) -> FillStatus:
if broker_order_id.startswith(_SUBMISSION_UNKNOWN_PREFIX):
return self._poll_unknown_submission(broker_order_id, side=side, symbol=symbol)
@@ -420,7 +424,7 @@ class KISBroker:
if row is None:
visible_grace = (
submitted_at is None
or (self._clock().astimezone(timezone.utc) - submitted_at.astimezone(timezone.utc))
or (self._clock().astimezone(UTC) - submitted_at.astimezone(UTC))
.total_seconds()
< _SUBMISSION_LOOKUP_GRACE_SECONDS
)
@@ -594,6 +598,126 @@ class KISBroker:
# SOR is routing provenance, not an execution exchange.
return None
@staticmethod
def _order_venue_from_row(row: dict) -> str | None:
"""Parse original routing venue (KRX/NXT/SOR) without time inference."""
for key in ("excg_id_dvsn_cd", "excg_id_dvsn_Cd", "EXCG_ID_DVSN_CD", "order_venue"):
value = str(row.get(key) or "").strip().upper()
if value in _SUPPORTED_ORDER_VENUES:
return value
if value in {"SOR-KRX", "SOR-NXT"}:
return "SOR"
return None
@staticmethod
def _side_from_row(row: dict) -> str | None:
code = str(row.get("sll_buy_dvsn_cd") or row.get("SLL_BUY_DVSN_CD") or "").strip()
if code == "01":
return "sell"
if code == "02":
return "buy"
return None
@staticmethod
def _parse_daily_ccld_execution_id(execution_id: str) -> dict[str, str]:
parts = str(execution_id or "").split(":")
if len(parts) != 5 or parts[0] != "kis-daily-ccld":
raise KISEvidenceLookupError("unsupported execution_id format")
order_date, orgno, odno, qty = parts[1], parts[2], parts[3], parts[4]
if len(order_date) != 8 or not order_date.isdigit():
raise KISEvidenceLookupError("execution_id missing order date")
if not odno or not qty.isdigit() or int(qty) <= 0:
raise KISEvidenceLookupError("execution_id missing order number or qty")
return {
"order_date": order_date,
"orgno": orgno,
"odno": odno,
"qty": qty,
}
@staticmethod
def _executed_at_from_row(row: dict, fallback_date: str) -> datetime:
order_date = str(row.get("ord_dt") or fallback_date).strip()
order_time = str(row.get("ord_tmd") or "").strip()[:6]
year = int(order_date[0:4])
month = int(order_date[4:6])
day = int(order_date[6:8])
if len(order_time) == 6 and order_time.isdigit():
executed_at = datetime(
year,
month,
day,
int(order_time[0:2]),
int(order_time[2:4]),
int(order_time[4:6]),
tzinfo=_KST,
)
return executed_at.astimezone(UTC)
return datetime(year, month, day, tzinfo=_KST).astimezone(UTC)
def lookup_fill_evidence(self, *, trading_date: str, execution_id: str) -> dict:
"""Read one exact filled/partial-cancel evidence row for a historical execution."""
parsed = self._parse_daily_ccld_execution_id(execution_id)
if trading_date != parsed["order_date"]:
raise KISEvidenceLookupError("execution_id date does not match --date")
tr_id, params = self._daily_fill_request(parsed["order_date"], order_number=parsed["odno"])
try:
resp = httpx.get(
f"{self._base_url}/uapi/domestic-stock/v1/trading/inquire-daily-ccld",
headers=self._headers(tr_id),
params=params,
timeout=self._timeout,
)
payload = resp.json()
except (httpx.HTTPError, ValueError) as exc:
raise KISEvidenceLookupError(
f"daily fill evidence lookup failed: {exc.__class__.__name__}"
) from exc
if getattr(resp, "status_code", 200) >= 400 or str(payload.get("rt_cd")) != "0":
raise KISEvidenceLookupError("daily fill evidence lookup rejected")
matches: list[dict] = []
normalized_odno = parsed["odno"].lstrip("0") or "0"
for row in payload.get("output1") or []:
row_odno = str(row.get("odno") or "").strip().lstrip("0") or "0"
if row_odno != normalized_odno:
continue
side = self._side_from_row(row)
if side is None:
continue
status = self._fill_status_from_row(
row, side=side, today=parsed["order_date"], fallback_odno=parsed["odno"]
)
if status.execution_id != execution_id:
continue
order_venue = self._order_venue_from_row(row)
if order_venue is None:
continue
orgno = str(row.get("ord_gno_brno") or "").strip()
odno = str(row.get("odno") or parsed["odno"]).strip()
symbol = str(row.get("pdno") or "").strip()
if not symbol:
continue
matches.append(
{
"symbol": symbol.zfill(6),
"side": side,
"qty": float(status.filled_qty),
"price": float(status.avg_price or 0.0),
"fee": float(status.fee),
"tax": float(status.tax),
"order_date": parsed["order_date"],
"execution_id": execution_id,
"order_id": f"{orgno}:{odno}" if orgno else odno,
"order_venue": order_venue,
"execution_venue": status.execution_venue,
"executed_at": self._executed_at_from_row(row, parsed["order_date"]),
"raw": dict(row),
}
)
if len(matches) != 1:
raise KISEvidenceLookupError("daily fill evidence not found or ambiguous")
return matches[0]
@staticmethod
def _fill_status_from_row(
row: dict,
@@ -623,7 +747,7 @@ class KISBroker:
if cancelled:
return FillStatus(state="cancelled", raw=row) # full external cancel
return FillStatus(state="accepted", raw=row)
if ccld_qty >= ord_qty and ord_qty > 0:
if ccld_qty >= ord_qty > 0:
# Deterministic execution_id (dcy-004 OQ-3, orbit kis.py rule):
# kis-daily-ccld:{ord_dt}:{ord_gno_brno}:{odno}:{tot_ccld_qty}
ord_dt = str(row.get("ord_dt") or today)
@@ -646,6 +770,7 @@ class KISBroker:
fee=fee,
tax=tax,
execution_venue=execution_venue,
executed_at=KISBroker._executed_at_from_row(row, ord_dt),
raw=row,
)
# 0 < ccld_qty < ord_qty
@@ -670,6 +795,7 @@ class KISBroker:
fee=fee,
tax=tax,
execution_venue=execution_venue,
executed_at=KISBroker._executed_at_from_row(row, ord_dt),
cancelled=True,
raw=row,
)
@@ -713,7 +839,7 @@ class KISBroker:
quantities: dict[str, int] = {}
seen_continuations: set[tuple[str, str]] = set()
continuation: Optional[tuple[str, str]] = None
continuation: tuple[str, str] | None = None
while True:
request_headers = headers
if continuation is not None:
@@ -830,7 +956,7 @@ class KISBroker:
raise BrokerBalanceError("malformed_response")
return amount
def get_current_price(self, symbol: str, market: str = "J") -> Optional[Decimal]:
def get_current_price(self, symbol: str, market: str = "J") -> Decimal | None:
"""Current traded price `stck_prpr` via KIS inquire-price (REQ-038, REQ-118).
tr_id: `FHKST01010100` per REQ-038 AC-7 (live and mock share the same tr_id).
+8 -8
View File
@@ -8,9 +8,8 @@ execution code path that production (KIS) takes.
from __future__ import annotations
import threading
from datetime import UTC, datetime
from decimal import Decimal
from datetime import datetime
from typing import Optional
from trex_core.adapters.broker.seq_store import SeqStore
from trex_core.application.broker_port import (
@@ -52,9 +51,9 @@ class PaperBroker:
self,
*,
fill_after_polls: int = 0,
qty_per_poll: Optional[int] = None,
reject_symbols: Optional[set[str]] = None,
seq_store: Optional[SeqStore] = None,
qty_per_poll: int | None = None,
reject_symbols: set[str] | None = None,
seq_store: SeqStore | None = None,
) -> None:
self._fill_after = fill_after_polls
self._qty_per_poll = qty_per_poll
@@ -90,8 +89,8 @@ class PaperBroker:
*,
side: str,
symbol: str,
order_date: Optional[str] = None,
submitted_at: Optional[datetime] = None,
order_date: str | None = None,
submitted_at: datetime | None = None,
) -> FillStatus:
with self._lock:
o = self._orders.get(broker_order_id)
@@ -122,6 +121,7 @@ class PaperBroker:
state="filled", filled_qty=o.filled, avg_price=o.price,
execution_id=execution_id, fee=fee, tax=tax,
execution_venue=o.execution_venue,
executed_at=datetime.now(tz=UTC),
)
return FillStatus(
state="partially_filled",
@@ -172,7 +172,7 @@ class PaperBroker:
with self._lock:
self._prices[symbol] = Decimal(str(price))
def get_current_price(self, symbol: str, market: str = "J") -> Optional[Decimal]:
def get_current_price(self, symbol: str, market: str = "J") -> Decimal | None:
"""Return the programmed current price for symbol, or None if unset.
market accepted for BrokerPort protocol conformance (REQ-118) but
@@ -0,0 +1,303 @@
from __future__ import annotations
from datetime import UTC, date, datetime, time, timedelta
from typing import Any
from uuid import UUID
from trex_core.adapters.persistence.postgresql import _require_psycopg
from trex_core.application.ledger_skip import LedgerSkipRecord, make_ledger_skip_key
class PostgreSQLLedgerSkipStore:
def __init__(self, dsn: str, *, psycopg_module: Any = None) -> None:
self._psycopg = _require_psycopg(psycopg_module)
self._dsn = dsn
def record_skip(
self,
*,
instance_key: str,
slot_id: UUID,
portfolio_id: str | None,
symbol: str | None,
order_id: str | None,
execution_id: str | None,
side: str,
qty: float,
price: float,
executed_at: datetime,
order_venue: str | None,
execution_venue: str | None,
skip_reason: str,
) -> tuple[LedgerSkipRecord, bool]:
skip_key = make_ledger_skip_key(
instance_key=instance_key,
slot_id=slot_id,
side=side,
order_id=order_id,
execution_id=execution_id,
skip_reason=skip_reason,
)
now = datetime.now(tz=UTC)
notify = False
for _ in range(2):
try:
with self._psycopg.connect(self._dsn, autocommit=False) as conn, conn.cursor() as cur:
cur.execute(
"INSERT INTO ledger_skip "
"(skip_key, instance_key, slot_id, portfolio_id, symbol, order_id, "
" execution_id, side, qty, price, executed_at, order_venue, "
" execution_venue, skip_reason, status, created_at, updated_at) "
"VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, "
" 'unresolved', %s, %s)",
(
skip_key,
instance_key,
slot_id,
portfolio_id,
symbol,
order_id,
execution_id,
side,
qty,
price,
executed_at,
order_venue or "unknown",
execution_venue or "unknown",
skip_reason,
now,
now,
),
)
conn.commit()
notify = True
break
except self._psycopg.errors.UniqueViolation:
with self._psycopg.connect(self._dsn, autocommit=False) as conn, conn.cursor() as cur:
cur.execute(
"SELECT status FROM ledger_skip WHERE skip_key = %s FOR UPDATE",
(skip_key,),
)
row = cur.fetchone()
if row is None:
continue
notify = row[0] == "resolved"
cur.execute(
"UPDATE ledger_skip SET portfolio_id=%s, symbol=%s, order_id=%s, "
"execution_id=%s, qty=%s, price=%s, executed_at=%s, "
"order_venue=%s, execution_venue=%s, status='unresolved', "
"resolution_reason=NULL, resolved_at=NULL, updated_at=%s "
"WHERE skip_key=%s",
(
portfolio_id,
symbol,
order_id,
execution_id,
qty,
price,
executed_at,
order_venue or "unknown",
execution_venue or "unknown",
now,
skip_key,
),
)
conn.commit()
break
else:
raise RuntimeError(f"failed to persist ledger_skip row for key {skip_key!r}")
record = self.get(skip_key)
if record is None:
raise RuntimeError(f"ledger_skip row missing after upsert for key {skip_key!r}")
return record, notify
def get(self, skip_key: str) -> LedgerSkipRecord | None:
with self._psycopg.connect(self._dsn) as conn, conn.cursor() as cur:
cur.execute(
"SELECT skip_key, instance_key, slot_id, portfolio_id, symbol, order_id, "
"execution_id, side, qty, price, executed_at, order_venue, execution_venue, "
"skip_reason, status, resolution_reason, resolved_at, created_at, updated_at "
"FROM ledger_skip WHERE skip_key = %s",
(skip_key,),
)
row = cur.fetchone()
return self._row_to_record(row) if row else None
def list_records(
self,
*,
instance_key: str | None = None,
portfolio_id: str | None = None,
on_date: date | None = None,
status: str = "unresolved",
limit: int = 100,
) -> list[LedgerSkipRecord]:
clauses: list[str] = []
params: list[object] = []
if status != "all":
clauses.append("status = %s")
params.append(status)
if instance_key is not None:
clauses.append("instance_key = %s")
params.append(instance_key)
if portfolio_id is not None:
clauses.append("portfolio_id = %s")
params.append(portfolio_id)
if on_date is not None:
start = datetime.combine(on_date, time.min, tzinfo=UTC)
end = start + timedelta(days=1)
clauses.append("created_at >= %s AND created_at < %s")
params.extend([start, end])
where = f"WHERE {' AND '.join(clauses)}" if clauses else ""
with self._psycopg.connect(self._dsn) as conn, conn.cursor() as cur:
cur.execute(
"SELECT skip_key, instance_key, slot_id, portfolio_id, symbol, order_id, "
"execution_id, side, qty, price, executed_at, order_venue, execution_venue, "
"skip_reason, status, resolution_reason, resolved_at, created_at, updated_at "
f"FROM ledger_skip {where} ORDER BY created_at DESC LIMIT %s",
(*params, limit),
)
rows = cur.fetchall()
return [self._row_to_record(row) for row in rows]
def resolve_for_fill(
self,
*,
instance_key: str,
slot_id: UUID,
side: str,
order_id: str | None,
execution_id: str | None,
resolution_reason: str,
) -> list[LedgerSkipRecord]:
now = datetime.now(tz=UTC)
with self._psycopg.connect(self._dsn, autocommit=False) as conn, conn.cursor() as cur:
if execution_id:
cur.execute(
"UPDATE ledger_skip SET status='resolved', resolution_reason=%s, "
"execution_id=COALESCE(execution_id, %s), "
"resolved_at=%s, updated_at=%s "
"WHERE status='unresolved' AND instance_key=%s AND slot_id=%s "
"AND side=%s AND (execution_id=%s OR (execution_id IS NULL AND order_id=%s)) "
"RETURNING skip_key, instance_key, slot_id, portfolio_id, symbol, "
"order_id, execution_id, side, qty, price, executed_at, order_venue, "
"execution_venue, skip_reason, status, resolution_reason, resolved_at, "
"created_at, updated_at",
(
resolution_reason,
execution_id,
now,
now,
instance_key,
slot_id,
side,
execution_id,
order_id,
),
)
else:
cur.execute(
"UPDATE ledger_skip SET status='resolved', resolution_reason=%s, "
"resolved_at=%s, updated_at=%s "
"WHERE status='unresolved' AND instance_key=%s AND slot_id=%s "
"AND side=%s AND order_id=%s "
"RETURNING skip_key, instance_key, slot_id, portfolio_id, symbol, "
"order_id, execution_id, side, qty, price, executed_at, order_venue, "
"execution_venue, skip_reason, status, resolution_reason, resolved_at, "
"created_at, updated_at",
(
resolution_reason,
now,
now,
instance_key,
slot_id,
side,
order_id,
),
)
rows = cur.fetchall()
conn.commit()
return [self._row_to_record(row) for row in rows]
def resolve_for_delivery(
self,
*,
instance_key: str,
portfolio_id: str,
side: str,
order_id: str,
execution_id: str,
resolution_reason: str,
) -> list[LedgerSkipRecord]:
now = datetime.now(tz=UTC)
with self._psycopg.connect(self._dsn, autocommit=False) as conn, conn.cursor() as cur:
cur.execute(
"UPDATE ledger_skip SET status='resolved', resolution_reason=%s, "
"execution_id=COALESCE(execution_id, %s), "
"resolved_at=%s, updated_at=%s "
"WHERE status='unresolved' AND instance_key=%s AND portfolio_id=%s AND side=%s "
"AND (execution_id=%s OR (execution_id IS NULL AND order_id=%s)) "
"RETURNING skip_key, instance_key, slot_id, portfolio_id, symbol, order_id, "
"execution_id, side, qty, price, executed_at, order_venue, execution_venue, "
"skip_reason, status, resolution_reason, resolved_at, created_at, updated_at",
(
resolution_reason,
execution_id,
now,
now,
instance_key,
portfolio_id,
side,
execution_id,
order_id,
),
)
rows = cur.fetchall()
conn.commit()
return [self._row_to_record(row) for row in rows]
def resolve_by_key(
self,
skip_key: str,
*,
instance_key: str,
resolution_reason: str,
) -> LedgerSkipRecord | None:
now = datetime.now(tz=UTC)
with self._psycopg.connect(self._dsn, autocommit=False) as conn, conn.cursor() as cur:
cur.execute(
"UPDATE ledger_skip SET status='resolved', resolution_reason=%s, "
"resolved_at=%s, updated_at=%s "
"WHERE skip_key=%s AND instance_key=%s AND status='unresolved' "
"RETURNING skip_key, instance_key, slot_id, portfolio_id, symbol, order_id, "
"execution_id, side, qty, price, executed_at, order_venue, execution_venue, "
"skip_reason, status, resolution_reason, resolved_at, created_at, updated_at",
(resolution_reason, now, now, skip_key, instance_key),
)
row = cur.fetchone()
conn.commit()
return self._row_to_record(row) if row else None
@staticmethod
def _row_to_record(row: tuple) -> LedgerSkipRecord:
return LedgerSkipRecord(
skip_key=row[0],
instance_key=row[1],
slot_id=row[2],
portfolio_id=row[3],
symbol=row[4],
order_id=row[5],
execution_id=row[6],
side=row[7],
qty=float(row[8]),
price=float(row[9]),
executed_at=row[10],
order_venue=row[11],
execution_venue=row[12],
skip_reason=row[13],
status=row[14],
resolution_reason=row[15],
resolved_at=row[16],
created_at=row[17],
updated_at=row[18],
)
@@ -0,0 +1,30 @@
CREATE TABLE IF NOT EXISTS ledger_skip (
skip_key TEXT PRIMARY KEY,
instance_key TEXT NOT NULL,
slot_id UUID NOT NULL,
portfolio_id TEXT NULL,
symbol TEXT NULL,
order_id TEXT NULL,
execution_id TEXT NULL,
side TEXT NOT NULL CHECK (side IN ('buy', 'sell')),
qty DOUBLE PRECISION NOT NULL,
price DOUBLE PRECISION NOT NULL,
executed_at TIMESTAMPTZ NOT NULL,
order_venue TEXT NOT NULL,
execution_venue TEXT NOT NULL,
skip_reason TEXT NOT NULL,
status TEXT NOT NULL CHECK (status IN ('unresolved', 'resolved')),
resolution_reason TEXT NULL,
resolved_at TIMESTAMPTZ NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_ledger_skip_status_created
ON ledger_skip (status, created_at);
CREATE INDEX IF NOT EXISTS idx_ledger_skip_instance_portfolio_created
ON ledger_skip (instance_key, portfolio_id, created_at);
CREATE INDEX IF NOT EXISTS idx_ledger_skip_execution_id
ON ledger_skip (execution_id);
@@ -0,0 +1,16 @@
-- req-0007 / ac-0024: durable mismatch notification lifecycle state.
-- PK: identity_key (slot_id TEXT for B-case; "C:<sha256_hex>" length-prefix-digest for C-case).
-- lifecycle_state: 'notified' | 'pending_resolved'.
-- ON CONFLICT DO UPDATE: replay-safe; re-running migration is a no-op.
CREATE TABLE IF NOT EXISTS reconcile_notification_state (
identity_key TEXT PRIMARY KEY,
case_code TEXT NOT NULL CHECK (case_code IN ('B', 'C')),
instance_key TEXT NOT NULL,
symbol TEXT NOT NULL,
quantity_a INTEGER NOT NULL,
quantity_b INTEGER NOT NULL,
lifecycle_state TEXT NOT NULL
CHECK (lifecycle_state IN ('notified', 'pending_resolved')),
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
@@ -0,0 +1,122 @@
"""PostgreSQL-backed durable reconcile notification lifecycle store (req-0007, ac-0024).
Tracks per-identity B/C-case mismatch notification fingerprints so that
duplicate alarms are suppressed across scheduler ticks and process restarts,
and pending resolved transitions can be retried.
PK: identity_key (slot_id TEXT for B-case; "C:<sha256_hex>" length-prefix-digest for C-case).
lifecycle_state: 'notified' | 'pending_resolved'.
"""
from __future__ import annotations
from datetime import UTC, datetime
from typing import Any
from trex_core.adapters.persistence.postgresql import _require_psycopg
class PostgreSQLReconcileNotificationStore:
"""`reconcile_notification_state` table CRUD (req-0007, ac-0024)."""
def __init__(self, dsn: str, *, psycopg_module: Any = None) -> None:
self._psycopg = _require_psycopg(psycopg_module)
self._dsn = dsn
def get_state(self, identity_key: str) -> dict | None:
"""Return state dict or None."""
with self._psycopg.connect(self._dsn) as conn, conn.cursor() as cur:
cur.execute(
"SELECT identity_key, case_code, instance_key, symbol, "
" quantity_a, quantity_b, lifecycle_state "
"FROM reconcile_notification_state "
"WHERE identity_key = %s LIMIT 1",
(identity_key,),
)
row = cur.fetchone()
if row is None:
return None
return {
"identity_key": row[0],
"case_code": row[1],
"instance_key": row[2],
"symbol": row[3],
"quantity_a": row[4],
"quantity_b": row[5],
"lifecycle_state": row[6],
}
def set_state(
self,
identity_key: str,
*,
case_code: str,
instance_key: str,
symbol: str,
quantity_a: int,
quantity_b: int,
lifecycle_state: str,
) -> None:
"""Upsert notification state."""
now = datetime.now(tz=UTC)
with self._psycopg.connect(self._dsn, autocommit=False) as conn:
with conn.cursor() as cur:
cur.execute(
"INSERT INTO reconcile_notification_state "
"(identity_key, case_code, instance_key, symbol, "
" quantity_a, quantity_b, lifecycle_state, created_at, updated_at) "
"VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s) "
"ON CONFLICT (identity_key) DO UPDATE "
"SET case_code = EXCLUDED.case_code, "
" instance_key = EXCLUDED.instance_key, "
" symbol = EXCLUDED.symbol, "
" quantity_a = EXCLUDED.quantity_a, "
" quantity_b = EXCLUDED.quantity_b, "
" lifecycle_state = EXCLUDED.lifecycle_state, "
" updated_at = EXCLUDED.updated_at",
(
identity_key,
case_code,
instance_key,
symbol,
quantity_a,
quantity_b,
lifecycle_state,
now,
now,
),
)
conn.commit()
def clear_state(self, identity_key: str) -> None:
"""Delete state row (called after successful resolved publish)."""
with self._psycopg.connect(self._dsn, autocommit=False) as conn:
with conn.cursor() as cur:
cur.execute(
"DELETE FROM reconcile_notification_state WHERE identity_key = %s",
(identity_key,),
)
conn.commit()
def list_states_by_lifecycle(self, lifecycle_state: str) -> list[dict]:
"""Return all state dicts with the given lifecycle_state."""
with self._psycopg.connect(self._dsn) as conn, conn.cursor() as cur:
cur.execute(
"SELECT identity_key, case_code, instance_key, symbol, "
" quantity_a, quantity_b, lifecycle_state "
"FROM reconcile_notification_state "
"WHERE lifecycle_state = %s",
(lifecycle_state,),
)
rows = cur.fetchall()
return [
{
"identity_key": row[0],
"case_code": row[1],
"instance_key": row[2],
"symbol": row[3],
"quantity_a": row[4],
"quantity_b": row[5],
"lifecycle_state": row[6],
}
for row in rows
]
@@ -68,6 +68,16 @@ class PostgreSQLReconcileWarningStore:
cur.execute("SELECT slot_id FROM slot_reconcile_warning")
return {row[0] for row in cur.fetchall()}
def list_c_case_warned(self) -> set[str]:
"""Return slot_id strings with an active case_code='C' warning (AC-0023)."""
with self._psycopg.connect(self._dsn) as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT slot_id FROM slot_reconcile_warning WHERE case_code = %s",
("C",),
)
return {row[0] for row in cur.fetchall()}
def get_warning_info(self, slot_id: str) -> dict | None:
"""Return {"portfolio_id", "case_code", "actor", "reason"} or None (REQ-105/106)."""
with self._psycopg.connect(self._dsn) as conn:
+26 -1
View File
@@ -49,6 +49,7 @@ class AccountSnapshotUnavailable(RuntimeError):
"account_mismatch",
"expired",
"live_read_failed",
"read_failed",
})
def __init__(self, code: str) -> None:
@@ -189,7 +190,15 @@ class AccountSnapshotService:
account_hash = self._broker.account_scope_hash()
if not account_hash:
raise AccountSnapshotUnavailable("account_mismatch")
snapshot = self._store.load(account_hash)
try:
snapshot = self._store.load(account_hash)
except Exception as exc:
_log.warning(
"account snapshot load failed: account_hash=%s",
account_hash,
exc_info=True,
)
raise AccountSnapshotUnavailable("read_failed") from exc
if snapshot is None or snapshot.account_hash != account_hash:
raise AccountSnapshotUnavailable("account_mismatch")
return snapshot
@@ -217,6 +226,22 @@ class AccountSnapshotService:
)
return list(snapshot.holdings)
def holdings_for_slots_rpc(self) -> tuple[list[BalanceItem], datetime]:
"""Return freshness-validated snapshot holdings for strategy slots reads.
Unlike ``holdings_for_sell()``, this read path never performs a live broker
call and therefore cannot mutate broker state or mask snapshot staleness.
Freshness uses the same ``sell_max_age_sec`` policy (`self._sell_max_age`)
as pending-sell quantity protection so strategy reads and sell dispatch
share one stale-snapshot threshold.
"""
snapshot = self._load_current()
captured_at = snapshot.holdings_captured_at
if captured_at is None:
raise AccountSnapshotUnavailable("not_ready")
self._age(captured_at, self._sell_max_age)
return list(snapshot.holdings), captured_at
def buy_guard(
self,
*,
+18 -13
View File
@@ -17,11 +17,10 @@ where no broker connection exists. In that case publish_alarm() logs and skips.
from __future__ import annotations
import logging
from datetime import datetime, timezone, timedelta
from typing import Optional
from datetime import datetime, timedelta, timezone
from trex_core.application.messaging import MessagingTransport, TransportUnavailableError
from trex_core.adapters.messaging.topology import alarm_routing_key
from trex_core.application.messaging import MessagingTransport, TransportUnavailableError
_log = logging.getLogger(__name__)
@@ -41,7 +40,7 @@ class AlarmEmitter:
- None-transport guard for non-daemon entry points
"""
def __init__(self, transport: Optional[MessagingTransport]) -> None:
def __init__(self, transport: MessagingTransport | None) -> None:
self._transport = transport
def publish_alarm(
@@ -49,9 +48,9 @@ class AlarmEmitter:
code: str,
title: str,
*,
fields: Optional[dict] = None,
fields: dict | None = None,
severity: str = "critical",
) -> None:
) -> bool:
"""Publish a single alarm envelope. Best-effort: never re-raises.
Args:
@@ -60,13 +59,16 @@ class AlarmEmitter:
exception bodies — callers must sanitize.
fields: Optional structured context dict (opaque, default {}).
severity: 'critical' | 'warn' | 'info'. Default 'critical'.
Returns:
True if the alarm was successfully published, False otherwise.
"""
if self._transport is None:
_log.info(
"alarm skipped (no transport): code=%s title=%s",
code, title,
)
return
return False
routing_key = alarm_routing_key(severity, _SOURCE)
occurred_at = datetime.now(tz=_KST).isoformat()
@@ -81,13 +83,16 @@ class AlarmEmitter:
}
try:
self._transport.publish_alarm_event(routing_key, payload)
except TransportUnavailableError as exc:
return True
except TransportUnavailableError:
_log.error(
"alarm publish skipped (transport unavailable): code=%s: %s",
code, exc,
"alarm publish skipped (transport unavailable): code=%s title=%s",
code, title,
)
except Exception: # noqa: BLE001 — alarm must never crash the caller
return False
except Exception as exc: # noqa: BLE001 — alarm must never crash the caller
_log.error(
"alarm publish failed: code=%s title=%s",
code, title, exc_info=True,
"alarm publish failed: code=%s title=%s error_type=%s",
code, title, type(exc).__name__,
)
return False
+1
View File
@@ -107,6 +107,7 @@ class FillStatus:
# Broker-authoritative execution exchange. For SOR orders this is KRX or
# NXT; None means the polling surface did not establish it.
execution_venue: Optional[str] = None
executed_at: Optional[datetime] = None
# dcy-013 / REQ-012: an external cancel was detected.
# order had a partial fill before the cancel, state stays "partially_filled"
# and this flag is set so execution records the filled portion and returns
+80 -16
View File
@@ -25,9 +25,9 @@ from __future__ import annotations
import inspect
import logging
from collections.abc import Callable
from dataclasses import dataclass, replace
from datetime import datetime, timedelta, timezone
from typing import Callable, Optional
from datetime import UTC, datetime, timedelta
from trex_core.application.alarm_emitter import AlarmEmitter
from trex_core.application.broker_port import (
@@ -39,6 +39,7 @@ from trex_core.application.broker_port import (
)
from trex_core.application.ledger_outbox import LedgerOutbox
from trex_core.application.ledger_port import LedgerTrade
from trex_core.application.ledger_skip import publish_ledger_skip_alarm
from trex_core.domain.pricing import ceil_to_tick, floor_to_tick
from trex_core.domain.slot import SlotService, SlotState, SlotStateMismatchError
from trex_core.runtime.logging_setup import set_correlation_id
@@ -57,7 +58,7 @@ _log = logging.getLogger(__name__)
# `slot.strategy_id` so that slot_filled_sell events reach the correct
# per-strategy queue instead of being NO_ROUTEd to the DLQ.
SlotEventEmitter = Callable[
[str, object, Optional[str]], None
[str, object, str | None], None
] # (event_type, slot, routing_strategy_id) -> None
_MAX_DISPATCH_PER_TICK = 64
@@ -81,13 +82,14 @@ class ExecutionService:
slot_service: SlotService,
repo,
broker: BrokerPort,
ledger_outbox: Optional[LedgerOutbox] = None,
event_emitter: Optional[SlotEventEmitter] = None,
stale_pending_threshold_sec: Optional[int] = None,
ledger_outbox: LedgerOutbox | None = None,
event_emitter: SlotEventEmitter | None = None,
stale_pending_threshold_sec: int | None = None,
instance_key: str = "default",
alarm_emitter: Optional[AlarmEmitter] = None,
balance_reader: Optional[Callable[[], list[BalanceItem]]] = None,
clock: Optional[Callable[[], datetime]] = None,
alarm_emitter: AlarmEmitter | None = None,
ledger_skip_store=None,
balance_reader: Callable[[], list[BalanceItem]] | None = None,
clock: Callable[[], datetime] | None = None,
) -> None:
self._svc = slot_service
self._repo = repo
@@ -108,11 +110,12 @@ class ExecutionService:
)
# dcy-033 / REQ-100: best-effort alarm emitter (None = no alarms sent).
self._alarm_emitter = alarm_emitter
self._ledger_skip_store = ledger_skip_store
# One emission per durable unresolved state/attempt in this process.
# A restart intentionally emits once again, covering a crash between a
# durable cancel reservation and best-effort alarm publication.
self._order_alarm_keys: set[tuple[str, str, int]] = set()
self._clock = clock or (lambda: datetime.now(tz=timezone.utc))
self._clock = clock or (lambda: datetime.now(tz=UTC))
# dcy-041 / GitHub #45: pending-sell quantity may use a bounded
# account snapshot only through this injected reader. Reconciliation
# remains wired directly to the live broker.
@@ -137,7 +140,7 @@ class ExecutionService:
# ------------------------------------------------------------
def _dispatch(self, r: ExecutionResult) -> None:
sell_balance: Optional[dict[str, int]] = None # lazy, once per tick
sell_balance: dict[str, int] | None = None # lazy, once per tick
for slot in self._repo.list_undispatched_pending(
self._instance_key, _MAX_DISPATCH_PER_TICK
):
@@ -266,7 +269,7 @@ class ExecutionService:
r.rolled_back += 1
def _poll(self, r: ExecutionResult) -> None:
now = self._clock().astimezone(timezone.utc)
now = self._clock().astimezone(UTC)
for slot in self._repo.list_dispatched_pending(self._instance_key):
side = "buy" if slot.status is SlotState.PENDING_BUY else "sell"
set_correlation_id(str(slot.slot_id)) # REQ-084: same key as submit
@@ -716,7 +719,7 @@ class ExecutionService:
},
)
def _emit_event(self, event_type: str, slot, routing_strategy_id: Optional[str] = None) -> None:
def _emit_event(self, event_type: str, slot, routing_strategy_id: str | None = None) -> None:
if self._event_emitter is None:
return
try:
@@ -754,6 +757,14 @@ class ExecutionService:
slot.symbol,
side,
)
self._persist_ledger_skip(
slot,
fs,
side,
broker_order_id,
skip_reason="missing_portfolio_id",
price=resolved_price,
)
return
if fs.execution_id is None:
# Broker contract regression — slot already transitioned; surface for ops.
@@ -765,6 +776,14 @@ class ExecutionService:
slot.portfolio_id,
side,
)
self._persist_ledger_skip(
slot,
fs,
side,
broker_order_id,
skip_reason="missing_execution_id",
price=resolved_price,
)
return
# REQ-095 KD-3: use pre-resolved price when available; fall back to
# fs.avg_price for any direct callers that do not pass resolved_price.
@@ -776,16 +795,26 @@ class ExecutionService:
execution_venue = fs.execution_venue
if execution_venue is None and slot.venue in {"KRX", "NXT"}:
execution_venue = slot.venue
if execution_venue not in {"KRX", "NXT"}:
if execution_venue is not None and execution_venue not in {"KRX", "NXT"}:
r.ledger_skipped += 1
_log.warning(
"ledger skip: slot=%s portfolio_id=%s side=%s "
"reason=missing_execution_venue order_venue=%s",
"reason=unsupported_execution_venue execution_venue=%s order_venue=%s",
slot.slot_id,
slot.portfolio_id,
side,
execution_venue,
slot.venue,
)
self._persist_ledger_skip(
slot,
fs,
side,
broker_order_id,
skip_reason="unsupported_execution_venue",
price=price,
execution_venue=execution_venue,
)
return
trade = LedgerTrade(
asset_id=str(slot.portfolio_id),
@@ -796,13 +825,48 @@ class ExecutionService:
price=price,
fee=float(fs.fee),
tax=float(fs.tax),
executed_at=datetime.now(tz=timezone.utc),
executed_at=fs.executed_at or self._clock(),
venue=execution_venue,
order_venue=slot.venue,
)
self._ledger_outbox.enqueue(trade)
r.ledger_enqueued += 1
def _persist_ledger_skip(
self,
slot,
fs,
side: str,
broker_order_id: str,
*,
skip_reason: str,
price: float | None,
execution_venue: str | None = None,
) -> None:
if self._ledger_skip_store is None:
return
record, should_notify = self._ledger_skip_store.record_skip(
instance_key=self._instance_key,
slot_id=slot.slot_id,
portfolio_id=slot.portfolio_id,
symbol=slot.symbol,
order_id=str(broker_order_id) if broker_order_id else None,
execution_id=str(fs.execution_id) if fs.execution_id is not None else None,
side=side,
qty=float(fs.filled_qty),
price=(
float(price)
if price is not None
else (float(fs.avg_price) if fs.avg_price is not None else 0.0)
),
executed_at=fs.executed_at or self._clock(),
order_venue=getattr(slot, "venue", None) or "unknown",
execution_venue=execution_venue or getattr(fs, "execution_venue", None) or "unknown",
skip_reason=skip_reason,
)
if should_notify:
publish_ledger_skip_alarm(self._alarm_emitter, record, resolved=False)
def cancel_pending_slot(self, slot_id) -> str:
"""Operator backstop: send cancel_order for a pending slot (dcy-022/REQ-071).
+80 -4
View File
@@ -28,6 +28,7 @@ from decimal import Decimal
from typing import Any, Callable, Optional
from uuid import UUID
from trex_core.application.alarm_emitter import AlarmEmitter
from trex_core.application.ports import IntentStore
from trex_core.domain.slot import (
SlotAdmissionRejectedError,
@@ -115,6 +116,7 @@ class IntentService:
max_active_slots: Optional[int] = None,
default_order_timeout_sec: Optional[int] = None,
warn_store=None, # Optional[ReconcileWarningStore] — BAT-050 / REQ-104
alarm_emitter: Optional[AlarmEmitter] = None,
entry_allowed_fn: Optional[Callable[[], bool]] = None,
buy_guard_fn: Optional[Callable[[BuyIntent], Optional[str]]] = None,
) -> None:
@@ -141,6 +143,8 @@ class IntentService:
# BAT-050 / REQ-104: reconcile-warning gate store (Optional).
# When None, gate is skipped (backward compat, NFR-4).
self._warn_store = warn_store
# Optional best-effort alarm publisher for reconcile_warned rejections.
self._alarm_emitter = alarm_emitter
# dcy-036 / REQ-114: day-start entry gate (Optional callable → bool).
# When None or returns True, gate passes (backward compat / opt-in=False).
# When returns False, BuyIntent is rejected with reason 'day_start_load_failed'.
@@ -178,6 +182,13 @@ class IntentService:
self._intents.finalize(
intent.intent_id, decision.status, decision.slot_id, decision.reason
)
if decision.status == "rejected" and decision.reason == "reconcile_warned":
self._publish_reconcile_warned_rejection_alarm(
intent_id=intent.intent_id,
intent_type="buy",
strategy_id=intent.strategy_id,
slot=self._resolve_buy_alarm_slot(intent),
)
return decision
def submit_sell(self, intent: SellIntent) -> IntentDecision:
@@ -196,8 +207,69 @@ class IntentService:
self._intents.finalize(
intent.intent_id, decision.status, decision.slot_id, decision.reason
)
if decision.status == "rejected" and decision.reason == "reconcile_warned":
self._publish_reconcile_warned_rejection_alarm(
intent_id=intent.intent_id,
intent_type="sell",
strategy_id=intent.sell_strategy_id,
slot=self._resolve_sell_alarm_slot(intent),
)
return decision
def _resolve_buy_alarm_slot(self, intent: BuyIntent) -> Any:
try:
if intent.slot_id is not None:
return self._repo.fetch_slot(intent.slot_id)
if intent.portfolio_id:
return self._repo.find_slot_by_portfolio(
self._instance_key, intent.portfolio_id
)
except Exception: # noqa: BLE001 - optional alarm context lookup
return None
return None
def _resolve_sell_alarm_slot(self, intent: SellIntent) -> Any:
try:
return self._repo.fetch_slot(intent.slot_id)
except Exception: # noqa: BLE001 - optional alarm context lookup
return None
def _publish_reconcile_warned_rejection_alarm(
self,
*,
intent_id: str,
intent_type: str,
strategy_id: Optional[str],
slot: Any,
) -> None:
if self._alarm_emitter is None:
return
fields: dict[str, Any] = {
"instance_key": self._instance_key,
"intent_id": intent_id,
"intent_type": intent_type,
"reason": "reconcile_warned",
"broker_submitted": False,
}
if slot is not None:
if getattr(slot, "slot_id", None) is not None:
fields["slot_id"] = str(slot.slot_id)
if getattr(slot, "portfolio_id", None):
fields["portfolio_id"] = slot.portfolio_id
if getattr(slot, "symbol", None):
fields["symbol"] = slot.symbol
if strategy_id:
fields["strategy_id"] = strategy_id
try:
self._alarm_emitter.publish_alarm(
"strategy_intent_rejected",
"Strategy intent rejected by reconcile warning gate",
fields=fields,
severity="warn",
)
except Exception: # noqa: BLE001 - alarm path must not affect intent decision
pass
# ------------------------------------------------------------
def _dedup(self, intent_id: str, intent_type: str, payload: dict) -> Optional[IntentDecision]:
"""Return prior decision if intent_id already seen; else None (and reserve it)."""
@@ -271,8 +343,10 @@ class IntentService:
return IntentDecision(
intent.intent_id, "rejected", None, "reconcile_warned"
)
except Exception: # noqa: BLE001 - fail-open
pass
except Exception: # noqa: BLE001 - fail-closed: warn-store down → reject
return IntentDecision(
intent.intent_id, "rejected", None, "reconcile_warning_unavailable"
)
# dcy-021: a buy on a HOLDING slot is a position-sizing increase.
is_increase = slot.status is SlotState.HOLDING
@@ -362,8 +436,10 @@ class IntentService:
return IntentDecision(
intent.intent_id, "rejected", None, "reconcile_warned"
)
except Exception: # noqa: BLE001 - fail-open
pass
except Exception: # noqa: BLE001 - fail-closed: warn-store down → reject
return IntentDecision(
intent.intent_id, "rejected", None, "reconcile_warning_unavailable"
)
if slot.status is not SlotState.HOLDING:
return IntentDecision(
+36 -1
View File
@@ -25,10 +25,11 @@ from __future__ import annotations
import logging
import random
from collections.abc import Callable
from dataclasses import dataclass, replace
from datetime import datetime, timedelta, timezone
from enum import Enum
from typing import Callable, Optional, Protocol
from typing import Optional, Protocol
from trex_core.application.alarm_emitter import AlarmEmitter
from trex_core.application.ledger_port import (
@@ -39,6 +40,7 @@ from trex_core.application.ledger_port import (
LedgerTrade,
LedgerWriteResult,
)
from trex_core.application.ledger_skip import publish_ledger_skip_alarm
_log = logging.getLogger(__name__)
@@ -171,6 +173,8 @@ class LedgerWriter:
clock: Optional[Callable[[], datetime]] = None,
rng: Optional[Callable[[], float]] = None,
alarm_emitter: Optional[AlarmEmitter] = None,
ledger_skip_store=None,
instance_key: str | None = None,
) -> None:
self._outbox = outbox
self._ledger = ledger
@@ -185,6 +189,8 @@ class LedgerWriter:
self._rng = rng or random.random # jitter source; inject for determinism
# dcy-033 / REQ-100: best-effort alarm emitter.
self._alarm_emitter = alarm_emitter
self._ledger_skip_store = ledger_skip_store
self._instance_key = instance_key
def enqueue_fill(self, trade: LedgerTrade) -> LedgerWorkItem:
return self._outbox.enqueue(trade)
@@ -205,6 +211,13 @@ class LedgerWriter:
stats.delivery_failed += 1
return stats
def process_correlation_id(self, correlation_id: str) -> OutboxOutcome | None:
"""Process one specific outbox row immediately."""
item = self._outbox.get(correlation_id)
if item is None:
return None
return self._process(item)
def _backoff_delay(self, attempts: int) -> float:
"""Exponential backoff with jitter: min(base*2^(n-1), cap) + up to 25%."""
base_delay = min(self._retry_base * (2 ** (attempts - 1)), self._retry_cap)
@@ -294,10 +307,32 @@ class LedgerWriter:
# execution_id either way (REQ-019 AC-1, AC-4).
if outcome.result is LedgerWriteResult.DUPLICATE_OK:
self._outbox.mark_outcome(item.correlation_id, OutboxOutcome.DUPLICATE_COMPLETED)
self._resolve_ledger_skip(item.trade)
return OutboxOutcome.DUPLICATE_COMPLETED
self._outbox.mark_outcome(item.correlation_id, OutboxOutcome.COMPLETED)
self._resolve_ledger_skip(item.trade)
return OutboxOutcome.COMPLETED
def _resolve_ledger_skip(self, trade: LedgerTrade) -> None:
if (
self._ledger_skip_store is None
or self._instance_key is None
or not trade.asset_id
or not trade.order_id
or not trade.execution_id
):
return
records = self._ledger_skip_store.resolve_for_delivery(
instance_key=self._instance_key,
portfolio_id=trade.asset_id,
side=trade.side,
order_id=trade.order_id,
execution_id=trade.execution_id,
resolution_reason="ledger_recorded",
)
for record in records:
publish_ledger_skip_alarm(self._alarm_emitter, record, resolved=True)
def _writeback(self, trade: LedgerTrade, outcome) -> None:
"""Option D cash/quantity write-back (dcy-018). best-effort; skips when
no SnapshotStore is wired or the response omits authoritative values."""
+98
View File
@@ -0,0 +1,98 @@
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime
from uuid import UUID
from trex_core.application.alarm_emitter import AlarmEmitter
@dataclass(frozen=True)
class LedgerSkipRecord:
skip_key: str
instance_key: str
slot_id: UUID
portfolio_id: str | None
symbol: str | None
order_id: str | None
execution_id: str | None
side: str
qty: float
price: float
executed_at: datetime
order_venue: str
execution_venue: str
skip_reason: str
status: str
resolution_reason: str | None
resolved_at: datetime | None
created_at: datetime
updated_at: datetime
def make_ledger_skip_key(
*,
instance_key: str,
slot_id: UUID,
side: str,
order_id: str | None,
execution_id: str | None,
skip_reason: str,
) -> str:
return ":".join(
[
"ledger-skip",
instance_key,
str(slot_id),
side,
order_id or "-",
execution_id or "-",
skip_reason,
]
)
def ledger_skip_alarm_fields(record: LedgerSkipRecord) -> dict[str, object]:
return {
"skip_key": record.skip_key,
"instance_key": record.instance_key,
"slot_id": str(record.slot_id),
"portfolio_id": record.portfolio_id,
"symbol": record.symbol,
"order_id": record.order_id,
"execution_id": record.execution_id,
"side": record.side,
"qty": record.qty,
"price": record.price,
"executed_at": record.executed_at.isoformat(),
"order_venue": record.order_venue,
"execution_venue": record.execution_venue,
"skip_reason": record.skip_reason,
"status": record.status,
"resolution_reason": record.resolution_reason,
"resolved_at": record.resolved_at.isoformat() if record.resolved_at else None,
"created_at": record.created_at.isoformat(),
"updated_at": record.updated_at.isoformat(),
}
def publish_ledger_skip_alarm(
alarm_emitter: AlarmEmitter | None,
record: LedgerSkipRecord,
*,
resolved: bool,
) -> None:
if alarm_emitter is None:
return
code = "ledger_skip_resolved" if resolved else "ledger_skip_unresolved"
title = (
f"원장 skip 해소: slot={record.slot_id} reason={record.resolution_reason or 'resolved'}"
if resolved
else f"원장 skip 발생: slot={record.slot_id} reason={record.skip_reason}"
)
alarm_emitter.publish_alarm(
code,
title,
fields=ledger_skip_alarm_fields(record),
severity="info" if resolved else "critical",
)
+82 -4
View File
@@ -9,10 +9,12 @@ from __future__ import annotations
import logging
from datetime import date as date_cls
from datetime import datetime, timezone
from typing import Optional
from typing import Optional, Protocol
from uuid import UUID
from trex_core.application.intents import BuyIntent, IntentService, SellIntent
from trex_core.application.account_snapshot import AccountSnapshotUnavailable
from trex_core.application.broker_port import BalanceItem
from trex_core.application.messaging import RpcRequest, RpcResponse
from trex_core.application.portfolio import (
PauseStore,
@@ -185,6 +187,10 @@ _FILTER_STATUS: dict[str, set[str]] = {
}
class SlotsBrokerQuantityReader(Protocol):
def holdings_for_slots_rpc(self) -> tuple[list[BalanceItem], datetime]: ...
def make_slots_rpc_handler(
slot_repo: SlotRepository,
*,
@@ -197,6 +203,7 @@ def make_slots_rpc_handler(
ledger_outbox=None,
max_active_slots: Optional[int] = None,
strategy_state_store=None,
broker_quantity_reader: Optional[SlotsBrokerQuantityReader] = None,
):
"""Build the RPC handler for `trex.slots.req` (REQ-073/074/116).
@@ -252,6 +259,37 @@ def make_slots_rpc_handler(
for s in slots
]
broker_qty_by_symbol: dict[str, int] = {}
broker_qty_available = False
broker_qty_captured_at: Optional[datetime] = None
if broker_quantity_reader is not None:
try:
holdings, broker_qty_captured_at = broker_quantity_reader.holdings_for_slots_rpc()
broker_qty_by_symbol = {
str(item.symbol): int(item.hldg_qty)
for item in holdings
}
broker_qty_available = True
except AccountSnapshotUnavailable:
broker_qty_available = False
for row, slot in zip(slot_rows, slots, strict=True):
row["slot_quantity"] = row.get("qty")
row["qty_source"] = None
row["broker_quantity_available"] = False
row["broker_quantity_captured_at"] = None
if slot.status.value != "holding":
continue
if broker_qty_available:
broker_qty = int(broker_qty_by_symbol.get(str(slot.symbol), 0))
row["qty"] = float(broker_qty)
row["qty_source"] = "broker"
row["broker_quantity_available"] = True
row["broker_quantity_captured_at"] = broker_qty_captured_at.isoformat()
else:
row["qty"] = None
row["qty_source"] = "unavailable"
# dcy-028 / REQ-073/074·079 (C-1): attach opaque strategy_state map per
# row via one batched store read (read piggyback — no extra round trip).
# Only entries matching the slot's current occupancy_seq are surfaced.
@@ -259,7 +297,7 @@ def make_slots_rpc_handler(
slot_ids = [s.slot_id for s in slots]
occ = {s.slot_id: getattr(s, "occupancy_seq", 0) for s in slots}
state_map = strategy_state_store.get_for_slots(slot_ids, occ)
for row, s in zip(slot_rows, slots):
for row, s in zip(slot_rows, slots, strict=True):
entries = state_map.get(s.slot_id)
if entries:
row["strategy_state"] = {
@@ -309,13 +347,53 @@ def make_slots_rpc_handler(
if strategy_filter is not None:
slot_rows = [r for r in slot_rows if r.get("strategy_name") == strategy_filter]
return RpcResponse(payload={
# AC-0023: exclude holding slots with active case_code='C' reconcile warning.
# Fail-closed: if warn_store exists but throws, return degraded=True / no slots.
excluded_count = 0
if warn_store is not None:
try:
c_warned_ids = warn_store.list_c_case_warned()
except Exception: # noqa: BLE001
_log.exception("warn_store.list_c_case_warned() failed — fail-closed")
return RpcResponse(payload={
"slots": [],
"available_slots": available_slots,
"degraded": True,
"reason": "warn_store_unavailable",
"message": (
"reconcile warning store unavailable; "
"returning no slots (fail-closed)"
),
"excluded_slots": {"reconcile_quantity_mismatch": 0},
})
if c_warned_ids:
# Rebuild slot lookup from the original `slots` list (before strategy
# filter reduced slot_rows). slot_rows[i] corresponds to slots[i]
# only before filtering; use slot_id stored in each row.
slot_by_id = {str(s.slot_id): s for s in slots}
filtered_rows: list[dict] = []
for row in slot_rows:
slot_obj = slot_by_id.get(str(row.get("slot_id", "")))
if (
slot_obj is not None
and str(slot_obj.slot_id) in c_warned_ids
and slot_obj.status.value == "holding"
):
excluded_count += 1
else:
filtered_rows.append(row)
slot_rows = filtered_rows
base_payload: dict = {
"slots": slot_rows,
"available_slots": available_slots,
"degraded": degraded,
"reason": reason,
"message": message,
})
}
if warn_store is not None:
base_payload["excluded_slots"] = {"reconcile_quantity_mismatch": excluded_count}
return RpcResponse(payload=base_payload)
return handler
+134 -32
View File
@@ -17,10 +17,9 @@ from __future__ import annotations
import threading
from dataclasses import dataclass, field, replace
from datetime import datetime, timezone
from datetime import UTC, datetime
from enum import Enum
from typing import Optional, Protocol
from typing import Protocol
# ---------------------------------------------------------------- portfolio-hub
@@ -38,10 +37,10 @@ class PortfolioItem:
symbol: str
cash_balance: float
asset_quantity: float # integer shares semantically; float for tolerance
active_strategy_id: Optional[str] = None
active_strategy_id: str | None = None
strategy_params: dict = field(default_factory=dict) # opaque, pass-through
# REQ-116 additive fields (BAT-055) — safe defaults for legacy hub responses
strategy_name: Optional[str] = None # portfolio-hub strategy_name; None if absent
strategy_name: str | None = None # portfolio-hub strategy_name; None if absent
markets: list = field(default_factory=list) # portfolio-hub markets list; [] if absent
@@ -63,7 +62,7 @@ class RealizedPnlItem:
Minimum required: symbol, realized_pnl, sell_qty.
Additional fields preserved for completeness.
"""
symbol: Optional[str]
symbol: str | None
realized_pnl: float
sell_qty: float
cost_basis_per_unit: float = 0.0
@@ -78,7 +77,7 @@ class RealizedPnlResult:
Returned by get_realized_pnl port method.
"""
items: list[RealizedPnlItem]
cost_basis_from: Optional[str] # ISO date string or None
cost_basis_from: str | None # ISO date string or None
data_completeness_warning: bool # True when portfolio-hub signals incomplete history
total_realized_pnl: float
@@ -96,7 +95,7 @@ class BrokerCredentialPayload:
account_ref: str
app_key: str
app_secret: str
credential_version: Optional[int] = None
credential_version: int | None = None
@dataclass(frozen=True)
@@ -106,7 +105,7 @@ class TradingContext:
Returned by fetch_broker_credential() in a single atomic read to avoid
double HTTP calls.
"""
broker: Optional[BrokerCredentialPayload]
broker: BrokerCredentialPayload | None
trading_enabled: bool
@@ -132,8 +131,8 @@ class PortfolioHubReadPort(Protocol):
from_dt: str,
to_dt: str,
*,
symbol: Optional[str] = None,
) -> "RealizedPnlResult":
symbol: str | None = None,
) -> RealizedPnlResult:
"""GET /v1/portfolios/{portfolio_id}/pnl/realized — WAC realized PnL (REQ-109).
Args:
@@ -151,7 +150,7 @@ class PortfolioHubReadPort(Protocol):
"""
...
def fetch_broker_credential(self) -> "TradingContext":
def fetch_broker_credential(self) -> TradingContext:
"""GET /v1/user_info — broker credential + trading_enabled (REQ-112).
Returns a TradingContext with:
@@ -181,7 +180,7 @@ class SnapshotAsset:
"""A portfolio item augmented with overlays (warning, sequence)."""
item: PortfolioItem
warning: bool = False
updated_at: Optional[datetime] = None
updated_at: datetime | None = None
class SnapshotStore:
@@ -198,14 +197,14 @@ class SnapshotStore:
def __init__(self) -> None:
self._lock = threading.Lock()
self._assets: dict[str, SnapshotAsset] = {}
self._degraded: Optional[DegradedReason] = None
self._degraded_message: Optional[str] = None
self._last_refresh_at: Optional[datetime] = None
self._degraded: DegradedReason | None = None
self._degraded_message: str | None = None
self._last_refresh_at: datetime | None = None
# --- universe ---
def swap(self, items: list[PortfolioItem]) -> None:
"""Replace the entire universe atomically (immutable swap, Decision 6)."""
now = datetime.now(tz=timezone.utc)
now = datetime.now(tz=UTC)
with self._lock:
prior_warnings = {pid for pid, a in self._assets.items() if a.warning}
new_assets: dict[str, SnapshotAsset] = {}
@@ -225,7 +224,7 @@ class SnapshotStore:
with self._lock:
return list(self._assets.values())
def get(self, portfolio_id: str) -> Optional[SnapshotAsset]:
def get(self, portfolio_id: str) -> SnapshotAsset | None:
with self._lock:
return self._assets.get(portfolio_id)
@@ -256,7 +255,7 @@ class SnapshotStore:
self._assets[portfolio_id] = replace(
a, item=replace(a.item, cash_balance=cash_balance,
asset_quantity=asset_quantity),
updated_at=datetime.now(tz=timezone.utc),
updated_at=datetime.now(tz=UTC),
)
# --- degraded signal (REQ-028 AC-2) ---
@@ -270,11 +269,11 @@ class SnapshotStore:
self._degraded = None
self._degraded_message = None
def degraded_state(self) -> tuple[Optional[DegradedReason], Optional[str]]:
def degraded_state(self) -> tuple[DegradedReason | None, str | None]:
with self._lock:
return self._degraded, self._degraded_message
def last_refresh_at(self) -> Optional[datetime]:
def last_refresh_at(self) -> datetime | None:
with self._lock:
return self._last_refresh_at
@@ -297,7 +296,7 @@ class PauseStore(Protocol):
class InMemoryPauseStore:
"""Process-local pause store for tests + CI."""
def __init__(self, initial: Optional[set[str]] = None) -> None:
def __init__(self, initial: set[str] | None = None) -> None:
self._lock = threading.Lock()
self._paused: set[str] = set(initial or set())
@@ -379,10 +378,10 @@ class ReconcileWarningStore(Protocol):
def is_warned(self, slot_id: str) -> bool:
...
def list_warned(self) -> "set[str]":
def list_warned(self) -> set[str]:
...
def get_warning_info(self, slot_id: str) -> "Optional[dict]":
def get_warning_info(self, slot_id: str) -> dict | None:
"""Return {"portfolio_id", "case_code", "actor", "reason"} or None.
Used by verify to determine whether the warning is case_code='C'
@@ -391,6 +390,13 @@ class ReconcileWarningStore(Protocol):
"""
...
def list_c_case_warned(self) -> set[str]:
"""Return set of slot_id strings that have an active case_code='C' warning.
AC-0023: used by strategy slots RPC to exclude C-case mismatched holding slots.
"""
...
class InMemoryReconcileWarningStore:
"""Process-local reconcile-warning store for tests + CI (REQ-102)."""
@@ -425,23 +431,119 @@ class InMemoryReconcileWarningStore:
with self._lock:
return slot_id in self._warned
def list_warned(self) -> "set[str]":
def list_warned(self) -> set[str]:
with self._lock:
return set(self._warned.keys())
def get_warning_info(self, slot_id: str) -> "Optional[dict]":
def get_warning_info(self, slot_id: str) -> dict | None:
with self._lock:
info = self._warned.get(slot_id)
return dict(info) if info is not None else None
def list_c_case_warned(self) -> set[str]:
with self._lock:
return {sid for sid, info in self._warned.items() if info.get("case_code") == "C"}
class ReconcileNotificationStore(Protocol):
"""Durable mismatch notification lifecycle store (req-0007, ac-0024).
Tracks the last successfully published fingerprint for each B/C-case identity
to suppress duplicate alarms across scheduler ticks and process restarts.
Identity keys:
B-case: slot_id (UUID string)
C-case: `"{instance_key}\\x00C\\x00{symbol}"` (null-separated to avoid collisions)
lifecycle_state values:
"notified" — detected or changed alarm was published for this fingerprint
"pending_resolved" — warning was cleared but resolved publish failed; retry pending
"""
def get_state(self, identity_key: str) -> dict | None:
"""Return state dict or None.
State dict keys: identity_key, case_code, instance_key, symbol,
quantity_a, quantity_b, lifecycle_state.
"""
...
def set_state(
self,
identity_key: str,
*,
case_code: str,
instance_key: str,
symbol: str,
quantity_a: int,
quantity_b: int,
lifecycle_state: str,
) -> None:
"""Upsert notification state (call after successful publish)."""
...
def clear_state(self, identity_key: str) -> None:
"""Remove state (call after successful resolved publish)."""
...
def list_states_by_lifecycle(self, lifecycle_state: str) -> list[dict]:
"""Return all state dicts with the given lifecycle_state."""
...
class InMemoryReconcileNotificationStore:
"""Process-local notification lifecycle store for tests + CI (req-0007)."""
def __init__(self) -> None:
self._lock = threading.Lock()
self._states: dict[str, dict] = {}
def get_state(self, identity_key: str) -> dict | None:
with self._lock:
s = self._states.get(identity_key)
return dict(s) if s is not None else None
def set_state(
self,
identity_key: str,
*,
case_code: str,
instance_key: str,
symbol: str,
quantity_a: int,
quantity_b: int,
lifecycle_state: str,
) -> None:
with self._lock:
self._states[identity_key] = {
"identity_key": identity_key,
"case_code": case_code,
"instance_key": instance_key,
"symbol": symbol,
"quantity_a": quantity_a,
"quantity_b": quantity_b,
"lifecycle_state": lifecycle_state,
}
def clear_state(self, identity_key: str) -> None:
with self._lock:
self._states.pop(identity_key, None)
def list_states_by_lifecycle(self, lifecycle_state: str) -> list[dict]:
with self._lock:
return [
dict(s) for s in self._states.values()
if s["lifecycle_state"] == lifecycle_state
]
def slot_to_dict(
s,
*,
snapshot_store: "Optional[SnapshotStore]" = None,
snapshot_store: SnapshotStore | None = None,
watchdog_store=None,
pause_store: "Optional[PauseStore]" = None,
warn_store: "Optional[ReconcileWarningStore]" = None,
pause_store: PauseStore | None = None,
warn_store: ReconcileWarningStore | None = None,
watchdog_timeout_sec: int = 120,
) -> dict:
"""Merged slot row for trex.slots.req (REQ-073/074).
@@ -468,9 +570,9 @@ def slot_to_dict(
# hub authority: asset_quantity (hub primary) + cash_balance
# REQ-116: strategy_name / strategy_params / markets also from PortfolioItem
asset_qty: Optional[float] = None
cash_balance: Optional[float] = None
strategy_name_val: Optional[str] = None
asset_qty: float | None = None
cash_balance: float | None = None
strategy_name_val: str | None = None
strategy_params_val: dict = {}
markets_val: list = []
if snapshot_store is not None and s.portfolio_id:
+396 -20
View File
@@ -19,13 +19,17 @@ durable store is additive and Optional (None => warn-log-only, backward compat).
BAT-051 / REQ-105: sync_slot_to_broker — operator CLI sync path.
BAT-051 / REQ-106: verify_slot_reconcile — operator CLI verify path (C-only, read-centric).
req-0007 / ac-0024: B/C-case mismatch alarms (detected, changed, resolved) are
published via AlarmEmitter with durable deduplication through ReconcileNotificationStore.
"""
from __future__ import annotations
import hashlib
import logging
import math
import re
from typing import Optional, cast
from typing import cast
from trex_core.application.alarm_emitter import AlarmEmitter
from trex_core.application.broker_port import BrokerPort
@@ -47,7 +51,7 @@ _COST_BASIS_REASON_CODES = frozenset({
_COST_BASIS_LEGACY_SOURCE_ALIASES = frozenset({"avg_cost", "wac", "cost_basis"})
def _sanitize_cost_basis_reason(reason: object) -> Optional[str]:
def _sanitize_cost_basis_reason(reason: object) -> str | None:
"""Keep public diagnostics in the bounded REQ-123 reason taxonomy."""
if not isinstance(reason, str):
return None
@@ -62,7 +66,7 @@ def _normalize_cost_basis_diagnostics(
value: object,
source: object,
reason: object,
) -> tuple[Optional[float], str, Optional[str]]:
) -> tuple[float | None, str, str | None]:
"""Normalize diagnostic-port output before it reaches reconciliation results."""
safe_reason = _sanitize_cost_basis_reason(reason)
if source == "canonical":
@@ -89,7 +93,7 @@ def _normalize_cost_basis_diagnostics(
return normalized_value, normalized_source, safe_reason
def _normalize_legacy_cost_basis(value: object) -> tuple[Optional[float], str, Optional[str]]:
def _normalize_legacy_cost_basis(value: object) -> tuple[float | None, str, str | None]:
"""Make the diagnostic-free compatibility seam obey safe WAC semantics."""
normalized_value, _, _ = _normalize_cost_basis_diagnostics(value, "legacy", "invalid")
if normalized_value is None:
@@ -97,14 +101,255 @@ def _normalize_legacy_cost_basis(value: object) -> tuple[Optional[float], str, O
return normalized_value, "legacy", None
# ---------------------------------------------------------------------------
# req-0007 / ac-0024: Notification lifecycle helpers
# ---------------------------------------------------------------------------
def _c_identity_key(instance_key: str, symbol: str) -> str:
"""Return an injective, PostgreSQL TEXT-safe key for a (instance_key, symbol) C-case pair.
Uses a length-prefixed SHA-256 hex digest so that *any* inputs — including
those containing separator characters — produce distinct keys with no
collision risk. The prefix ``C:`` makes the key case-code-scoped and
human-scannable in the DB; the digest is 64 hex chars (256-bit).
Injectivity: the raw string fed to SHA-256 is ``"<N>:<instance_key>:<symbol>"``
where ``N = len(instance_key)``. The leading ``N:`` uniquely delimits
``instance_key`` from the rest, so two distinct ``(instance_key, symbol)``
pairs always produce different raw strings (and thus different digests).
Example: _c_identity_key("a\\x1fC", "b") != _c_identity_key("a", "C\\x1fb")
"""
raw = f"{len(instance_key)}:{instance_key}:{symbol}"
digest = hashlib.sha256(raw.encode()).hexdigest()
return f"C:{digest}"
def _build_alarm_fields(
*,
case_code: str,
instance_key: str,
symbol: str,
quantity_a: int,
quantity_b: int,
slot_id: str | None = None,
) -> dict:
"""Build sanitized alarm context fields (ac-0024 allowlist).
Excluded: account number/hash, credential, token, DSN, raw payload, exception.
"""
fields: dict = {
"instance_key": instance_key,
"symbol": symbol,
"case_code": case_code,
"broker_quantity": quantity_b,
}
if case_code == "B":
fields["slot_quantity"] = quantity_a
if slot_id is not None:
fields["slot_id"] = slot_id
else:
fields["hub_quantity"] = quantity_a
return fields
def _notify_mismatch(
alarm_emitter: AlarmEmitter,
notification_store,
*,
identity_key: str,
case_code: str,
instance_key: str,
symbol: str,
quantity_a: int,
quantity_b: int,
slot_id: str | None = None,
) -> None:
"""Publish detected or changed alarm; advance fingerprint only on publish success.
Suppresses if the identity fingerprint is unchanged from the last notified state.
Retries on every detection if publish previously failed (fingerprint not advanced).
"""
try:
state = notification_store.get_state(identity_key)
except Exception: # noqa: BLE001
_log.exception(
"reconcile_notification: get_state failed for %s — skipping notify", identity_key,
)
return
if state is not None and state.get("lifecycle_state") == "notified":
if state.get("quantity_a") == quantity_a and state.get("quantity_b") == quantity_b:
return # unchanged — suppress
event_type = "changed"
else:
# No prior state, OR pending_resolved (new mismatch after failed resolved) → re-detect
event_type = "detected"
fields = _build_alarm_fields(
case_code=case_code,
instance_key=instance_key,
symbol=symbol,
quantity_a=quantity_a,
quantity_b=quantity_b,
slot_id=slot_id,
)
published = alarm_emitter.publish_alarm(
f"reconcile_mismatch_{event_type}",
f"reconciliation mismatch {event_type}: {case_code}-case symbol={symbol}",
fields=fields,
)
if published:
try:
notification_store.set_state(
identity_key,
case_code=case_code,
instance_key=instance_key,
symbol=symbol,
quantity_a=quantity_a,
quantity_b=quantity_b,
lifecycle_state="notified",
)
except Exception: # noqa: BLE001
# Fingerprint write failure: at-least-once duplicate on next detection.
_log.exception(
"reconcile_notification: fingerprint persist failed for %s — "
"at-least-once duplicate possible on next detection",
identity_key,
)
def _notify_resolved(
alarm_emitter: AlarmEmitter,
notification_store,
*,
identity_key: str,
case_code: str,
instance_key: str,
symbol: str,
slot_id: str | None = None,
) -> None:
"""Publish resolved alarm; clear or mark pending_resolved on publish failure.
Warning clear is always preserved by the caller regardless of this function's outcome.
- Publish success: clear notification state.
- Publish failure + state writable: mark pending_resolved for retry.
- State persistence failure: log loss risk (business path unaffected).
"""
try:
state = notification_store.get_state(identity_key)
except Exception: # noqa: BLE001
_log.exception(
"reconcile_notification: get_state failed for %s — skipping resolved notify",
identity_key,
)
return
if state is None:
return # idempotent: already resolved or never detected
quantity_a = state["quantity_a"]
quantity_b = state["quantity_b"]
fields = _build_alarm_fields(
case_code=case_code,
instance_key=instance_key,
symbol=symbol,
quantity_a=quantity_a,
quantity_b=quantity_b,
slot_id=slot_id,
)
published = alarm_emitter.publish_alarm(
"reconcile_mismatch_resolved",
f"reconciliation mismatch resolved: {case_code}-case symbol={symbol}",
fields=fields,
)
if published:
try:
notification_store.clear_state(identity_key)
except Exception: # noqa: BLE001
# At-least-once: retry may produce duplicate resolved event.
_log.exception(
"reconcile_notification: clear_state failed for %s — "
"at-least-once duplicate resolved possible",
identity_key,
)
else:
# Publish failed — persist pending_resolved so the next opportunity retries.
try:
notification_store.set_state(
identity_key,
case_code=case_code,
instance_key=instance_key,
symbol=symbol,
quantity_a=quantity_a,
quantity_b=quantity_b,
lifecycle_state="pending_resolved",
)
except Exception: # noqa: BLE001
# State persistence failed: warning clear is kept; resolved event may be lost.
_log.error(
"reconcile_notification: resolved publish failed AND state persist failed "
"for %s — resolved event may be lost",
identity_key,
)
def _retry_pending_resolved(
alarm_emitter: AlarmEmitter,
notification_store,
*,
case_code: str,
instance_key: str,
active_identity_keys: set[str],
) -> None:
"""Retry pending_resolved publishes for identities no longer actively mismatched.
Called at the end of each reconcile pass to drain the pending_resolved queue
for identities that have since been cleared by operator sync/verify.
Only processes states belonging to ``instance_key`` so that a shared
PostgreSQL store used by multiple scheduler instances does not let instance A
re-publish resolved events for instance B's pending states.
"""
try:
pending = notification_store.list_states_by_lifecycle("pending_resolved")
except Exception: # noqa: BLE001
_log.exception(
"reconcile_notification: list_states_by_lifecycle failed — skipping pending retry",
)
return
for state in pending:
if state.get("case_code") != case_code:
continue
if state.get("instance_key") != instance_key:
continue # belongs to a different scheduler instance — do not touch
identity_key = state["identity_key"]
if identity_key in active_identity_keys:
continue # still mismatched — do not retry resolved yet
_notify_resolved(
alarm_emitter,
notification_store,
identity_key=identity_key,
case_code=state["case_code"],
instance_key=state["instance_key"],
symbol=state["symbol"],
slot_id=identity_key if case_code == "B" else None,
)
def reconcile_core_holdings(
*,
slot_repo,
slot_service,
broker: BrokerPort,
instance_key: str,
alarm_emitter: Optional[AlarmEmitter] = None,
warn_store: Optional[ReconcileWarningStore] = None,
alarm_emitter: AlarmEmitter | None = None,
warn_store: ReconcileWarningStore | None = None,
notification_store=None,
) -> dict[str, str]:
"""dcy-020 / REQ-062 (BAT-029) + REQ-103 (BAT-050): surface core-slot holding
vs broker divergence and set durable reconcile-warning for B case.
@@ -122,6 +367,7 @@ def reconcile_core_holdings(
→ set by reconcile_positions
warn_store: Optional — when None, WARN-log-only (backward compat, REQ-103 AC-5).
notification_store: Optional — when None, alarm deduplication is skipped (backward compat).
Returns {key: 'error'|'qty_warning'|'ok'|'unmanaged_position'|
'managed_non_holding_divergence'} for telemetry.
"""
@@ -130,6 +376,7 @@ def reconcile_core_holdings(
}
decisions: dict[str, str] = {}
matched: set[str] = set()
active_b_identity_keys: set[str] = set() # req-0007: track active B mismatches for retry
for slot in slot_repo.list_holding_slots(instance_key):
sym = slot.symbol
matched.add(sym)
@@ -157,6 +404,7 @@ def reconcile_core_holdings(
elif slot.qty is not None and int(slot.qty) != bq: # B case (REQ-103) — qty mismatch
slot_id_str = str(slot.slot_id)
decisions[slot_id_str] = "qty_warning"
active_b_identity_keys.add(slot_id_str)
# REQ-107: structured WARNING log for B case.
_log.warning(
"reconcile_warning set: slot_id=%s portfolio_id=%s symbol=%s "
@@ -177,6 +425,24 @@ def reconcile_core_holdings(
_log.exception(
"failed to set reconcile_warning for slot %s", slot.slot_id
)
# req-0007 / ac-0024: publish B-case detected/changed alarm (deduped).
if alarm_emitter is not None and notification_store is not None:
try:
_notify_mismatch(
alarm_emitter,
notification_store,
identity_key=slot_id_str,
case_code="B",
instance_key=instance_key,
symbol=sym,
quantity_a=int(slot.qty),
quantity_b=bq,
slot_id=slot_id_str,
)
except Exception: # noqa: BLE001
_log.exception(
"reconcile_core_holdings: notify_mismatch failed for slot=%s", slot_id_str,
)
else:
decisions[str(slot.slot_id)] = "ok"
for sym, bq in broker_qty.items(): # unmatched broker position
@@ -196,6 +462,18 @@ def reconcile_core_holdings(
"unmanaged position: symbol=%s qty=%d instance_key=%s",
sym, bq, instance_key,
)
# req-0007 / ac-0024: retry pending_resolved for B-case identities no longer mismatched.
if alarm_emitter is not None and notification_store is not None:
try:
_retry_pending_resolved(
alarm_emitter,
notification_store,
case_code="B",
instance_key=instance_key,
active_identity_keys=active_b_identity_keys,
)
except Exception: # noqa: BLE001
_log.exception("reconcile_core_holdings: pending_resolved retry failed")
return decisions
@@ -205,7 +483,9 @@ def reconcile_positions(
broker: BrokerPort,
slot_repo=None,
instance_key: str = "default",
warn_store: Optional[ReconcileWarningStore] = None,
warn_store: ReconcileWarningStore | None = None,
alarm_emitter: AlarmEmitter | None = None,
notification_store=None,
) -> dict[str, str]:
"""Run one reconciliation pass (REQ-031 + REQ-103 BAT-050).
@@ -216,6 +496,10 @@ def reconcile_positions(
(ledger/hub qty vs broker) also set durable reconcile-warning on the corresponding
slot (portfolio_id -> slot lookup via slot_repo). When either is None, falls back
to in-memory warning only (backward compat, REQ-103 AC-5).
req-0007 / ac-0024: when alarm_emitter and notification_store are provided,
C-case detected/changed alarms are published per-symbol (one aggregate event
per symbol regardless of portfolio count).
"""
broker_qty: dict[str, int] = {
item.symbol: int(item.hldg_qty) for item in broker.get_balance()
@@ -230,6 +514,7 @@ def reconcile_positions(
by_symbol.setdefault(sym, []).append((a.item.portfolio_id, a.item.asset_quantity))
decisions: dict[str, str] = {}
active_c_identity_keys: set[str] = set() # req-0007: track active C mismatches for retry
for symbol, entries in by_symbol.items():
hub_total = int(sum(qty for _, qty in entries))
broker_total = broker_qty.get(symbol, 0)
@@ -277,6 +562,42 @@ def reconcile_positions(
decisions[pid] = "cleared"
else:
decisions[pid] = "ok"
# req-0007 / ac-0024: publish one C-case detected/changed alarm per symbol
# (aggregate mismatch lifecycle — not per portfolio).
if alarm_emitter is not None and notification_store is not None:
c_key = _c_identity_key(instance_key, symbol)
if not match:
active_c_identity_keys.add(c_key)
try:
_notify_mismatch(
alarm_emitter,
notification_store,
identity_key=c_key,
case_code="C",
instance_key=instance_key,
symbol=symbol,
quantity_a=hub_total,
quantity_b=broker_total,
)
except Exception: # noqa: BLE001
_log.exception(
"reconcile_positions: notify_mismatch failed for symbol=%s", symbol,
)
# req-0007 / ac-0024: retry pending_resolved for C-case identities no longer mismatched.
if alarm_emitter is not None and notification_store is not None:
try:
_retry_pending_resolved(
alarm_emitter,
notification_store,
case_code="C",
instance_key=instance_key,
active_identity_keys=active_c_identity_keys,
)
except Exception: # noqa: BLE001
_log.exception("reconcile_positions: pending_resolved retry failed")
return decisions
@@ -291,9 +612,11 @@ def sync_slot_to_broker(
slot_service,
broker: BrokerPort,
instance_key: str,
warn_store: "ReconcileWarningStore",
warn_store: ReconcileWarningStore,
cost_basis_port=None, # Optional[PortfolioHubCostBasisPort]
dry_run: bool = False,
alarm_emitter: AlarmEmitter | None = None,
notification_store=None,
) -> dict:
"""Sync one slot's qty to the broker ground truth (REQ-105, BAT-051).
@@ -373,9 +696,9 @@ def sync_slot_to_broker(
avg_price = slot.avg_price
seeded_avg_price = False
cost_basis_source = "none"
cost_basis_reason: Optional[str] = None
cost_basis_reason: str | None = None
def _set_cost_basis_unavailable_message() -> tuple[str, Optional[str]]:
def _set_cost_basis_unavailable_message() -> tuple[str, str | None]:
if pid:
return "cost_basis_unavailable", f"inactive slot {slot_id} portfolio_id={pid}"
return "cost_basis_unavailable", "inactive slot without portfolio_id"
@@ -559,6 +882,22 @@ def sync_slot_to_broker(
)
except Exception: # noqa: BLE001
_log.exception("sync: failed to clear_warned for slot=%s", slot_id)
# req-0007 / ac-0024: publish B-case resolved alarm only after warning is cleared.
if warning_cleared and alarm_emitter is not None and notification_store is not None:
try:
_notify_resolved(
alarm_emitter,
notification_store,
identity_key=slot_id,
case_code="B",
instance_key=instance_key,
symbol=sym,
slot_id=slot_id,
)
except Exception: # noqa: BLE001
_log.exception(
"sync: resolved notify failed for slot=%s — warning clear kept", slot_id,
)
return {
"slot_id": slot_id, "symbol": sym, "portfolio_id": pid,
@@ -583,7 +922,9 @@ def verify_slot_reconcile(
broker: BrokerPort,
slot_repo,
instance_key: str,
warn_store: "ReconcileWarningStore",
warn_store: ReconcileWarningStore,
alarm_emitter: AlarmEmitter | None = None,
notification_store=None,
) -> dict:
"""Re-check C-case reconcile-warning for a portfolio (REQ-106, BAT-051).
@@ -648,31 +989,66 @@ def verify_slot_reconcile(
}
# Re-run the C-case check: hub qty vs broker qty for this symbol.
# AC-0023: use symbol-level aggregate across all portfolios (not single portfolio).
sym = slot.symbol or ""
broker_qty_map: dict[str, int] = {
item.symbol: int(item.hldg_qty) for item in broker.get_balance()
}
broker_qty = broker_qty_map.get(sym, 0)
# Hub qty: sum asset_quantity for this portfolio_id from snapshot.
# Hub qty: sum asset_quantity for ALL portfolios with this symbol in the snapshot.
hub_qty: int = 0
snap = store.get(portfolio_id)
if snap is not None:
hub_qty = int(snap.item.asset_quantity)
for a in store.list_assets():
if a.item.symbol == sym:
hub_qty += int(a.item.asset_quantity)
match = hub_qty == broker_qty
if match:
# AC-0023: clear C-case warnings for ALL slots (any status) with this symbol.
# Only rows with case_code='C' are touched; B-case (or other) rows are preserved.
cleared_slots: list[str] = []
try:
warn_store.clear_warned(slot_id_str)
store.set_warning(portfolio_id, False)
for s in slot_repo.list_slots(instance_key):
if s.symbol != sym:
continue
sid_str = str(s.slot_id)
warn_info = warn_store.get_warning_info(sid_str)
if warn_info is None or warn_info.get("case_code") != "C":
continue
warn_store.clear_warned(sid_str)
store.set_warning(s.portfolio_id, False)
cleared_slots.append(sid_str)
_log.info(
"verify: reconcile-warning cleared (C-case resolved) "
"slot=%s portfolio_id=%s symbol=%s hub_qty=%d broker_qty=%d",
slot_id_str, portfolio_id, sym, hub_qty, broker_qty,
"symbol=%s hub_qty=%d broker_qty=%d cleared_slots=%s",
sym, hub_qty, broker_qty, cleared_slots,
)
except Exception: # noqa: BLE001
_log.exception("verify: failed to clear_warned for slot=%s", slot_id_str)
_log.exception("verify: failed to clear C-case warned for symbol=%s", sym)
return {
"portfolio_id": portfolio_id,
"slot_id": slot_id_str,
"result": "clear_failed",
"hub_qty": hub_qty,
"broker_qty": broker_qty,
"reason": "warn_store error during C-case clear — warnings NOT fully cleared",
}
# req-0007 / ac-0024: publish C-case resolved alarm (best-effort, warning clear kept).
if alarm_emitter is not None and notification_store is not None:
try:
_notify_resolved(
alarm_emitter,
notification_store,
identity_key=_c_identity_key(instance_key, sym),
case_code="C",
instance_key=instance_key,
symbol=sym,
)
except Exception: # noqa: BLE001
_log.exception(
"verify: resolved notify failed for symbol=%s — warning clear kept", sym,
)
return {
"portfolio_id": portfolio_id,
"slot_id": slot_id_str,
@@ -22,6 +22,7 @@ from trex_core.application.portfolio import (
DegradedReason,
PortfolioHubReadPort,
PortfolioHubUnavailableError,
ReconcileNotificationStore,
ReconcileWarningStore,
SnapshotStore,
TradingContext,
@@ -49,6 +50,7 @@ class SnapshotScheduler:
instance_key: str = "default",
alarm_emitter: Optional[AlarmEmitter] = None,
warn_store: Optional[ReconcileWarningStore] = None,
notification_store: Optional[ReconcileNotificationStore] = None,
credential_fetch_enabled: bool = False,
kis_reinit_callback: Optional[Callable[[BrokerCredentialPayload], None]] = None,
credential_refresh: Optional[Callable[[], object]] = None,
@@ -71,6 +73,8 @@ class SnapshotScheduler:
self._alarm_emitter = alarm_emitter
# BAT-050 / REQ-103: durable reconcile-warning store (Optional).
self._warn_store = warn_store
# req-0007 / ac-0024: durable notification deduplication store (Optional).
self._notification_store = notification_store
# dcy-036 / REQ-113/114/115: credential fetch opt-in + break-glass detection.
# credential_fetch_enabled=False (default) → env-only, no portfolio-hub fetch.
self._credential_fetch_enabled: bool = credential_fetch_enabled
@@ -312,6 +316,8 @@ class SnapshotScheduler:
slot_repo=self._slot_repo,
instance_key=self._instance_key,
warn_store=self._warn_store,
alarm_emitter=self._alarm_emitter,
notification_store=self._notification_store,
)
except Exception: # noqa: BLE001
_log.exception("reconciliation pass failed")
@@ -329,6 +335,7 @@ class SnapshotScheduler:
broker=self._broker, instance_key=self._instance_key,
alarm_emitter=self._alarm_emitter, # dcy-033 / REQ-100
warn_store=self._warn_store, # BAT-050 / REQ-103
notification_store=self._notification_store, # req-0007 / ac-0024
)
except Exception: # noqa: BLE001
_log.exception("core-holding divergence reconcile failed")
+671 -79
View File
@@ -11,6 +11,9 @@ Subcommands:
cancel <slot_id> — cancel a pending order (control RPC; safeguarded)
sync — sync slot qty to broker ground truth (direct-DB; REQ-105)
verify — re-check C-case reconcile-warning (direct-DB; REQ-106)
ledger backfill — evidence-backed historical ledger recovery
ledger skips list — query durable skipped-ledger records
ledger skips resolve — explicitly resolve a skipped-ledger record
universe/slots/enter/exit go through the daemon's RabbitMQ RPC surface so the
single-writer admission + invariants are preserved (no direct slot writes,
@@ -26,18 +29,28 @@ sync safeguards (REQ-105): `--yes` required (skip confirmation prompt);
live accounts require `--yes --live` (double ack); `--dry-run` previews
before/after without writing. sync is direct-DB (no RPC).
verify (REQ-106): read-centric (no --yes/--live gate needed).
ledger backfill: verifies KIS daily-ccld evidence, reconstructs one ledger
trade, enqueues it through ledger_outbox, immediately drives the
LedgerWriter path, and resolves matching durable skipped-ledger records.
"""
from __future__ import annotations
import argparse
import json as _json
import logging
import re
import sys
import uuid
from datetime import datetime, timedelta, timezone
from datetime import UTC, datetime, timedelta, timezone
from pathlib import Path
from zoneinfo import ZoneInfo
from trex_core.adapters.messaging import topology
from trex_core.adapters.portfolio.pause_store import PostgreSQLPauseStore
from trex_core.application.alarm_emitter import AlarmEmitter
from trex_core.application.ledger_outbox import LedgerWriter, OutboxOutcome
from trex_core.application.ledger_port import LedgerTrade
from trex_core.application.ledger_skip import ledger_skip_alarm_fields, publish_ledger_skip_alarm
from trex_core.runtime.config import (
DaemonConfig,
DaemonInstanceConfig,
@@ -46,6 +59,9 @@ from trex_core.runtime.config import (
)
_DEFAULT_TIMEOUT = 10.0
_DEFAULT_LEDGER_BACKFILL_AUDIT_DIR = "artifacts/validation/ledger_backfill"
_DEFAULT_LEDGER_SKIP_RESOLVE_AUDIT_DIR = "artifacts/validation/ledger_skip_resolve"
_log = logging.getLogger(__name__)
# --------------------------------------------------------------------------- #
@@ -82,6 +98,14 @@ def _default_rpc(cfg: DaemonConfig):
return _rpc
def _default_alarm_emitter(cfg: DaemonConfig):
from trex_core.adapters.messaging import InMemoryTransport, RabbitMQTransport
mq = getattr(cfg, "rabbitmq", None)
transport = RabbitMQTransport(mq.url, timeout=mq.timeout) if mq else InMemoryTransport()
return AlarmEmitter(transport), transport
def _default_confirm(prompt: str) -> bool:
try:
return input(f"{prompt} [y/N] ").strip().lower() in ("y", "yes")
@@ -101,6 +125,10 @@ def _emit(data: dict, *, as_json: bool) -> None:
print(_json.dumps(data, ensure_ascii=False, default=str))
def _emit_pretty_json(data: dict) -> None:
print(_json.dumps(data, ensure_ascii=False, indent=2, default=str))
# --------------------------------------------------------------------------- #
# RPC subcommand handlers
# --------------------------------------------------------------------------- #
@@ -124,7 +152,8 @@ def _cmd_universe(args, cfg, rpc) -> int:
f"degraded={resp.get('degraded')}")
for s in slots:
print(f" {s.get('portfolio_id')}\t{s.get('symbol')}\t"
f"qty(hub)={s.get('asset_quantity')}\tqty(slot)={s.get('qty')}")
f"qty(hub)={s.get('asset_quantity')}\t"
f"qty(slot)={s.get('slot_quantity', s.get('qty'))}")
return 0
@@ -164,7 +193,8 @@ def _cmd_slots(args, cfg, rpc) -> int:
wd = s.get("watchdog") or {}
tag = " [WATCHDOG: strategy absent]" if wd.get("expired") else ""
print(f" {s.get('slot_id')}\t{s.get('status')}\t{s.get('symbol')}\t"
f"qty={s.get('qty')}\taqty={s.get('asset_quantity')}\t"
f"qty={s.get('qty')}\tsqty={s.get('slot_quantity')}\t"
f"aqty={s.get('asset_quantity')}\t"
f"limit={s.get('entry_price')}\tavg={s.get('avg_price')}\t"
f"pid={pid}\tpaused={s.get('paused')}\twarned={s.get('warned')}{tag}")
return 0
@@ -177,7 +207,11 @@ def _print_slot_detail(s: dict) -> None:
("status", s.get("status")),
("symbol", s.get("symbol")),
("portfolio_id", s.get("portfolio_id")),
("qty (slot mirror)", s.get("qty")),
("qty (authoritative)", s.get("qty")),
("slot_quantity (core mirror)", s.get("slot_quantity")),
("qty_source", s.get("qty_source")),
("broker_quantity_available", s.get("broker_quantity_available")),
("broker_quantity_captured_at", s.get("broker_quantity_captured_at")),
("asset_quantity (hub)", s.get("asset_quantity")),
("warned", s.get("warned")),
("avg_price", s.get("avg_price")),
@@ -352,6 +386,148 @@ class _CachedBalanceBroker:
return getattr(self._broker, name)
def _parse_iso_date(raw: str) -> str:
return datetime.fromisoformat(f"{raw}T00:00:00+00:00").date().isoformat()
def _sanitize_path_component(value: str, *, limit: int = 80) -> str:
cleaned = re.sub(r"[^A-Za-z0-9._-]+", "_", str(value or "").strip()).strip("._")
return (cleaned or "value")[:limit]
def _ledger_backfill_audit_path(args, execution_id: str) -> Path:
base = Path(getattr(args, "audit_dir", _DEFAULT_LEDGER_BACKFILL_AUDIT_DIR))
stamp = datetime.now(tz=UTC).strftime("%Y%m%dT%H%M%SZ")
name = (
f"{stamp}--{_sanitize_path_component(_resolve_instance(args))}"
f"--{_sanitize_path_component(execution_id)}.json"
)
return base.resolve() / name
def _write_ledger_backfill_audit(args, report: dict) -> str:
path = _ledger_backfill_audit_path(args, report["request"]["execution_id"])
report["audit_path"] = str(path)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(_json.dumps(report, ensure_ascii=False, indent=2, default=str) + "\n",
encoding="utf-8")
return str(path)
def _ledger_skip_resolve_audit_path(args, skip_key: str) -> Path:
base = Path(getattr(args, "audit_dir", _DEFAULT_LEDGER_SKIP_RESOLVE_AUDIT_DIR))
stamp = datetime.now(tz=UTC).strftime("%Y%m%dT%H%M%SZ")
name = (
f"{stamp}--{_sanitize_path_component(_resolve_instance(args))}"
f"--{_sanitize_path_component(skip_key)}.json"
)
return base.resolve() / name
def _write_ledger_skip_resolve_audit(args, report: dict) -> str:
path = _ledger_skip_resolve_audit_path(args, report["request"]["skip_key"])
report["audit_path"] = str(path)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(_json.dumps(report, ensure_ascii=False, indent=2, default=str) + "\n",
encoding="utf-8")
return str(path)
def _lookup_ledger_backfill_evidence(cfg: DaemonConfig, *, trading_date: str, execution_id: str):
from trex_core.runtime.broker_provider import build_broker_provider
broker, source = build_broker_provider(cfg).get_broker_for_read()
lookup = getattr(broker, "lookup_fill_evidence", None)
if not callable(lookup):
raise TypeError("configured broker does not support KIS ledger backfill evidence lookup")
evidence = lookup(trading_date=trading_date, execution_id=execution_id)
return evidence, source
def _resolve_ledger_backfill_slot(dsn: str, *, instance: str, symbol: str) -> dict:
import psycopg
with psycopg.connect(dsn) as conn, conn.cursor() as cur:
cur.execute(
"SELECT slot_id::text, portfolio_id, status "
"FROM daily_slot "
"WHERE instance_key = %s AND symbol = %s "
"AND portfolio_id IS NOT NULL AND retired_at IS NULL "
"ORDER BY updated_at DESC NULLS LAST, slot_id ASC "
"LIMIT 2",
(instance, symbol),
)
rows = cur.fetchall()
if not rows:
raise RuntimeError(f"no daemon-owned slot found for instance={instance} symbol={symbol}")
if len(rows) != 1:
raise RuntimeError(f"ambiguous daemon-owned slot mapping for instance={instance} symbol={symbol}")
return {
"slot_id": str(rows[0][0]),
"portfolio_id": str(rows[0][1]),
"status": str(rows[0][2]),
"symbol": symbol,
}
def _lookup_outbox_by_execution_id(dsn: str, *, execution_id: str) -> list[dict]:
import psycopg
with psycopg.connect(dsn) as conn, conn.cursor() as cur:
cur.execute(
"SELECT correlation_id, asset_id, order_id, execution_id, status, "
"attempts, last_outcome, last_error "
"FROM ledger_outbox WHERE execution_id = %s "
"ORDER BY created_at, correlation_id",
(execution_id,),
)
rows = cur.fetchall()
return [
{
"correlation_id": str(cid),
"asset_id": str(asset_id),
"order_id": str(order_id),
"execution_id": str(exec_id),
"status": str(status),
"attempts": int(attempts or 0),
"last_outcome": str(last_outcome) if last_outcome else None,
"last_error": str(last_error) if last_error else None,
}
for cid, asset_id, order_id, exec_id, status, attempts, last_outcome, last_error in rows
]
def _normalize_existing_execution(existing_rows: list[dict], trade: LedgerTrade) -> dict | None:
if not existing_rows:
return None
if len(existing_rows) != 1:
raise RuntimeError(f"execution_id={trade.execution_id} already maps to multiple outbox rows")
existing = existing_rows[0]
if existing["correlation_id"] != trade.correlation_id:
raise RuntimeError(
f"execution_id={trade.execution_id} already belongs to {existing['correlation_id']}"
)
return existing
def _ledger_outcome_result(outcome: OutboxOutcome | None, existing_status: str | None = None) -> tuple[str, int]:
if outcome is OutboxOutcome.COMPLETED:
return "completed", 0
if outcome is OutboxOutcome.DUPLICATE_COMPLETED:
return "duplicate_completed", 0
if outcome is OutboxOutcome.RETRYABLE_FAILURE:
return "retryable_failure", 1
if outcome is OutboxOutcome.CONFLICT_BLOCKED:
return "conflict_blocked", 1
if outcome in {OutboxOutcome.PERMANENT_FAILURE, OutboxOutcome.DEAD_LETTERED}:
return "delivery_failed", 1
if existing_status == "completed":
return "already_recorded", 0
if existing_status:
return f"already_present:{existing_status}", 0
return "no_action", 1
# --------------------------------------------------------------------------- #
# arg parsing
# --------------------------------------------------------------------------- #
@@ -464,6 +640,64 @@ def _build_parser() -> argparse.ArgumentParser:
)
_add_common(p_verify)
p_ledger = sub.add_parser(
"ledger",
help="Ledger operator tools (direct-DB enqueue + immediate writer path)",
)
p_ledger_sub = p_ledger.add_subparsers(dest="ledger_cmd", required=True)
p_backfill = p_ledger_sub.add_parser(
"backfill",
help="Evidence-backed historical ledger backfill (KIS daily-ccld)",
)
p_backfill.add_argument("--date", required=True, help="KST trading date (YYYY-MM-DD)")
p_backfill.add_argument("--execution-id", dest="execution_id", required=True)
p_backfill.add_argument("--yes", action="store_true", help="skip confirmation")
p_backfill.add_argument("--live", action="store_true",
help="confirm a LIVE ledger mutation (with --yes)")
p_backfill.add_argument("--dry-run", dest="dry_run", action="store_true",
help="preview the exact ledger trade, do not write")
p_backfill.add_argument(
"--audit-dir",
default=_DEFAULT_LEDGER_BACKFILL_AUDIT_DIR,
help="directory for persistent JSON audit reports",
)
_add_common(p_backfill)
p_skips = p_ledger_sub.add_parser(
"skips",
help="Query or resolve durable skipped-ledger records",
)
p_skips_sub = p_skips.add_subparsers(dest="ledger_skips_cmd", required=True)
p_skips_list = p_skips_sub.add_parser(
"list",
help="List skipped-ledger records by instance/date/portfolio/state",
)
p_skips_list.add_argument("--date", default=None, help="UTC date filter (YYYY-MM-DD)")
p_skips_list.add_argument("--portfolio-id", dest="portfolio_id", default=None)
p_skips_list.add_argument(
"--status",
choices=("unresolved", "resolved", "all"),
default="unresolved",
)
p_skips_list.add_argument("--limit", type=int, default=100)
_add_common(p_skips_list)
p_skips_resolve = p_skips_sub.add_parser(
"resolve",
help="Mark one skipped-ledger record resolved and emit a resolved alert",
)
p_skips_resolve.add_argument("--skip-key", required=True)
p_skips_resolve.add_argument("--reason", default="operator_reconciled")
p_skips_resolve.add_argument("--yes", action="store_true", help="skip confirmation")
p_skips_resolve.add_argument("--live", action="store_true",
help="confirm a LIVE ledger skip mutation (with --yes)")
p_skips_resolve.add_argument(
"--audit-dir",
default=_DEFAULT_LEDGER_SKIP_RESOLVE_AUDIT_DIR,
help="directory for persistent JSON audit reports",
)
_add_common(p_skips_resolve)
return parser
@@ -473,6 +707,9 @@ def _cmd_sync(args, cfg, confirm) -> int:
from trex_core.adapters.portfolio.reconcile_warning_store import (
PostgreSQLReconcileWarningStore,
)
from trex_core.adapters.portfolio.reconcile_notification_store import (
PostgreSQLReconcileNotificationStore,
)
from trex_core.application.reconciliation import sync_slot_to_broker
if not args.portfolio_id and not args.all:
@@ -499,10 +736,11 @@ def _cmd_sync(args, cfg, confirm) -> int:
return 1
# REQ-119/120: resolve and validate broker truth before opening/enumerating DB.
from trex_core.runtime.broker_provider import (
BrokerCredentialUnavailable, build_broker_provider,
)
from trex_core.application.broker_port import BrokerBalanceError
from trex_core.runtime.broker_provider import (
BrokerCredentialUnavailable,
build_broker_provider,
)
try:
broker_port, _source = build_broker_provider(cfg).get_broker_for_read()
broker_port = _CachedBalanceBroker(broker_port, broker_port.get_balance())
@@ -510,6 +748,9 @@ def _cmd_sync(args, cfg, confirm) -> int:
return _broker_read_failure(exc, args.json)
warn_store = PostgreSQLReconcileWarningStore(cfg.dsn)
notification_store = PostgreSQLReconcileNotificationStore(cfg.dsn)
# req-0007 / ac-0024: alarm emitter for resolved notifications.
alarm_emitter_sync, _alarm_transport_sync = _default_alarm_emitter(cfg)
# Build slot_repo + slot_service
from trex_core.adapters.persistence.postgresql import PostgreSQLSlotRepository
@@ -529,14 +770,13 @@ def _cmd_sync(args, cfg, confirm) -> int:
# Determine target slot_ids
import psycopg
if args.portfolio_id:
with psycopg.connect(cfg.dsn) as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT slot_id FROM daily_slot "
"WHERE instance_key = %s AND portfolio_id = %s LIMIT 1",
(instance, args.portfolio_id),
)
row = cur.fetchone()
with psycopg.connect(cfg.dsn) as conn, conn.cursor() as cur:
cur.execute(
"SELECT slot_id FROM daily_slot "
"WHERE instance_key = %s AND portfolio_id = %s LIMIT 1",
(instance, args.portfolio_id),
)
row = cur.fetchone()
if row is None:
_fail(f"no slot found for portfolio_id={args.portfolio_id} instance={instance}", args.json)
return 1
@@ -544,34 +784,38 @@ def _cmd_sync(args, cfg, confirm) -> int:
else:
# --all: target all warned slots (B case) plus all inactive slots with broker holding
warned = warn_store.list_warned()
with psycopg.connect(cfg.dsn) as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT slot_id FROM daily_slot "
"WHERE instance_key = %s AND status IN ('inactive', 'holding')",
(instance,),
)
all_ids = {str(r[0]) for r in cur.fetchall()}
with psycopg.connect(cfg.dsn) as conn, conn.cursor() as cur:
cur.execute(
"SELECT slot_id FROM daily_slot "
"WHERE instance_key = %s AND status IN ('inactive', 'holding')",
(instance,),
)
all_ids = {str(r[0]) for r in cur.fetchall()}
slot_ids = sorted(all_ids | warned)
results = []
exit_code = 0
for sid in slot_ids:
res = sync_slot_to_broker(
slot_id=sid,
slot_repo=slot_repo,
slot_service=slot_service,
broker=broker_port,
instance_key=instance,
warn_store=warn_store,
cost_basis_port=cost_basis_port,
dry_run=dry_run,
)
results.append(res)
# A skipped result can still be a fail-closed live error (REQ-123).
# Do not let the no-write marker hide a failed sync from automation.
if res.get("error"):
exit_code = 1
try:
for sid in slot_ids:
res = sync_slot_to_broker(
slot_id=sid,
slot_repo=slot_repo,
slot_service=slot_service,
broker=broker_port,
instance_key=instance,
warn_store=warn_store,
cost_basis_port=cost_basis_port,
dry_run=dry_run,
alarm_emitter=alarm_emitter_sync,
notification_store=notification_store,
)
results.append(res)
# A skipped result can still be a fail-closed live error (REQ-123).
# Do not let the no-write marker hide a failed sync from automation.
if res.get("error"):
exit_code = 1
finally:
_alarm_transport_sync.close()
if args.json:
print(_json.dumps({"instance": instance, "dry_run": dry_run, "results": results},
@@ -617,15 +861,19 @@ def _cmd_verify(args, cfg) -> int:
from trex_core.adapters.portfolio.reconcile_warning_store import (
PostgreSQLReconcileWarningStore,
)
from trex_core.adapters.portfolio.reconcile_notification_store import (
PostgreSQLReconcileNotificationStore,
)
from trex_core.application.portfolio import SnapshotStore
from trex_core.application.reconciliation import verify_slot_reconcile
instance = _resolve_instance(args)
# REQ-120 no-write/no-enumeration barrier also applies to verify.
from trex_core.runtime.broker_provider import (
BrokerCredentialUnavailable, build_broker_provider,
)
from trex_core.application.broker_port import BrokerBalanceError
from trex_core.runtime.broker_provider import (
BrokerCredentialUnavailable,
build_broker_provider,
)
try:
broker_port, _source = build_broker_provider(cfg).get_broker_for_read()
broker_port = _CachedBalanceBroker(broker_port, broker_port.get_balance())
@@ -633,6 +881,9 @@ def _cmd_verify(args, cfg) -> int:
return _broker_read_failure(exc, args.json)
warn_store = PostgreSQLReconcileWarningStore(cfg.dsn)
notification_store = PostgreSQLReconcileNotificationStore(cfg.dsn)
# req-0007 / ac-0024: alarm emitter for resolved notifications.
alarm_emitter_verify, _alarm_transport_verify = _default_alarm_emitter(cfg)
slot_repo = PostgreSQLSlotRepository(cfg.dsn)
# Build a minimal snapshot store (we only need asset_quantity for C-case check).
@@ -652,7 +903,7 @@ def _cmd_verify(args, cfg) -> int:
items = hub.get_portfolios(daemon_id)
store.swap(items)
except Exception: # noqa: BLE001 — degrade: verify with empty store
pass
_log.info("verify snapshot preload skipped", exc_info=True)
if args.portfolio_id:
target_pids = [args.portfolio_id]
@@ -675,18 +926,23 @@ def _cmd_verify(args, cfg) -> int:
results = []
exit_code = 0
for pid in target_pids:
res = verify_slot_reconcile(
portfolio_id=pid,
store=store,
broker=broker_port,
slot_repo=slot_repo,
instance_key=instance,
warn_store=warn_store,
)
results.append(res)
if res["result"] == "rejected":
exit_code = 1
try:
for pid in target_pids:
res = verify_slot_reconcile(
portfolio_id=pid,
store=store,
broker=broker_port,
slot_repo=slot_repo,
instance_key=instance,
warn_store=warn_store,
alarm_emitter=alarm_emitter_verify,
notification_store=notification_store,
)
results.append(res)
if res["result"] == "rejected":
exit_code = 1
finally:
_alarm_transport_verify.close()
if args.json:
print(_json.dumps({"instance": instance, "results": results},
@@ -735,30 +991,29 @@ def _cmd_report(args, cfg) -> int:
holdings: list[Holding] = []
pending: list[PendingOrder] = []
trades: list[TradeRow] = []
with psycopg.connect(cfg.dsn) as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT symbol, portfolio_id, qty, avg_price FROM daily_slot "
"WHERE status = 'holding' ORDER BY symbol"
)
for symbol, pid, qty, avg in cur.fetchall():
holdings.append(Holding(symbol or "", pid or "",
float(qty or 0), float(avg or 0)))
cur.execute(
"SELECT symbol, portfolio_id, status FROM daily_slot "
"WHERE status IN ('pending_buy','pending_sell') ORDER BY symbol"
)
for symbol, pid, status in cur.fetchall():
pending.append(PendingOrder(symbol or "", pid or "", status))
cur.execute(
"SELECT asset_id, side, qty, price, fee, tax FROM ledger_outbox "
"WHERE COALESCE(executed_at, created_at) >= %s "
"AND COALESCE(executed_at, created_at) < %s",
(start_utc, end_utc),
)
for asset_id, side, qty, price, fee, tax in cur.fetchall():
trades.append(TradeRow(asset_id, side, float(qty), float(price),
float(fee or 0), float(tax or 0)))
with psycopg.connect(cfg.dsn) as conn, conn.cursor() as cur:
cur.execute(
"SELECT symbol, portfolio_id, qty, avg_price FROM daily_slot "
"WHERE status = 'holding' ORDER BY symbol"
)
for symbol, pid, qty, avg in cur.fetchall():
holdings.append(Holding(symbol or "", pid or "",
float(qty or 0), float(avg or 0)))
cur.execute(
"SELECT symbol, portfolio_id, status FROM daily_slot "
"WHERE status IN ('pending_buy','pending_sell') ORDER BY symbol"
)
for symbol, pid, status in cur.fetchall():
pending.append(PendingOrder(symbol or "", pid or "", status))
cur.execute(
"SELECT asset_id, side, qty, price, fee, tax FROM ledger_outbox "
"WHERE COALESCE(executed_at, created_at) >= %s "
"AND COALESCE(executed_at, created_at) < %s",
(start_utc, end_utc),
)
for asset_id, side, qty, price, fee, tax in cur.fetchall():
trades.append(TradeRow(asset_id, side, float(qty), float(price),
float(fee or 0), float(tax or 0)))
rep = build_report(holdings, pending, trades)
@@ -821,6 +1076,328 @@ def _cmd_report(args, cfg) -> int:
return 0
def _render_ledger_skip_record(record) -> dict[str, object]:
return ledger_skip_alarm_fields(record)
def _cmd_ledger_skips_list(args, cfg) -> int:
from trex_core.adapters.persistence.ledger_skip_store import PostgreSQLLedgerSkipStore
instance = _resolve_instance(args)
on_date = _parse_iso_date(args.date) if args.date else None
records = PostgreSQLLedgerSkipStore(cfg.dsn).list_records(
instance_key=instance,
portfolio_id=args.portfolio_id,
on_date=on_date,
status=args.status,
limit=args.limit,
)
payload = {
"instance": instance,
"status": args.status,
"count": len(records),
"records": [_render_ledger_skip_record(record) for record in records],
}
if args.json:
_emit(payload, as_json=True)
else:
print(
f"instance={instance} status={args.status} count={len(records)} "
f"portfolio_id={args.portfolio_id or '*'} date={args.date or '*'}"
)
for record in records:
print(
f" {record.skip_key}\t{record.status}\t{record.skip_reason}\t"
f"slot={record.slot_id}\tportfolio={record.portfolio_id or '-'}\t"
f"execution={record.execution_id or '-'}"
)
return 0
def _cmd_ledger_skips_resolve(args, cfg, confirm) -> int:
from trex_core.adapters.persistence.ledger_skip_store import PostgreSQLLedgerSkipStore
instance = _resolve_instance(args)
broker_name, is_live = _broker_mode(cfg)
report = {
"request": {
"instance": instance,
"broker": broker_name,
"live_account": is_live,
"skip_key": args.skip_key,
"resolution_reason": args.reason,
},
"result": None,
"record": None,
"audit_path": None,
}
if is_live and (not getattr(args, "yes", False) or not getattr(args, "live", False)):
report["result"] = {"status": "rejected", "reason": "live account requires --yes --live"}
report["audit_path"] = _write_ledger_skip_resolve_audit(args, report)
_fail("live account requires --yes --live", args.json)
if args.json:
_emit(report, as_json=True)
else:
_emit_pretty_json(report)
return 2
if not getattr(args, "yes", False):
if args.json:
report["result"] = {"status": "rejected", "reason": "--json ledger skips resolve requires --yes"}
report["audit_path"] = _write_ledger_skip_resolve_audit(args, report)
_fail("--json ledger skips resolve requires --yes", True)
_emit(report, as_json=True)
return 2
prompt = (
f"Resolve skipped ledger record on instance={instance} broker={broker_name}"
f"{' LIVE' if is_live else ''}: {args.skip_key}?"
)
if not confirm(prompt):
report["result"] = {"status": "aborted", "reason": "operator declined confirmation"}
report["audit_path"] = _write_ledger_skip_resolve_audit(args, report)
print("aborted")
_emit_pretty_json(report)
return 1
store = PostgreSQLLedgerSkipStore(cfg.dsn)
record = store.resolve_by_key(
args.skip_key,
instance_key=instance,
resolution_reason=args.reason,
)
if record is None:
report["result"] = {"status": "not_found", "reason": f"unresolved ledger skip not found: {args.skip_key}"}
report["audit_path"] = _write_ledger_skip_resolve_audit(args, report)
_fail(f"unresolved ledger skip not found: {args.skip_key}", args.json)
if args.json:
_emit(report, as_json=True)
else:
_emit_pretty_json(report)
return 1
emitter, transport = _default_alarm_emitter(cfg)
try:
publish_ledger_skip_alarm(emitter, record, resolved=True)
finally:
transport.close()
payload = {"result": "resolved", "record": _render_ledger_skip_record(record)}
report["result"] = {"status": "resolved"}
report["record"] = payload["record"]
report["audit_path"] = _write_ledger_skip_resolve_audit(args, report)
if args.json:
_emit({**payload, "audit_path": report["audit_path"]}, as_json=True)
else:
print(f"resolved {record.skip_key} reason={record.resolution_reason}")
return 0
def _cmd_ledger_backfill(args, cfg, confirm) -> int:
from trex_core.adapters.broker.kis import KISEvidenceLookupError
from trex_core.adapters.ledger.portfolio_hub import PortfolioHubLedger
from trex_core.adapters.persistence.ledger_skip_store import PostgreSQLLedgerSkipStore
from trex_core.adapters.persistence.postgresql import PostgreSQLLedgerOutbox
from trex_core.runtime.broker_provider import BrokerCredentialUnavailable
instance = _resolve_instance(args)
broker_name, is_live = _broker_mode(cfg)
if broker_name != "kis":
_fail("ledger backfill is only supported with the KIS broker", args.json)
return 1
try:
trading_date = _parse_iso_date(args.date)
except ValueError:
_fail(f"invalid --date (expected YYYY-MM-DD): {args.date!r}", args.json)
return 1
report = {
"request": {
"instance": instance,
"date": trading_date,
"execution_id": str(args.execution_id),
"dry_run": bool(args.dry_run),
"broker": broker_name,
"live": bool(is_live),
},
"evidence": None,
"slot_binding": None,
"trade": None,
"outbox_precheck": None,
"result": None,
"resolved_skips": [],
"audit_path": None,
}
try:
evidence, broker_source = _lookup_ledger_backfill_evidence(
cfg, trading_date=trading_date.replace("-", ""), execution_id=str(args.execution_id)
)
report["broker_source"] = {
"mode": broker_source.mode,
"identity_label": broker_source.identity_label,
}
report["evidence"] = {
"symbol": evidence["symbol"],
"side": evidence["side"],
"qty": evidence["qty"],
"price": evidence["price"],
"fee": evidence["fee"],
"tax": evidence["tax"],
"order_date": trading_date,
"order_id": evidence["order_id"],
"order_venue": evidence["order_venue"],
"execution_venue": evidence["execution_venue"],
"executed_at": evidence["executed_at"],
"raw": evidence["raw"],
}
slot_binding = _resolve_ledger_backfill_slot(
cfg.dsn, instance=instance, symbol=str(evidence["symbol"])
)
report["slot_binding"] = slot_binding
trade = LedgerTrade(
asset_id=str(slot_binding["portfolio_id"]),
order_id=str(evidence["order_id"]),
execution_id=str(args.execution_id),
side=str(evidence["side"]),
qty=float(evidence["qty"]),
price=float(evidence["price"]),
fee=float(evidence["fee"]),
tax=float(evidence["tax"]),
executed_at=evidence["executed_at"],
venue=evidence["execution_venue"],
order_venue=str(evidence["order_venue"]),
)
report["trade"] = {
"correlation_id": trade.correlation_id,
"asset_id": trade.asset_id,
"order_id": trade.order_id,
"execution_id": trade.execution_id,
"side": trade.side,
"qty": trade.qty,
"price": trade.price,
"fee": trade.fee,
"tax": trade.tax,
"executed_at": trade.executed_at,
"order_venue": trade.order_venue,
"execution_venue": trade.venue,
}
existing = _normalize_existing_execution(
_lookup_outbox_by_execution_id(cfg.dsn, execution_id=trade.execution_id),
trade,
)
report["outbox_precheck"] = {
"execution_id": trade.execution_id,
"existing": existing,
}
except (BrokerCredentialUnavailable, KISEvidenceLookupError, RuntimeError, TypeError) as exc:
report["result"] = {"status": "rejected", "reason": str(exc)}
report["audit_path"] = _write_ledger_backfill_audit(args, report)
if args.json:
_emit(report, as_json=True)
else:
_emit_pretty_json(report)
return 1
if args.dry_run:
report["result"] = {
"status": "dry_run",
"reason": "no writes performed",
}
report["audit_path"] = _write_ledger_backfill_audit(args, report)
if args.json:
_emit(report, as_json=True)
else:
_emit_pretty_json(report)
return 0
if is_live and not getattr(args, "live", False):
report["result"] = {"status": "rejected", "reason": "live account requires --live (with --yes)"}
report["audit_path"] = _write_ledger_backfill_audit(args, report)
_fail("live account requires --live (with --yes)", args.json)
if args.json:
_emit(report, as_json=True)
else:
_emit_pretty_json(report)
return 2
if not getattr(args, "yes", False):
if args.json:
report["result"] = {"status": "rejected", "reason": "--json ledger backfill requires --yes"}
report["audit_path"] = _write_ledger_backfill_audit(args, report)
_fail("--json ledger backfill requires --yes", True)
_emit(report, as_json=True)
return 2
prompt = (
f"Backfill ledger trade on instance={instance} broker={broker_name}"
f"{' LIVE' if is_live else ''}: {report['trade']}?"
)
if not confirm(prompt):
report["result"] = {"status": "aborted", "reason": "operator declined confirmation"}
report["audit_path"] = _write_ledger_backfill_audit(args, report)
print("aborted")
_emit_pretty_json(report)
return 1
outbox = PostgreSQLLedgerOutbox(cfg.dsn)
if not cfg.portfolio_hub or not cfg.portfolio_hub.base_url:
report["result"] = {"status": "rejected", "reason": "portfolio-hub ledger adapter is not configured"}
report["audit_path"] = _write_ledger_backfill_audit(args, report)
if args.json:
_emit(report, as_json=True)
else:
_emit_pretty_json(report)
return 1
writer = LedgerWriter(
outbox,
PortfolioHubLedger(
cfg.portfolio_hub.base_url,
token=cfg.portfolio_hub.token or None,
timeout=cfg.portfolio_hub.timeout,
),
)
existing = (report.get("outbox_precheck") or {}).get("existing")
if existing is None:
outbox.enqueue(trade)
outcome = writer.process_correlation_id(trade.correlation_id)
elif existing.get("status") == "pending":
outcome = writer.process_correlation_id(trade.correlation_id)
else:
outcome = None
item = outbox.get(trade.correlation_id)
result_status, exit_code = _ledger_outcome_result(
outcome,
existing_status=str(existing["status"]) if isinstance(existing, dict) else None,
)
report["result"] = {
"status": result_status,
"correlation_id": trade.correlation_id,
"outcome": outcome.value if outcome is not None else None,
"work_item_status": item.status.value if item is not None else None,
"attempts": item.attempts if item is not None else None,
"last_outcome": item.last_outcome.value if item and item.last_outcome else None,
"last_error": item.last_error if item is not None else None,
}
if exit_code == 0:
resolved = PostgreSQLLedgerSkipStore(cfg.dsn).resolve_for_fill(
instance_key=instance,
slot_id=uuid.UUID(str(slot_binding["slot_id"])),
side=trade.side,
order_id=trade.order_id,
execution_id=trade.execution_id,
resolution_reason="manual_ledger_backfill",
)
report["resolved_skips"] = [_render_ledger_skip_record(record) for record in resolved]
if resolved:
emitter, transport = _default_alarm_emitter(cfg)
try:
for record in resolved:
publish_ledger_skip_alarm(emitter, record, resolved=True)
finally:
transport.close()
report["audit_path"] = _write_ledger_backfill_audit(args, report)
if args.json:
_emit(report, as_json=True)
else:
_emit_pretty_json(report)
return exit_code
def main(argv: list[str] | None = None, *, rpc=None, confirm=None) -> int:
"""CLI entrypoint.
@@ -858,6 +1435,21 @@ def main(argv: list[str] | None = None, *, rpc=None, confirm=None) -> int:
if args.cmd == "verify":
return _cmd_verify(args, cfg)
if args.cmd == "ledger" and getattr(args, "ledger_cmd", None) == "backfill":
return _cmd_ledger_backfill(args, cfg, confirm)
if (
args.cmd == "ledger"
and getattr(args, "ledger_cmd", None) == "skips"
and getattr(args, "ledger_skips_cmd", None) == "list"
):
return _cmd_ledger_skips_list(args, cfg)
if (
args.cmd == "ledger"
and getattr(args, "ledger_cmd", None) == "skips"
and getattr(args, "ledger_skips_cmd", None) == "resolve"
):
return _cmd_ledger_skips_resolve(args, cfg, confirm)
# direct-DB control commands (pause/resume stay direct-DB, OQ-3).
if args.cmd in ("pause", "resume", "paused"):
store = PostgreSQLPauseStore(cfg.dsn)
+13
View File
@@ -35,6 +35,7 @@ from trex_core.adapters.persistence.migrate import apply_migrations
from trex_core.adapters.persistence.account_snapshot import (
PostgreSQLAccountSnapshotStore,
)
from trex_core.adapters.persistence.ledger_skip_store import PostgreSQLLedgerSkipStore
from trex_core.adapters.persistence.postgresql import (
PgStrategyStateStore,
PostgreSQLIntentStore,
@@ -48,6 +49,9 @@ from trex_core.adapters.portfolio import (
StubPortfolioRead,
)
from trex_core.adapters.portfolio.killswitch_store import PostgreSQLKillSwitchStore
from trex_core.adapters.portfolio.reconcile_notification_store import (
PostgreSQLReconcileNotificationStore,
)
from trex_core.application.alarm_emitter import AlarmEmitter
from trex_core.application.account_snapshot import AccountSnapshotService
from trex_core.application.broker_port import BrokerPort
@@ -188,6 +192,7 @@ class Daemon:
)
self.outbox = PostgreSQLLedgerOutbox(config.dsn)
self.ledger_skip_store = PostgreSQLLedgerSkipStore(config.dsn)
# BAT-008: portfolio-hub read + snapshot store + durable pause + scheduler.
# (Created before the ledger writer so option-D write-back can target it.)
self.snapshot_store = SnapshotStore()
@@ -196,6 +201,8 @@ class Daemon:
self.pause_store = PostgreSQLPauseStore(config.dsn)
# BAT-050 / REQ-102: durable reconcile-warning store (slot_id PK).
self.warn_store = PostgreSQLReconcileWarningStore(config.dsn)
# req-0007 / ac-0024: durable reconcile notification deduplication store.
self.notification_store = PostgreSQLReconcileNotificationStore(config.dsn)
# BAT-012: day-scoped kill-switch exclusion store + service.
self.killswitch_store = PostgreSQLKillSwitchStore(config.dsn)
self.portfolio_read = _build_portfolio_read(config)
@@ -248,6 +255,8 @@ class Daemon:
retry_cap_sec=config.ledger_retry_cap_sec,
max_attempts=config.ledger_max_attempts,
alarm_emitter=self.alarm_emitter, # dcy-033 / REQ-100
ledger_skip_store=self.ledger_skip_store,
instance_key=config.instance_key,
)
self.kill_switch = KillSwitchService(
slot_repo=self.slot_repo,
@@ -269,6 +278,7 @@ class Daemon:
instance_key=config.instance_key,
alarm_emitter=self.alarm_emitter, # dcy-033 / REQ-100
warn_store=self.warn_store, # BAT-050 / REQ-103: durable reconcile-warning
notification_store=self.notification_store, # req-0007 / ac-0024
credential_fetch_enabled=config.credential_fetch_enabled, # dcy-036 / REQ-113
credential_refresh=self.broker_provider.refresh_day_start_credential,
# dcy-036 / REQ-113 AC-2 / REQ-115 AC-1: KISBroker reinit callback.
@@ -289,6 +299,7 @@ class Daemon:
max_active_slots=config.max_active_slots, # dcy-020: asset-centric gate
default_order_timeout_sec=config.default_order_timeout_sec, # dcy-022
warn_store=self.warn_store, # BAT-050 / REQ-104: reconcile-warning gate
alarm_emitter=self.alarm_emitter, # req-0008 / ac-0025
entry_allowed_fn=entry_allowed_fn, # dcy-036 / REQ-114: day-start gate
buy_guard_fn=(
self._account_snapshot_buy_guard
@@ -303,6 +314,7 @@ class Daemon:
stale_pending_threshold_sec=config.pending_stale_threshold_sec,
instance_key=config.instance_key,
alarm_emitter=self.alarm_emitter, # dcy-033 / REQ-100
ledger_skip_store=self.ledger_skip_store,
balance_reader=(
self.account_snapshot_service.holdings_for_sell
if self.account_snapshot_service is not None
@@ -440,6 +452,7 @@ class Daemon:
ledger_outbox=self.outbox,
max_active_slots=self.config.max_active_slots,
strategy_state_store=self.strategy_state_store, # dcy-028 C-1
broker_quantity_reader=self.account_snapshot_service,
)
transport.consume_rpc_forever(
topology.slots_queue(self.config.instance_key), handler, self.stop)