portfolio-hub/docs/data-model.md

Portfolio Hub Data Model

• Baseline Mode: observed
• Baseline Seed: .stagepilot/bootstrap/baseline.yaml
• Status: confirmed
• Owner: repository-maintainers
• Last Updated (KST): 2026-05-21 00:00
• Source Discovery / Batch: bootstrap-baseline, bat-001, bat-006, bat-008, bat-010, bat-003, bat-004, bat-005, BAT-006(dcy-004), BAT-007(dcy-004), BAT-008(dcy-004)

Project Summary

• PortfolioHub exposes portfolio management and trading ledger workflows through a FastAPI service and a companion CLI.

Primary Domain

• Portfolio management and trading ledger services.

Primary Persistence Backend

• PostgreSQL or TimescaleDB via psycopg with Alembic-managed migrations.

Purpose

• This document captures the approved cross-cutting data model baseline for the project.
• It is a shared reference for REQ drafting, batch design, implementation, verification, migration planning, and handoff.

Scope

• Covers core entities, relationships, states, persistence mappings, and shared consistency rules.
• Does not duplicate full functional acceptance criteria already owned by REQ documents.
• Does not replace ORM definitions, migration files, or generated schema artifacts.

Model Summary

• Portfolio :: owned investment portfolio with balances, strategy references, tag_ids, comments, and ledger history
• User :: authenticated principal that owns portfolios and receives client or service tokens
• Daemon :: stable logical identity for a trading daemon process; registered per user; bound to service tokens and assigned to assets
• AuthToken :: issued service token (or session token for login flows) with permission, runtime_id, and optional daemon_id binding; client token type removed (REQ-016/BAT-006)
• BrokerCredential :: encrypted broker API credentials per user-broker pair
• LedgerTrade :: idempotent trade execution record keyed by (asset_id, order_id, execution_id)
• Asset :: catalog entry representing a tradeable asset; portfolios reference assets via FK
• Tag :: master tag record with ownership (created_by_user_id), admin flag, and active/inactive lifecycle

Entity: Tag

• Purpose: represent a named master tag with ownership and admin classification; applied to portfolios via tag_ids UUID[] column
• Source of Truth: tags table (added by BAT-003 migration 20260519_0017)
• Lifecycle: created (is_active=true) → optionally deactivated (is_active=false); no hard delete

Key Fields

• tag_id :: uuid :: server-generated UUID primary key (gen_random_uuid())
• name :: text UNIQUE :: tag label; normalized to lowercase and stripped; unique across all tags
• created_by_user_id :: text :: user_id of the creator; used for portfolio tag apply policy
• is_admin_tag :: boolean :: if true, tag was created with admin permission; can be applied to any portfolio regardless of ownership
• is_active :: boolean :: inactive tags cannot be applied; existing tag_ids arrays are not modified on deactivation
• created_at :: timestamptz :: creation timestamp (server_default=NOW())

Uniqueness Constraints

• tags_name_key :: UNIQUE (name) :: one tag entry per unique lowercase name

Relationships

• Portfolio :: portfolios.tag_ids UUID[] soft-references tags.tag_id (no FK constraint); array_append/array_remove manage membership

State Rules

• Only is_active=true tags can be applied via POST /v1/assets/{id}/tags or POST /v1/portfolios/{id}/tags.
• Deactivating a tag does not remove it from portfolios.tag_ids arrays (lazy cleanup).
• Admin tags (is_admin_tag=true) can be applied to any portfolio; non-admin tags can only be applied to portfolios owned by the tag creator.
• Asset tag application requires is_admin_tag=true (asset tags are always admin-controlled).

Removed Entity: portfolio_tags (BAT-003)

• The old portfolio_tags table (string-keyed free-form tags per portfolio) was replaced by portfolios.tag_ids UUID[] in BAT-003.
• portfolio_tags DROP was included in migration 20260519_0017.

Entity: Asset

• Purpose: represent a tradeable asset in the catalog; serves as FK anchor for portfolios
• Source of Truth: assets table; managed exclusively via admin CLI / admin API
• Lifecycle: created → optionally deactivated; never deleted (portfolios reference it with RESTRICT FK)

Key Fields

• asset_id :: uuid :: server-generated UUID primary key (gen_random_uuid())
• symbol :: text UNIQUE :: exchange symbol (e.g. 005930, BTC)
• markets :: text[] :: array of market codes the asset is listed on (e.g. ["KRX", "NASDAQ"])
• display_name :: text|null :: human-readable name for display
• status :: text :: "active" | "inactive"; only active assets can be referenced by new portfolios

Uniqueness Constraints

• assets_symbol_key :: UNIQUE (symbol) :: one catalog entry per symbol

Indexes

• idx_assets_markets_gin :: GIN index on markets array :: supports efficient array containment queries

Relationships

• Portfolio :: portfolios.asset_id FK → assets.asset_id ON DELETE RESTRICT; one asset → many portfolios

State Rules

• Only status="active" assets can be referenced when creating a new portfolio.
• Deactivated assets do not affect existing portfolios; only new creation is blocked.

Entity: StrategySet

• Purpose: stable, reusable strategy catalog entry selected by portfolio mutations through its ID
• Source of Truth: strategy_sets; migration 20260814_0029 adds lifecycle and revision fields and seeds/reuses the canonical target_value / {} entry
• Lifecycle: active → updated with monotonically increasing revision → optionally inactive → deleted only when unreferenced

Key Fields

• id :: text :: stable strategy ID; never changes during PATCH
• name/config :: text :: strategy identity/configuration pair; unique together
• nick :: text :: unique derived display key (name:sha1(config)[:12]), recomputed from the latest row state under the PATCH row lock
• status :: text :: active or inactive; inactive entries remain valid historical references but cannot be newly assigned
• revision :: integer :: starts at 1 and increments once per effective mutation; no-op PATCH does not increment
• created_at/updated_at :: timestamptz :: creation and last effective mutation timestamps

Relationships and Deletion

• portfolios.active_strategy_id and entries in portfolios.avabile_strategies reference strategy IDs.
• New selection uses a strategy FOR SHARE lock in the same transaction as portfolio persistence; PATCH/DELETE use FOR UPDATE so concurrent mutation, inactivation, and deletion serialize.
• Deletion is rejected while either reference form exists. The reference check and delete share one transaction, preventing check-then-delete orphan races. Inactivation does not rewrite existing portfolio references.
• Trade snapshot persistence does not write either strategy-selection column. Strategy selection is owned only by portfolio create/update paths, so stale BUY/SELL state cannot resurrect a deleted strategy reference.

Entity: Portfolio

• Purpose: represent an owned investment portfolio with balances, strategy references, tags, comments, and trading history
• Source of Truth: portfolios table with dependent portfolio_ledger, portfolio_comments, and portfolio_tags tables
• Lifecycle: created -> updated by trade and cash operations -> reset or deleted

Key Fields

• portfolio_id :: uuid :: server-generated UUID primary key; assigned at creation and never changed; BAT-006 migrated all pre-existing TEXT portfolio_id values to UUIDs
• name :: text :: human-readable portfolio name; unique per owner_user_id; validated against [a-zA-Z0-9._:-]{1,100}
• asset_id :: uuid :: FK → assets.asset_id; catalog reference for the traded asset; BAT-001 migrated symbol/market columns to this FK
• symbol :: text :: display symbol populated via JOIN on assets (read-only, not stored in portfolios)
• status :: text :: "enabled" (active, default) | "disabled" (soft-deleted); set to "disabled" by DELETE /v1/portfolios/{id}; only status can be mutated on a disabled portfolio (restoration via PATCH); all other writes → HTTP 409
• owner_user_id :: text|null :: owning user reference for attribution and authorization
• active_strategy_id :: text|null :: currently selected stable strategy-catalog ID; create requires an active strategy and update may select another active strategy
• strategy_revision :: integer|null :: read-only revision joined from the currently referenced StrategySet; subsequent API reads expose the latest committed shared config revision
• cash_balance :: double precision :: current cash snapshot after the latest ledger event
• asset_quantity :: double precision :: current holdings snapshot after the latest ledger event
• managed_by_daemon_id :: text|null :: soft reference to daemons.id; controls daemon-scoped API access; NULL = unassigned (fail-closed for ledger writes and asset listing); set to NULL on soft delete; added by migration 20260513_0016
• tag_ids :: uuid[] :: array of tag UUIDs from the tags master table; replaces the old string-based portfolio_tags table (BAT-003 migration 20260519_0017); preserved on soft delete; default empty array

Uniqueness Constraints

• uq_portfolios_owner_name :: UNIQUE (owner_user_id, name) :: enforces per-user portfolio name uniqueness at the DB level

Relationships

• Asset :: asset_id FK ties each portfolio to exactly one catalog asset
• User :: owner_user_id links a portfolio to its owning user when ownership is assigned
• Daemon :: managed_by_daemon_id links a portfolio to exactly one daemon; NULL means unassigned
• StrategySet :: active_strategy_id and avabile_strategies connect reusable strategy definitions
• LedgerEntry, PortfolioComment :: child records are keyed by portfolio_id (UUID)
• Tag :: tag_ids UUID[] references tags.tag_id (no FK — soft reference for flexibility)

State Rules

• Portfolio names are required, must be unique per owner_user_id, and are validated at the schema layer before persistence.
• portfolio_id is server-generated at creation; clients must never supply or assume a specific portfolio_id value.
• Balance-changing commands must update snapshot fields and append a ledger row together.
• tag_ids stores UUID references to active tags; duplicates are prevented by conditional array_append; comments are retrieved in created_at descending order.
• status="disabled" (soft delete): managed_by_daemon_id is set to NULL; portfolio is excluded from GET /v1/portfolios by default. Dependent data (portfolio_comments, tag_ids) is preserved — never deleted on soft delete. READ operations on sub-resources (GET tags, GET comments) remain allowed; write mutations raise HTTP 409.
• Status restoration: PATCH with {"status": "enabled"} re-activates a disabled portfolio; no other fields may be set in the same request.
• Daemon assignment: a non-null managed_by_daemon_id on create or PATCH must reference an existing, active daemon owned by the target portfolio owner (admin user_id override uses the final target owner). The daemon FOR SHARE row lock and the portfolio INSERT/UPDATE share one transaction, so validation and write form a single atomic decision boundary; a concurrently deactivated daemon is rejected (HTTP 409) rather than committed. Deactivation after an assignment commits preserves the portfolio reference and follows the existing token-revocation contract.
• Daemon owner transfer: update_daemon_owner() locks the daemon row FOR UPDATE, rejects a referenced daemon with a dedicated conflict, and deletes broker credentials only after the no-reference check. Assignment and owner transfer therefore serialize on the same daemon row: transfer-first makes the old owner's pending assignment foreign, while assignment-first makes the transfer conflict. Neither path auto-unassigns, migrates, or reassigns portfolios.

Entity: User

• Purpose: represent an authenticated principal that can own portfolios and receive service tokens
• Source of Truth: users table with dependent auth_tokens rows
• Lifecycle: bootstrapped or created -> updated -> deactivated or deleted

Key Fields

• id :: text :: stable user primary key
• username :: text :: unique login and CLI lookup key
• email :: text|null :: unique contact identity when present
• permission :: text :: baseline role of read, trade-write, or admin
• is_active :: boolean :: authentication and authorization gate
• last_login_at :: timestamptz|null :: latest successful login timestamp

Relationships

• AuthToken :: user_id links issued service tokens to the owning user
• Portfolio :: portfolios.owner_user_id ties portfolios to a managing user
• TokenPrincipal :: runtime auth resolution derives principals from persisted tokens and users

State Rules

• username and email remain unique identifiers when present.
• Bootstrap admin creation only runs when the users table is empty and DB mode is enabled.
• Permission changes affect both API guards and CLI mutation capabilities.

Entity: AuthToken

• Purpose: represent an issued service token (or session token for login flows) linked to a user with permission and optional runtime identifier; client token type (token_type="client") removed by REQ-016/BAT-006
• Source of Truth: auth_tokens table
• Lifecycle: issued -> optionally rotated -> revoked or expired

Key Fields

• id :: text :: primary key; raw token value (stored hashed in practice)
• user_id :: text :: owning user reference
• token_type :: text :: "session" (login tokens) | "service" (daemon-bound operational tokens); "client" type removed by REQ-016/BAT-006
• token_source :: text :: "db" for persisted tokens; "env" for environment-injected tokens
• permission :: text :: read, trade-write, or admin
• service_name :: text|null :: logical name for service tokens (unique within token_type="service")
• runtime_id :: text|null :: per-token opaque runtime identifier; regenerated on every rotate; added by migration 20260502_0014
• daemon_id :: text|null :: soft reference to daemons.id; stable across rotates (rotate invariant); NULL = legacy/unbound token (fail-closed for daemon-scoped endpoints); added by migration 20260513_0016
• expires_at :: timestamptz|null :: token expiry; null for non-expiring tokens
• created_at :: timestamptz :: issuance timestamp

Uniqueness Constraints

• idx_auth_tokens_service_name_unique :: UNIQUE (name) WHERE token_type='service' AND revoked_at IS NULL (BAT-006/REQ-017: revoked_at IS NULL 조건 추가로 rotate 후 구 token과 신 token 이름 충돌 해소)
• idx_auth_tokens_runtime_id_unique :: UNIQUE (runtime_id) WHERE runtime_id IS NOT NULL

State Rules

• Service tokens are the only tokens accepted by v1 API endpoints.
• runtime_id is per-token and changes on every rotate; daemon_id is stable across rotates (daemon_id invariant).
• daemon_id=NULL tokens are fail-closed on GET /v1/assets and POST /v1/assets/{asset_id}/ledger/trades (HTTP 403); GET /v1/user_info returns 200 + daemon_id: null as diagnostic.
• Token permission gates both HTTP API guards and CLI mutation capabilities.
• Deprecated: token_type="client" issuance path removed by REQ-016/BAT-006; existing "client" type rows in legacy DBs remain readable but new issuance is blocked.

Entity: Daemon

• Purpose: represent a stable logical identity for a trading daemon process; registered per user; bound to service tokens at issuance; assigned to assets
• Source of Truth: daemons table (added by migration 20260513_0016)
• Lifecycle: created by operator via pfhub daemon create -> optionally updated -> deactivated

Key Fields

• id :: text :: server-generated UUID primary key; stable logical identity across service-token rotates
• owner_user_id :: text :: owning user reference (part of composite unique constraint)
• daemon_slug :: text :: operator-assigned short identifier; unique per owner_user_id
• display_name :: text|null :: human-readable label for the daemon
• is_active :: boolean :: if false, service-token issuance for this daemon is rejected; default true
• created_at :: timestamptz :: registration timestamp
• updated_at :: timestamptz :: last update timestamp

Uniqueness Constraints

• uq_daemons_owner_slug :: UNIQUE (owner_user_id, daemon_slug) :: one slug per user

Relationships

• User :: owner_user_id links a daemon to its owning user
• AuthToken :: auth_tokens.daemon_id binds a service token to this daemon (1 token : 1 daemon)
• Portfolio :: portfolios.managed_by_daemon_id assigns an asset to this daemon (1 asset : 1 daemon)

State Rules

• A daemon must exist and be active before a service token can be bound to it at issuance.
• A daemon must exist, be active, and be owned by the target portfolio owner before a portfolio can be assigned to it (create or PATCH). Assignment validation locks the daemon row FOR SHARE inside the portfolio write transaction (Issue #27 / dcy-0004).
• Owner transfer locks the daemon row FOR UPDATE and is rejected when any portfolio references the daemon. The rejection preserves the daemon owner, all portfolio assignments, and broker credentials; an unreferenced transfer retains the existing credential invalidation policy.
• Deactivating a daemon (is_active=false) immediately revokes all bound service tokens via revoke_tokens_by_daemon_id (REQ-022/BAT-007) and blocks new token issuance. It does not auto-remove existing portfolio references; deactivation racing a new assignment is rejected with HTTP 409 when the deactivation commits first.
• daemon_slug is immutable after creation; only display_name and is_active can be updated.

Entity: BrokerCredential

• Purpose: store Fernet-encrypted broker API credentials per user-broker pair for the trex-basic integration
• Source of Truth: broker_credentials table (added by migration 20260502_0014)
• Lifecycle: upserted via CLI credential command -> optionally rotated

Key Fields

• user_id :: text :: owning user reference (part of composite PK)
• broker :: text :: broker identifier, e.g., "KIS" (part of composite PK)
• account_ref :: text :: broker account reference or alias
• encrypted_payload :: text :: Fernet-encrypted JSON ciphertext; plaintext is {"app_key": ..., "app_secret": ...}; decrypted at query time using CREDENTIAL_ENCRYPTION_KEY (REQ-010)
• credential_version :: integer :: monotonically increasing rotation counter; server_default=1
• last_rotated_at :: timestamptz :: last credential upsert or rotation timestamp; server_default=NOW()

Uniqueness Constraints

• PRIMARY KEY (user_id, broker) :: one credential entry per user per broker

State Rules

• encrypted_payload must never contain plaintext credentials; Fernet encryption is mandatory before persistence.
• Decryption failures (wrong key, corrupted payload) log a warning and return credential_payload: null in GET /v1/user_info.

Entity: LedgerTrade

• Purpose: record idempotent trade execution entries submitted by trex-basic via POST /v1/portfolios/{portfolio_id}/ledger/trades
• Source of Truth: ledger_trades table (added by migration 20260502_0014; column asset_id renamed to portfolio_id by migration 20260601_0024 — REQ-033 / dcy-005 FR-1)
• Lifecycle: inserted once per unique (portfolio_id, order_id, execution_id) tuple; never updated or deleted

Key Fields

• ledger_entry_id :: bigint (Identity) :: auto-generated sequential primary key
• portfolio_id :: text :: portfolios.portfolio_id (UUID-format text) referencing the trading portfolio (renamed from asset_id by REQ-033)
• order_id :: text :: broker-assigned order identifier
• execution_id :: text :: broker-assigned execution/fill identifier
• side :: text :: "buy" or "sell"
• quantity :: numeric(20,8) :: executed quantity
• price :: numeric(20,8) :: execution price
• fee_amount :: numeric(20,8) :: brokerage fee; server_default=0
• tax_amount :: numeric(20,8) :: transaction tax; server_default=0
• executed_at :: timestamptz :: broker-reported execution timestamp
• recorded_at :: timestamptz :: insertion timestamp; server_default=NOW()

Uniqueness Constraints

• uq_ledger_trades_idempotency :: UNIQUE (portfolio_id, order_id, execution_id) :: idempotency guarantee; duplicate inserts are detected and return HTTP 200 + status="duplicate"
• idx_ledger_trades_asset_executed :: INDEX (portfolio_id, executed_at) :: time-range query optimization (constraint/index objects retain their original names; only the underlying column was renamed)

State Rules

• Rows are immutable after insertion; no UPDATE path exists.
• Payload conflict detection: if unique key matches but payload differs → HTTP 409 Conflict.
• portfolio_id is TEXT (not a FK) but must correspond to a valid portfolios.portfolio_id before insertion.

Entity: PortfolioLedger manual extensions (REQ-056 / BAT-024)

• Purpose: unified append-only portfolio ledger table now also records owner-scoped manual ledger entries for trex-web.
• Source of Truth: portfolio_ledger table.
• Lifecycle: manual API appends rows with source='manual'; previous rows are never updated/deleted. Corrections use REVERSAL rows referencing a target ledger entry.

Additional Key Fields

• source :: text :: broker for existing daemon trade writes by default, manual for owner-scoped manual ledger API rows.
• target_ledger_entry_id :: bigint|null :: required for REVERSAL; references the reversed ledger entry logically.
• confirmation_evidence :: text|null :: records API-level explicit confirmation evidence for daemon-managed portfolio manual writes.
• actor_user_id :: text|null :: user/session actor that appended the manual ledger entry.

Manual Action Semantics

• INVEST: external capital -> profit_buffer; increases net contributed capital.
• WITHDRAW: profit_buffer -> external capital as contributed-capital withdrawal; decreases net contributed capital.
• PROFIT: profit_buffer -> external capital as pure profit withdrawal; does not affect net contributed capital and is exposed separately as cumulative_profit_withdrawn.
• BUFFER_OUT: profit_buffer -> cash_balance.
• BUFFER_IN: cash_balance -> profit_buffer.
• BUY: cash_balance -> asset_quantity.
• SELL: asset_quantity -> cash_balance.
• REVERSAL: append-only inverse effect of target_ledger_entry_id.

State Rules

• Manual writes lock the portfolio row while calculating/appending/updating snapshots to avoid lost updates.
• Manual writes reject insufficient cash/quantity/buffer with HTTP 409 rather than creating a new negative state.
• total_value_after for manual entries is the portfolio snapshot value after the mutation and includes cash, asset valuation, and profit buffer.

Shared Consistency Rules

• auth_tokens.user_id and portfolios.owner_user_id should resolve to existing users before protected flows run.
• strategy_sets.name plus config, strategy_sets.nick, and service token names have uniqueness constraints.
• Portfolio snapshot fields and portfolio_ledger history must stay synchronized across mutations.
• (owner_user_id, name) uniqueness is enforced at both the DB constraint level and the application layer; violations surface as a 409 Conflict or ValueError with a Korean message.

Persistence / Integration Notes

• Alembic revisions in migrations/versions/ are the ordered schema history for persistent environments.
• app/repositories/postgres.py includes ensure_schema() for bootstrap safety, but long-term schema evolution should run through Alembic.
• Current Alembic head is 20260814_0029; its strategy columns, default seed, and constraints are guarded for ensure_schema-first and repeatable execution.
• Comments, tags, auth tokens, and ledger rows are normalized into dedicated tables keyed to users or portfolios.

Current Gaps / Planned Changes

• Production-like migration verification against a real PostgreSQL or TimescaleDB instance is still pending.
• Retention and archival rules are not yet documented for ledger, comment, or revoked-token records.
• Future authorization changes may require stronger ownership constraints in schema or query paths.

Update Triggers

• Update this document when a core entity, key field, or relationship changes.
• Update this document when lifecycle or persistence behavior changes.
• Update this document when a batch introduces a migration, backfill, or breaking data assumption.

Change Log

• 2026-07-14 KST: BAT-024 (REQ-056) — documented portfolio_ledger manual extensions (source, target_ledger_entry_id, confirmation_evidence, actor_user_id) and PROFIT/WITHDRAW return semantics.
• 2026-05-21 KST: BAT-010 (REQ-032) — v1 portfolio surface 데이터 모델 보강: Portfolio.status 필드 신설 ("enabled" | "disabled" 의미, soft delete 정책); Portfolio.managed_by_daemon_id에 soft delete 시 NULL 설정 명시; tag_ids 소프트 삭제 보존 정책 추가; Portfolio State Rules에 disabled 상태 정책 + 의존 데이터 보존 + 상태 복원 규칙 추가; Source Discovery/Batch 헤더 갱신.
• 2026-05-21 KST: BAT-008 (REQ-019) — 토큰 surface 정합성 회복: AuthToken token_type "client" 제거 반영; idx_auth_tokens_service_name_unique에 AND revoked_at IS NULL 조건 추가(REQ-017); Daemon 비활성화 시 바인딩 서비스 토큰 자동 revoke 정책 반영(REQ-022); User/AuthToken Purpose 문구 갱신.
• 2026-05-20 KST: bat-005 — REQ-008 AC-1: added Entity: Tag (tags master table, BAT-003 migration 20260519_0017); updated Portfolio.tag_ids UUID[] field; removed portfolio_tags table note; updated Portfolio Relationships and State Rules; updated Model Summary.
• 2026-05-19 KST: bat-002 — REQ-015: added Entity: Asset (asset catalog with symbol UNIQUE, markets[], status); updated Portfolio.asset_id FK and symbol/market field notes.
• 2026-05-13 KST: bat-010 — REQ-009/023/028: added Entity: Daemon; added portfolios.managed_by_daemon_id; added auth_tokens.daemon_id; updated AuthToken State Rules (runtime_id vs daemon_id distinction, fail-closed policy); updated Portfolio Relationships; updated Model Summary.
• 2026-05-11 KST: bat-008 — REQ-011 AC-2: added portfolios.market column; added Entity: AuthToken (runtime_id field); added Entity: BrokerCredential; added Entity: LedgerTrade; updated Model Summary.
• 2026-05-11 KST: bat-006 — portfolios.portfolio_id changed to server-generated UUID PK; name column added with UNIQUE (owner_user_id, name) constraint; uq_portfolios_owner_name constraint documented; state rules updated.
• 2026-04-30 KST: Bootstrap baseline created from repository observation for initial StagePilot adoption.