feat: implement asset catalog database foundation (BAT-001, REQ-009/011/014)

Asset Catalog Model Implementation (BAT-001):
- Introduce assets table with UUID PK, symbol UNIQUE, markets TEXT[] array
- Define MarketCode enum (KRX, NXT, SOR, NASDAQ, NYSE, UPBIT, BITHUMB)
- Add Asset domain model with catalog properties
- Implement asset CRUD protocols in portfolio_repository
- Add PostgreSQL asset CRUD operations with GIN index on markets

Portfolio Model Refactoring (REQ-011):
- Add portfolios.asset_id UUID FK to assets(asset_id) ON DELETE RESTRICT
- Remove portfolios.symbol and portfolios.market columns
- Update Portfolio domain model to reference asset_id instead of symbol/market
- Refactor all portfolio queries to JOIN assets catalog
- Update all repository methods to use asset references

Data Migration (REQ-014):
- Create 20260519_0017 Alembic revision for assets table creation
- Create 20260519_0018 Alembic revision for atomic data migration:
  * Extract unique symbol combinations from portfolios
  * Aggregate markets using ARRAY_AGG(DISTINCT market)
  * Populate assets catalog with extracted data
  * Update portfolios.asset_id to reference new assets
  * Remove legacy symbol/market columns in single transaction
- Implement migration with rollback support

Requirements Approved (REQ-001~008):
- REQ-001~008 transitioned from Proposed to Approved
- Complete tag model requirement set with implementation readiness

New Requirements Approved (REQ-009~015):
- REQ-009: assets table schema (approved, included in BAT-001)
- REQ-010: admin asset catalog CRUD (approved, future BAT-002)
- REQ-011: portfolios asset_id FK (approved, included in BAT-001)
- REQ-012: portfolio create catalog validation (approved, future BAT-002)
- REQ-013: GET /v1/assets token branching (approved, future BAT-002)
- REQ-014: portfolios data migration (approved, included in BAT-001)
- REQ-015: asset catalog baseline docs (approved)

Code Changes:
- app/domain/models.py: Add Asset model, MarketCode enum, Portfolio asset_id FK
- app/repositories/portfolio_repository.py: Define asset CRUD protocol
- app/repositories/postgres.py: Implement asset operations with full CRUD
- app/api/v1_assets.py: Remove symbol/market direct references
- app/cli/commands/portfolio.py: Update portfolio commands for asset references
- app/schemas/portfolio.py: Update portfolio schema with asset_id
- All portfolio operations throughout codebase: Use asset references

Testing:
- Add test_asset_catalog.py for asset CRUD operations
- Verify REQ-009 AC-1~5 (asset catalog creation, lookup, uniqueness)
- Verify REQ-011 AC-1~5 (FK constraint, portfolio asset reference)
- Verify REQ-014 AC-1~6 (migration atomicity, data correctness, rollback)
- Update existing tests for asset model usage

Documentation:
- Create BAT-001 batch documents (planning, design, implementation, verification)
- Update dcy-001 with implementation details and confirmed architecture
- Add 8 new requirement documents (REQ-009~015)
- Update docs/discovery/index.md with batch entry

Impact:
- Database: assets catalog table, portfolios refactored to reference assets
- Migration: 2 new Alembic revisions (0017 assets, 0018 portfolio migration)
- API: Symbol/market now accessed through asset references
- CLI: Portfolio commands require asset reference pattern
- All existing portfolio data preserved and correctly migrated
This commit is contained in:
2026-05-19 15:59:20 +00:00
parent 1082f041a1
commit e2226e28b6
40 changed files with 2118 additions and 168 deletions
+172 -71
View File
@@ -11,7 +11,7 @@ from typing import Any
import psycopg
from psycopg import sql
from app.domain.models import AuthToken, BrokerCredential, Daemon, LedgerEntry, Portfolio, PortfolioComment, PortfolioTag, StrategySet, TokenPrincipal, User
from app.domain.models import Asset, AuthToken, BrokerCredential, Daemon, LedgerEntry, Portfolio, PortfolioComment, PortfolioTag, StrategySet, TokenPrincipal, User
_SELECT_USER = """
SELECT
@@ -46,23 +46,37 @@ _SELECT_AUTH_TOKEN = """
_SELECT_PORTFOLIO = """
SELECT
portfolio_id,
broker,
status,
name,
p.portfolio_id,
p.broker,
p.status,
p.name,
a.symbol,
p.cash_balance,
p.asset_quantity,
p.asset_price,
p.profit_buffer,
p.active_strategy_id,
p.avabile_strategies,
p.owner_user_id,
p.created_at,
p.updated_at,
p.last_trade_at,
p.managed_by_daemon_id,
p.asset_id
FROM portfolios p
JOIN assets a ON p.asset_id = a.asset_id
"""
_SELECT_ASSET = """
SELECT
asset_id,
symbol,
cash_balance,
asset_quantity,
asset_price,
profit_buffer,
active_strategy_id,
avabile_strategies,
owner_user_id,
markets,
display_name,
status,
created_at,
updated_at,
last_trade_at,
managed_by_daemon_id
FROM portfolios
updated_at
FROM assets
"""
@@ -177,6 +191,25 @@ class PostgresPortfolioRepository:
)
"""
)
cur.execute(
"""
CREATE TABLE IF NOT EXISTS assets (
asset_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
symbol TEXT NOT NULL UNIQUE,
markets TEXT[] NOT NULL DEFAULT '{}',
display_name TEXT,
status TEXT NOT NULL DEFAULT 'active',
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
)
"""
)
cur.execute(
"""
CREATE INDEX IF NOT EXISTS idx_assets_markets_gin
ON assets USING GIN (markets)
"""
)
cur.execute(
"""
CREATE TABLE IF NOT EXISTS portfolios (
@@ -184,7 +217,7 @@ class PostgresPortfolioRepository:
broker TEXT NOT NULL DEFAULT 'kis',
status TEXT NOT NULL DEFAULT 'enabled',
name TEXT NOT NULL,
symbol TEXT NOT NULL,
asset_id UUID NOT NULL REFERENCES assets(asset_id) ON DELETE RESTRICT,
owner_user_id TEXT,
cash_balance DOUBLE PRECISION NOT NULL,
asset_quantity DOUBLE PRECISION NOT NULL,
@@ -201,7 +234,8 @@ class PostgresPortfolioRepository:
cur.execute("ALTER TABLE portfolios ADD COLUMN IF NOT EXISTS name TEXT")
cur.execute("ALTER TABLE portfolios ADD COLUMN IF NOT EXISTS owner_user_id TEXT")
cur.execute("ALTER TABLE portfolios ADD COLUMN IF NOT EXISTS managed_by_daemon_id TEXT")
cur.execute("UPDATE portfolios SET name = symbol WHERE name IS NULL OR name = ''")
cur.execute("ALTER TABLE portfolios ADD COLUMN IF NOT EXISTS asset_id UUID")
cur.execute("UPDATE portfolios SET name = portfolio_id WHERE name IS NULL OR name = ''")
cur.execute("ALTER TABLE portfolios ALTER COLUMN name SET NOT NULL")
cur.execute(
"""
@@ -375,6 +409,7 @@ class PostgresPortfolioRepository:
updated_at=row[13] if len(row) > 13 else None,
last_trade_at=row[14] if len(row) > 14 else None,
managed_by_daemon_id=row[15] if len(row) > 15 else None,
asset_id=str(row[16]) if len(row) > 16 and row[16] is not None else None,
)
@staticmethod
@@ -383,6 +418,20 @@ class PostgresPortfolioRepository:
raise ValueError("태그를 찾을 수 없습니다")
return PortfolioTag(portfolio_id=row[0], tag=row[1], created_at=row[2] if len(row) > 2 else None)
@staticmethod
def _row_to_asset(row: Any) -> Asset:
if row is None:
raise ValueError("asset을 찾을 수 없습니다")
return Asset(
asset_id=str(row[0]),
symbol=row[1],
markets=list(row[2]) if row[2] else [],
display_name=row[3],
status=row[4] or "active",
created_at=row[5],
updated_at=row[6],
)
def _load_tags_for_portfolios(self, cur: psycopg.Cursor, portfolio_ids: list[str]) -> dict[str, list[str]]:
if not portfolio_ids:
return {}
@@ -737,7 +786,7 @@ class PostgresPortfolioRepository:
def get(self, portfolio_id: str) -> Portfolio:
self.ensure_schema()
with self._connect().cursor() as cur:
cur.execute(_SELECT_PORTFOLIO + "WHERE portfolio_id = %s", (portfolio_id,))
cur.execute(_SELECT_PORTFOLIO + "WHERE p.portfolio_id = %s", (portfolio_id,))
row = cur.fetchone()
portfolio = self._row_to_portfolio(row)
self._attach_tags(cur, [portfolio])
@@ -747,7 +796,7 @@ class PostgresPortfolioRepository:
self.ensure_schema()
with self._connect().cursor() as cur:
cur.execute(
_SELECT_PORTFOLIO + "WHERE owner_user_id = %s AND name = %s",
_SELECT_PORTFOLIO + "WHERE p.owner_user_id = %s AND p.name = %s",
(owner_user_id, name),
)
row = cur.fetchone()
@@ -772,10 +821,15 @@ class PostgresPortfolioRepository:
) -> Portfolio:
self.ensure_schema()
with self._connect().cursor() as cur:
cur.execute("SELECT asset_id FROM assets WHERE symbol = %s", (symbol,))
row = cur.fetchone()
if row is None:
raise ValueError(f"asset not found for symbol: {symbol}")
asset_id = row[0]
cur.execute(
"""
INSERT INTO portfolios (
portfolio_id, broker, status, name, symbol,
portfolio_id, broker, status, name, asset_id,
owner_user_id, cash_balance, asset_quantity, asset_price, profit_buffer,
active_strategy_id, avabile_strategies
)
@@ -787,7 +841,7 @@ class PostgresPortfolioRepository:
broker,
status,
name or symbol,
symbol,
asset_id,
owner_user_id,
float(initial_cash),
0.0,
@@ -797,7 +851,7 @@ class PostgresPortfolioRepository:
avabile_strategies,
),
)
cur.execute(_SELECT_PORTFOLIO + "WHERE portfolio_id = %s", (portfolio_id,))
cur.execute(_SELECT_PORTFOLIO + "WHERE p.portfolio_id = %s", (portfolio_id,))
row = cur.fetchone()
return self._row_to_portfolio(row)
@@ -809,7 +863,7 @@ class PostgresPortfolioRepository:
broker: str,
status: str,
name: str,
symbol: str,
asset_id: str,
cash_balance: float,
asset_quantity: float,
asset_price: float,
@@ -825,21 +879,17 @@ class PostgresPortfolioRepository:
cur.execute(
"""
INSERT INTO portfolios (
portfolio_id, broker, status, name, symbol,
portfolio_id, broker, status, name, asset_id,
owner_user_id, cash_balance, asset_quantity, asset_price, profit_buffer,
active_strategy_id, avabile_strategies
) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
RETURNING
portfolio_id, broker, status, name, symbol,
cash_balance, asset_quantity, asset_price, profit_buffer,
active_strategy_id, avabile_strategies, owner_user_id, created_at, updated_at, last_trade_at
""",
(
portfolio_id,
broker,
status,
name,
symbol,
asset_id,
owner_user_id,
float(cash_balance),
float(asset_quantity),
@@ -849,9 +899,8 @@ class PostgresPortfolioRepository:
avabile_strategies,
),
)
row = cur.fetchone()
portfolio = self._row_to_portfolio(row)
portfolio.tags = []
portfolio = self.get(portfolio_id)
portfolio.tags = []
return portfolio
def update_portfolio(self, portfolio_id: str, **fields: object) -> Portfolio:
@@ -860,7 +909,6 @@ class PostgresPortfolioRepository:
"broker": "broker",
"status": "status",
"name": "name",
"symbol": "symbol",
"asset_price": "asset_price",
"active_strategy_id": "active_strategy_id",
"avabile_strategies": "avabile_strategies",
@@ -887,15 +935,10 @@ class PostgresPortfolioRepository:
UPDATE portfolios
SET {', '.join(assignments)}
WHERE portfolio_id = %s
RETURNING
portfolio_id, broker, status, name, symbol,
cash_balance, asset_quantity, asset_price, profit_buffer,
active_strategy_id, avabile_strategies, owner_user_id, created_at, updated_at, last_trade_at
""",
(*params, portfolio_id),
)
row = cur.fetchone()
return self._row_to_portfolio(row)
return self.get(portfolio_id)
def list_portfolios(
self,
@@ -913,23 +956,23 @@ class PostgresPortfolioRepository:
clauses: list[str] = []
params: list[object] = []
if broker is not None:
clauses.append("broker = %s")
clauses.append("p.broker = %s")
params.append(broker)
if status is not None:
clauses.append("status = %s")
clauses.append("p.status = %s")
params.append(status)
if owner_user_id is not None:
clauses.append("owner_user_id = %s")
clauses.append("p.owner_user_id = %s")
params.append(owner_user_id)
if managed_by_daemon_id is not None:
clauses.append("managed_by_daemon_id = %s")
clauses.append("p.managed_by_daemon_id = %s")
params.append(managed_by_daemon_id)
normalized_mode = (match_mode or "OR").upper()
if include_tags:
if normalized_mode == "AND":
clauses.append(
"""
portfolio_id IN (
p.portfolio_id IN (
SELECT portfolio_id
FROM portfolio_tags
WHERE tag_value = ANY(%s)
@@ -942,17 +985,17 @@ class PostgresPortfolioRepository:
params.append(len(set(include_tags)))
else:
clauses.append(
"portfolio_id IN (SELECT DISTINCT portfolio_id FROM portfolio_tags WHERE tag_value = ANY(%s))"
"p.portfolio_id IN (SELECT DISTINCT portfolio_id FROM portfolio_tags WHERE tag_value = ANY(%s))"
)
params.append(include_tags)
if exclude_tags:
clauses.append(
"portfolio_id NOT IN (SELECT DISTINCT portfolio_id FROM portfolio_tags WHERE tag_value = ANY(%s))"
"p.portfolio_id NOT IN (SELECT DISTINCT portfolio_id FROM portfolio_tags WHERE tag_value = ANY(%s))"
)
params.append(exclude_tags)
if clauses:
query += "WHERE " + " AND ".join(clauses) + " "
query += "ORDER BY updated_at DESC, portfolio_id ASC"
query += "ORDER BY p.updated_at DESC, p.portfolio_id ASC"
with self._connect().cursor() as cur:
cur.execute(query, tuple(params))
rows = cur.fetchall()
@@ -1097,7 +1140,6 @@ class PostgresPortfolioRepository:
UPDATE portfolios
SET broker = %s,
status = %s,
symbol = %s,
cash_balance = %s,
asset_quantity = %s,
asset_price = %s,
@@ -1111,7 +1153,6 @@ class PostgresPortfolioRepository:
(
portfolio.broker,
portfolio.status,
portfolio.symbol,
float(portfolio.cash_balance),
float(portfolio.asset_quantity),
float(current_price),
@@ -1250,18 +1291,13 @@ class PostgresPortfolioRepository:
updated_at = NOW(),
last_trade_at = NULL
WHERE portfolio_id = %s
RETURNING
portfolio_id, broker, status, name, symbol,
cash_balance, asset_quantity, asset_price, profit_buffer,
active_strategy_id, avabile_strategies, owner_user_id, created_at, updated_at, last_trade_at
""",
(portfolio_id,),
)
row = cur.fetchone()
if row is None:
if cur.rowcount == 0:
raise ValueError("포트폴리오를 찾을 수 없습니다")
cur.execute("DELETE FROM portfolio_ledger WHERE portfolio_id = %s", (portfolio_id,))
return self._row_to_portfolio(row)
return self.get(portfolio_id)
def adjust_buffer(self, portfolio_id: str, amount: float, action: str, note: str = "") -> Portfolio:
self.ensure_schema()
@@ -1271,7 +1307,7 @@ class PostgresPortfolioRepository:
if amount <= 0:
raise ValueError("amount는 0보다 커야 합니다")
with self._connect().cursor() as cur:
cur.execute(_SELECT_PORTFOLIO + "WHERE portfolio_id = %s", (portfolio_id,))
cur.execute(_SELECT_PORTFOLIO + "WHERE p.portfolio_id = %s", (portfolio_id,))
row = cur.fetchone()
if row is None:
raise ValueError("포트폴리오를 찾을 수 없습니다")
@@ -1289,14 +1325,9 @@ class PostgresPortfolioRepository:
SET profit_buffer = %s,
updated_at = NOW()
WHERE portfolio_id = %s
RETURNING
portfolio_id, broker, status, name, symbol,
cash_balance, asset_quantity, asset_price, profit_buffer,
active_strategy_id, avabile_strategies, owner_user_id, created_at, updated_at, last_trade_at
""",
(updated_buffer, portfolio_id),
)
updated_row = cur.fetchone()
total_value = float(portfolio.cash_balance) + float(portfolio.asset_quantity) * float(portfolio.price)
cur.execute(
"""
@@ -1321,7 +1352,7 @@ class PostgresPortfolioRepository:
note,
),
)
return self._row_to_portfolio(updated_row)
return self.get(portfolio_id)
def transfer_buffer(self, portfolio_id: str, amount: float, direction: str, note: str = "") -> Portfolio:
self.ensure_schema()
@@ -1331,7 +1362,7 @@ class PostgresPortfolioRepository:
if amount <= 0:
raise ValueError("amount는 0보다 커야 합니다")
with self._connect().cursor() as cur:
cur.execute(_SELECT_PORTFOLIO + "WHERE portfolio_id = %s", (portfolio_id,))
cur.execute(_SELECT_PORTFOLIO + "WHERE p.portfolio_id = %s", (portfolio_id,))
row = cur.fetchone()
if row is None:
raise ValueError("포트폴리오를 찾을 수 없습니다")
@@ -1358,14 +1389,9 @@ class PostgresPortfolioRepository:
profit_buffer = %s,
updated_at = NOW()
WHERE portfolio_id = %s
RETURNING
portfolio_id, broker, status, name, symbol,
cash_balance, asset_quantity, asset_price, profit_buffer,
active_strategy_id, avabile_strategies, owner_user_id, created_at, updated_at, last_trade_at
""",
(cash, buf, portfolio_id),
)
updated_row = cur.fetchone()
total_value = float(cash) + float(portfolio.asset_quantity) * float(portfolio.price)
cur.execute(
"""
@@ -1390,7 +1416,82 @@ class PostgresPortfolioRepository:
note,
),
)
return self._row_to_portfolio(updated_row)
return self.get(portfolio_id)
def get_asset_by_id(self, asset_id: str) -> Asset | None:
self.ensure_schema()
with self._connect().cursor() as cur:
cur.execute(_SELECT_ASSET + "WHERE asset_id = %s", (asset_id,))
row = cur.fetchone()
return None if row is None else self._row_to_asset(row)
def get_asset_by_symbol(self, symbol: str) -> Asset | None:
self.ensure_schema()
with self._connect().cursor() as cur:
cur.execute(_SELECT_ASSET + "WHERE symbol = %s", (symbol,))
row = cur.fetchone()
return None if row is None else self._row_to_asset(row)
def list_assets(self) -> list[Asset]:
self.ensure_schema()
with self._connect().cursor() as cur:
cur.execute(_SELECT_ASSET + "ORDER BY symbol ASC")
rows = cur.fetchall()
return [self._row_to_asset(row) for row in rows]
def insert_asset(
self,
*,
symbol: str,
markets: list[str],
display_name: str | None = None,
status: str = "active",
) -> Asset:
self.ensure_schema()
with self._connect().cursor() as cur:
cur.execute(
"""
INSERT INTO assets (symbol, markets, display_name, status)
VALUES (%s, %s, %s, %s)
RETURNING asset_id, symbol, markets, display_name, status, created_at, updated_at
""",
(symbol, markets, display_name, status),
)
row = cur.fetchone()
return self._row_to_asset(row)
def update_asset(
self,
asset_id: str,
*,
markets: list[str] | None = None,
display_name: str | None = None,
status: str | None = None,
) -> Asset:
self.ensure_schema()
assignments: list[str] = ["updated_at = NOW()"]
params: list[object] = []
if markets is not None:
assignments.append("markets = %s")
params.append(markets)
if display_name is not None:
assignments.append("display_name = %s")
params.append(display_name)
if status is not None:
assignments.append("status = %s")
params.append(status)
params.append(asset_id)
with self._connect().cursor() as cur:
cur.execute(
f"""
UPDATE assets SET {', '.join(assignments)}
WHERE asset_id = %s
""",
tuple(params),
)
if cur.rowcount == 0:
raise ValueError(f"asset을 찾을 수 없습니다: {asset_id}")
return self.get_asset_by_id(asset_id) # type: ignore[return-value]
def delete(self, portfolio_id: str, *, clear_ledger: bool = True) -> None:
self.ensure_schema()