portfolio-hub/docs/runbooks/rel-004-m5-manual-ledger-deployment-runbook.md

REL-004 운영 배포 런북 — M5 수동 장부 API 및 ledger metadata migration

• Service: portfolio-hub
• Release: REL-004
• Target source baseline: origin/main@914b3dc6e6b9e2cbd7068fb30435c83f0581eab5
• Database revision: 20260715_0028
• Classification: production DB migration + app-service deployment
• Required approver: Justin Ko
• Required operator: trex-op
• Estimated active window: 20–40 minutes plus observation

Do not begin from this document alone. Production execution needs an explicit REL-004 approval, a maintenance window, and the exact deployment-platform command approved by trex-op. This repository contains no production Docker Compose, systemd, Kubernetes, or Helm manifest; do not infer one from docker-compose.local-test.yml, which is local-test infrastructure only.

1. Safety model

• The migration is additive. It adds/normalizes four portfolio_ledger metadata columns and records Alembic revision 20260715_0028.
• The permitted recovery path after a successful migration is compatible code rollback while retaining the schema.
• alembic downgrade, column deletion, and automatic pg_restore are prohibited in this runbook. Any restore is a separately approved incident recovery action.
• The new API can mutate ledger state. A production write smoke requires an explicitly designated test portfolio, a named owner, and approval; otherwise stop after read-only health and schema checks.
• Do not paste DB passwords, bearer tokens, credential payloads, or backups into this document, chat, CI logs, or release evidence. Record secret-free command outcomes only.

2. Roles and stop authority

Role	Responsibility
Justin Ko	grants release-window approval; decides whether production mutation smoke is allowed; approves exceptional restore
trex-op	captures evidence, performs backup/preflight/migration/deployment using the real platform tooling, stops on any failed gate
TREX lead	reviews completed evidence and records final release decision; does not perform production commands under this draft

Any operator may stop the rollout for a failed gate, unexpected active workload, missing backup, wrong baseline, migration error, or unexpected API behavior.

3. Inputs to obtain before window

• approved window and approver/time recorded in REL-004
• immutable deployable artifact built from 914b3dc or reviewed descendant, with image/package digest
• exact production deployment command supplied by trex-op’s platform (for example, its existing rollout command); do not substitute a local-test Compose command
• secure database environment available to the migration runner:
  • TIMESCALEDB_HOST, TIMESCALEDB_PORT, TIMESCALEDB_NAME, TIMESCALEDB_USER, TIMESCALEDB_PASSWORD
  • PORTFOLIOHUB_DB_ENABLED=true
• confirmed DB backup destination and restore owner
• approved health base URL
• optional, separately approved test portfolio ID and owner-token handling plan for mutation smoke

4. Preflight — capture, then decide go/no-go

Run from the approved artifact or a clean checkout at the approved source baseline, with the secured DB environment loaded.

# Identity: must resolve to the approved release source or its artifact provenance.
git rev-parse HEAD
alembic heads
alembic current

Expected head is 20260715_0028. current may be an earlier valid deployed revision before the upgrade, but an unknown/branched state is a stop condition.

Capture active work and locks before touching schema:

SELECT pid, usename, application_name, state, wait_event_type, wait_event, query
FROM pg_stat_activity
WHERE query ILIKE '%portfolio_ledger%'
  AND state <> 'idle';

SELECT locktype, mode, granted, pid, relation::regclass
FROM pg_locks
WHERE relation = 'portfolio_ledger'::regclass;

Capture schema and row-count baseline:

SELECT column_name, data_type, is_nullable, column_default
FROM information_schema.columns
WHERE table_name = 'portfolio_ledger'
  AND column_name IN ('source', 'target_ledger_entry_id', 'confirmation_evidence', 'actor_user_id')
ORDER BY column_name;

SELECT COUNT(*) AS rows_before FROM portfolio_ledger;
SELECT COUNT(*) AS null_source_before FROM portfolio_ledger WHERE source IS NULL;

Go only when: backup is complete, active workload/locks are absent or the window owner explicitly accepts them, heads has one expected head, and the current state is understood. Otherwise stop and record the result.

5. Backup gate

Perform the established production backup procedure. The following is an interface, not a command to run blindly:

<ops-approved pg_dump/snapshot procedure> → <backup reference>

Record in REL-004:

• backup reference/path (not credentials), timestamp, byte size/checksum if available;
• DB/database identifier;
• restore owner and how restore viability is known.

Stop if the backup has not completed or cannot be located by the restore owner.

6. Migration gate

Execute exactly the reviewed target revision:

alembic upgrade 20260715_0028
alembic current

Then execute and retain secret-free output for:

SELECT column_name, data_type, is_nullable, column_default
FROM information_schema.columns
WHERE table_name = 'portfolio_ledger'
  AND column_name IN ('source', 'target_ledger_entry_id', 'confirmation_evidence', 'actor_user_id')
ORDER BY column_name;

SELECT COUNT(*) AS rows_after FROM portfolio_ledger;
SELECT COUNT(*) AS null_source_rows
FROM portfolio_ledger
WHERE source IS NULL;

SELECT column_name, is_nullable
FROM information_schema.columns
WHERE table_name = 'portfolio_ledger' AND column_name = 'source';

SELECT column_default
FROM information_schema.columns
WHERE table_name = 'portfolio_ledger' AND column_name = 'source';

Pass criteria:

• alembic current reports 20260715_0028;
• all four columns exist with the REL-004/REQ-057 types;
• source is NOT NULL, default is 'broker', and null_source_rows = 0;
• rows_after = rows_before.

Stop on a migration error, mismatched revision, missing/incorrect column, remaining null source, or row-count mismatch. Do not start/restart the new service code.

7. Artifact deployment and activation gate

Only after Section 6 passes:

1. Record the exact immutable artifact ref/digest and source provenance in REL-004.
2. Use the already-approved production deploy/rolling-restart procedure maintained by trex-op.
3. Capture the deployment command class, rollout/restart result, active artifact digest, and service logs/status without secrets.
4. Do not invoke ensure_schema() as a substitute for this reviewed Alembic preflight/migration flow.

There is no repo-owned production rollout command. docker-compose.local-test.yml must not be used for a production service replacement.

8. Post-deploy verification

8.1 Mandatory read-only checks

curl --fail-with-body --silent --show-error "$PORTFOLIO_HUB_BASE_URL/health"

Pass: HTTP 200 and JSON includes db_enabled: true.

Use authorized existing read probes to check that the service is reachable and that existing portfolio reads still work. Do not log bearer tokens or response PII in REL-004.

8.2 Manual-ledger API authorization checks

With an approved non-sensitive test identity or non-production endpoint:

• owner user/session token: GET /v1/portfolios/{portfolio_id}/ledger/manual succeeds with HTTP 200;
• service/daemon token calling POST .../ledger/manual is rejected HTTP 403;
• non-owner user/session token is rejected HTTP 403.

8.3 Optional mutation smoke — only with explicit approval

Only after Justin Ko explicitly authorizes the named test portfolio and operation:

• submit one bounded INVEST entry through the new manual endpoint;
• verify HTTP 201, returned source="manual", and expected after-snapshot;
• retrieve the entry via manual history;
• if business policy permits, exercise approved reversal of that exact test entry and verify append-only behavior.

This is business-data mutation. Never use an unapproved customer, live strategy-managed, or production trading portfolio as a convenience test target. A daemon-managed portfolio needs the REQ-056 confirmation evidence flow.

9. Observation and stop/rollback decision

Observe the service for the agreed window. Stop and roll back code if health fails, 5xx/authorization anomalies appear, or existing /ledger/trades/portfolio reads regress.

• Migration not successful: do not deploy the new artifact; preserve command output and consult DB owner.
• Migration successful, code problem: use the platform’s approved rollback to a reviewed compatible prior artifact; retain revision 20260715_0028 and all four columns.
• Restore needed: halt and obtain separate incident/recovery approval. Do not self-initiate downgrade, column deletion, or restore.

10. Evidence handoff template

Send the following completed facts to the TREX lead for REL-004 decision:

REL-004 execution
- approval: <approver, time, window>
- operator: <name>
- source/artifact: <commit>, <immutable digest>
- backup: <reference, time, size/checksum, restore owner>
- preflight: <heads/current>, <locks result>, <rows_before>, <null_source_before>
- migration: <command exit/result>, <current_after>, <rows_after>, <null_source_rows>, <schema/default evidence>
- deployment: <platform procedure/result>, <active artifact digest>
- read-only smoke: <health result>, <existing read probes>
- authorization smoke: <owner/read, service 403, non-owner 403>
- mutation smoke: <not authorized | test portfolio approval + result>
- observation: <duration, alerts/errors>
- rollback: <not needed | reason/action>