145 lines
6.6 KiB
Markdown
145 lines
6.6 KiB
Markdown
# REL-011 M5 trex-core operator broker truth hotfix — deployment runbook
|
||
|
||
- Release: `REL-011`
|
||
- Target source: `6d90f8951dbf354eb41301492d0bcaf0af24d882`
|
||
- Runtime implementation: BAT-057 / PR #42 merge `94e148940adb008db0bbd215052a1e984f7c566b`
|
||
- Execution owner: 운영팀
|
||
- Approval required: team lead review + user release approval
|
||
- Safety classification: production daemon restart; no DB migration; broker-read smoke only
|
||
|
||
> Do not paste credentials, tokens, account numbers, DSNs, `.env` contents, raw KIS responses, or full command lines containing secrets into the evidence log. Use `[REDACTED]`.
|
||
|
||
## 1. Scope and hard stops
|
||
|
||
Allowed after release approval:
|
||
|
||
- Pinned source checkout, existing trex-core daemon stop/start, startup log inspection.
|
||
- Existing approved instance configuration used without semantic changes.
|
||
- `sync --all --dry-run` and `verify` as broker-read/CLI smoke checks.
|
||
|
||
Not allowed in this runbook:
|
||
|
||
- `sync --live`, `--yes` with a live write, `enter`, `exit`, `cancel`, or any automated remediation.
|
||
- Changing `credential_fetch_enabled`, `TREX_KIS_BROKER_BREAK_GLASS`, broker mode, instance key, portfolio-hub endpoint, or secrets.
|
||
- Database schema migration, manual DB update/delete, AMQP topology change, or strategy daemon deployment.
|
||
|
||
If any prohibited action appears necessary, stop and return to the release approver.
|
||
|
||
## 2. Preflight — record before modifying the host
|
||
|
||
1. Confirm the approved release, target SHA, and approved 장외 window.
|
||
2. From the operating checkout, capture only redacted metadata:
|
||
|
||
```bash
|
||
pwd
|
||
git status --short
|
||
git rev-parse HEAD
|
||
git rev-parse --abbrev-ref HEAD
|
||
```
|
||
|
||
- Record the exact pre-deploy revision as `ROLLBACK_REVISION`.
|
||
- If the checkout is dirty or target source cannot be verified, **stop**.
|
||
|
||
3. Identify the active instance configuration and process supervisor without printing secrets. Confirm the existing config’s `credential_fetch_enabled` and break-glass state will remain unchanged.
|
||
4. Confirm the expected process command remains the established runtime form:
|
||
|
||
```bash
|
||
.venv/bin/trex-core-daemon --config <approved-instance-config>
|
||
```
|
||
|
||
5. Confirm target/source characteristics from a clean checkout:
|
||
|
||
```bash
|
||
git fetch origin
|
||
git show -s --format='%H %s' 6d90f8951dbf354eb41301492d0bcaf0af24d882
|
||
git diff --name-only "$ROLLBACK_REVISION" 6d90f8951dbf354eb41301492d0bcaf0af24d882 -- pyproject.toml trex_core/adapters/persistence/migrations
|
||
```
|
||
|
||
Expected for the reviewed M4 baseline: no dependency or migration path. If actual output differs, **stop** and request lead review; do not infer safety from this draft.
|
||
|
||
6. Ensure the deployment host’s virtual environment has the existing runtime dependencies. Do not run dependency upgrades as an implicit part of this release.
|
||
|
||
## 3. Deploy source and restart daemon
|
||
|
||
Use the host’s approved supervisor procedure. The following expresses the source boundary, not a replacement for host-specific service management:
|
||
|
||
```bash
|
||
git fetch origin
|
||
git checkout --detach 6d90f8951dbf354eb41301492d0bcaf0af24d882
|
||
# Stop the existing trex-core daemon through the approved host supervisor.
|
||
# Start exactly one daemon with the unchanged approved instance config.
|
||
.venv/bin/trex-core-daemon --config <approved-instance-config>
|
||
```
|
||
|
||
Record, with sensitive values redacted:
|
||
|
||
- checkout SHA;
|
||
- prior process stop confirmation;
|
||
- new process identifier or supervisor status;
|
||
- selected instance key (not account details);
|
||
- startup log interval and log location.
|
||
|
||
## 4. Startup acceptance
|
||
|
||
Before any CLI smoke, confirm:
|
||
|
||
- the daemon completed startup and RabbitMQ topology declaration;
|
||
- worker threads are running according to the instance profile;
|
||
- no `ERROR`, `CRITICAL`, or traceback appears in the post-restart interval;
|
||
- the daemon is using the unchanged configuration mode. Source evidence may show only safe mode/label (`fetched`, `env`, or `break_glass`), never credential payload or account number.
|
||
|
||
Any KIS credential-resolution or balance-read failure must be treated as a failure; BAT-057 intentionally prevents treating it as a normal empty broker balance.
|
||
|
||
## 5. Read-only/dry-run smoke
|
||
|
||
Run against the same explicit approved configuration. These calls may read the configured broker and database but must not write a slot, warning, broker order, or ledger record.
|
||
|
||
```bash
|
||
.venv/bin/trex-core sync --all --dry-run --config <approved-instance-config> --json
|
||
.venv/bin/trex-core verify --config <approved-instance-config> --json
|
||
```
|
||
|
||
Acceptance:
|
||
|
||
- Commands either succeed with sanitized source/balance evidence, or fail loudly with safe error code/message.
|
||
- A failure must not be reinterpreted as an empty balance or “zero quantity” condition.
|
||
- For `unmanaged_position` or `managed_non_holding_divergence`, record classification only and stop for triage; no remediation is performed.
|
||
- Never add `--live` or `--yes` in this runbook.
|
||
|
||
## 6. Rollback
|
||
|
||
Trigger rollback for any daemon startup failure, unexpected source-mode change, unexpected credential fetch/fallback behavior, broker-read failure not safely surfaced, AMQP connectivity regression, or smoke behavior inconsistent with BAT-057’s fail-loud/no-auto-remediation contract.
|
||
|
||
```bash
|
||
# Stop the new daemon via the approved host supervisor.
|
||
git checkout --detach "$ROLLBACK_REVISION"
|
||
# Restart with the same unchanged approved configuration via the supervisor.
|
||
```
|
||
|
||
Then confirm the previous daemon revision is running and collect redacted logs. Do not run database rollback commands; REL-011 introduces no migration.
|
||
|
||
## 7. Evidence record template
|
||
|
||
```text
|
||
Release: REL-011
|
||
Approved by / timestamp: [REDACTED AS NEEDED]
|
||
Window: [REDACTED]
|
||
Host identity: [REDACTED]
|
||
Instance key: [REDACTED]
|
||
Pre-deploy revision (ROLLBACK_REVISION): <sha>
|
||
Target revision: 6d90f8951dbf354eb41301492d0bcaf0af24d882
|
||
Configuration mode unchanged: yes/no (credential_fetch_enabled / break-glass values not recorded)
|
||
Daemon stop/start evidence: <supervisor status or PID, redacted>
|
||
Startup log interval: <path/time range, redacted>
|
||
Startup acceptance: pass/fail
|
||
sync --dry-run result: pass/fail + sanitized exit code/output summary
|
||
verify result: pass/fail + sanitized exit code/output summary
|
||
Unexpected divergence classification: none / unmanaged_position / managed_non_holding_divergence
|
||
Rollback performed: no / yes (reason and restored SHA)
|
||
Operator: [REDACTED]
|
||
```
|
||
|
||
## 8. Completion criteria
|
||
|
||
REL-011 may move from `Release-Pending` only after the exact target revision, restart evidence, startup acceptance, dry-run smoke result, verify result, and rollback disposition are reviewed by the team lead. Production live sync remains a separate approval gate even after REL-011 is released.
|