jarca-ledger-mcp/docs/batches/bat-006_20260630_pytest-apptest-smoke-ci-automation/verification.md

BAT-006 Verification

Verification Status

• Status: PASS_WITH_NOTES
• Ready for Release: true
• Ready for QC: false
• Verified At (KST): 2026-06-30 15:29 KST
• Verified By: jarcaledgerqc
• QC Verdict: PASS_WITH_NOTES
• QC Boundary: satisfied

Acceptance Mapping

REQ-006 AC-1

.github/workflows/ 아래에 GitHub Actions workflow가 생성되고, pull_request, main 대상 push, workflow_dispatch 트리거가 정의된다.

Evidence:

• Workflow file: .github/workflows/ci.yml.
• Trigger excerpt verified in file:
  • pull_request:
  • push: with branches: [main]
  • workflow_dispatch:
• Stable job/check names in workflow:
  • pytest-regression
  • dashboard-app-test
  • dashboard-http-smoke
  • db-safety-smoke

Result: PASS.

REQ-006 AC-2

CI에서 pytest 회귀 스위트가 자동 실행되고, 단일 테스트 실패 시 workflow가 실패 상태로 종료된다.

Evidence:

• Workflow job: pytest-regression.
• Workflow command: uv run pytest --ignore=tests/test_dashboard_app.py.
• Local equivalent command:
  • uv run pytest --ignore=tests/test_dashboard_app.py -q
  • Result: PASS, exit code 0.
  • Output: ............ss...............ss... [100%]
• GitHub Actions run: step uses normal shell exit-code behavior, so a pytest failure fails the job/workflow.

Result: PASS.

REQ-006 AC-3

CI에서 Streamlit AppTest가 synthetic fixture DB 기준으로 자동 실행되고 통과/실패 결과가 로그에 남는다.

Evidence:

• Workflow job: dashboard-app-test.
• Workflow command: uv run pytest tests/test_dashboard_app.py.
• Test evidence: tests/test_dashboard_app.py calls create_ci_fixture_db(tmp_path / "fixture.sqlite3"), sets JARCA_LEDGER_DB_PATH, and runs Streamlit AppTest across core pages.
• Local equivalent command:
  • uv run pytest tests/test_dashboard_app.py -q
  • Result: PASS, exit code 0.
  • Output: .... [100%]
• Fixture policy: scripts/ci_fixture_db.py refuses DEFAULT_DB_PATH and seeds synthetic data only.

Result: PASS.

REQ-006 AC-4

CI에서 대시보드 HTTP smoke가 fixture DB 기준으로 Streamlit을 기동한 뒤 HTTP 200 응답을 확인하고, 종료 후 프로세스를 정리한다.

Evidence:

• Workflow job: dashboard-http-smoke.
• Workflow command: uv run python scripts/ci_dashboard_http_smoke.py --host 127.0.0.1 --port 18501 --timeout 45.
• Wrapper: scripts/ci_dashboard_http_smoke.py creates a temp synthetic fixture DB, starts Streamlit with JARCA_LEDGER_DB_PATH, polls a loopback URL for HTTP 200, and performs process cleanup.
• Local command:
  • uv run python scripts/ci_dashboard_http_smoke.py --host 127.0.0.1 --port 18501 --timeout 45
  • Result: PASS, exit code 0.
  • Output summary: success: true, fixture: synthetic, fixture_integrity: ok, http_status: 200, attempts: 3, cleanup: terminated.

Result: PASS.

REQ-006 AC-5

CI에서 DB safety smoke가 운영 DB 기본 경로를 사용하지 않고 임시 fixture/검증 DB에서 통과한다.

Evidence:

• Workflow job: db-safety-smoke.
• Workflow command: uv run python scripts/ci_db_safety_smoke.py --timeout 45.
• Wrapper: scripts/ci_db_safety_smoke.py creates temp source, backup, and target DB files and refuses DEFAULT_DB_PATH before invoking scripts/ledger_restore_smoke.py.
• Local command:
  • uv run python scripts/ci_db_safety_smoke.py --timeout 45
  • Result: PASS, exit code 0.
  • Output summary: success: true, fixture: synthetic, fixture_integrity: ok, restore_smoke_exit_code: 0, restored_exists: true, restored_integrity: ok, health_success: true, operating_db_used: false.

Result: PASS.

REQ-006 AC-6

HTTP smoke/DB safety smoke는 CI 전용 wrapper entrypoint를 통해 실행되며, wrapper가 종료코드·timeout·readiness·cleanup을 일관되게 처리한다.

Evidence:

• HTTP wrapper path: scripts/ci_dashboard_http_smoke.py.
  • Readiness: polls URL until HTTP 200 or timeout.
  • Timeout/failure: returns non-zero status with status-only JSON.
  • Cleanup: terminates/kills Streamlit process and reports cleanup state.
• DB safety wrapper path: scripts/ci_db_safety_smoke.py.
  • Timeout: catches subprocess.TimeoutExpired and returns non-zero status with timeout JSON.
  • Exit code: reports restore smoke exit code and returns non-zero on failed restore/integrity/health conditions.
  • Safety: refuses operating DB path and reports operating_db_used: false on pass.
• Local command results for both wrappers exited 0 and reported success JSON.

Result: PASS.

REQ-006 AC-7

CI 로그·아티팩트에는 실계좌번호, 실잔액, raw_payload, 원본 금융 파일명, 비밀번호가 평문으로 포함되지 않는다.

Evidence:

• Fixture builder uses synthetic labels such as CI Synthetic Checking, CI_SYNTHETIC_*, and ci synthetic ...; it does not load real statement files or passwords.
• Wrapper outputs are status-only JSON. Observed local outputs include gate name, fixture type, integrity, HTTP status, cleanup, restore smoke status, and operating_db_used: false; they do not include account numbers, real balances, raw payloads, source financial filenames, or passwords.
• scripts/ci_fixture_db.py refuses the operating ledger DB default path.

Result: PASS.

REQ-006 AC-8

docs/project-structure.md와 docs/runtime-flows.md가 CI 게이트와 로컬 browser evidence 경계를 반영한다.

Evidence:

• docs/project-structure.md documents .github/workflows/ci.yml, stable gate names, scripts/ci_fixture_db.py, scripts/ci_dashboard_http_smoke.py, scripts/ci_db_safety_smoke.py, and synthetic fixture policy.
• docs/runtime-flows.md includes Flow: CI validation gates and the constraints that CI gates use synthetic/temp DBs only.
• docs/runbooks/ci-smoke.md provides reproduction commands and failure triage.

Result: PASS.

REQ-006 AC-9

browser-screenshot E2E는 CI 필수 차단 게이트에서 제외되고, dcy-004 로컬 evidence 절차와의 역할 분담이 문서에 명시된다.

Evidence:

• .github/workflows/ci.yml contains the four required CI jobs and no browser screenshot/visual regression required job.
• docs/runtime-flows.md states browser screenshot/DOM evidence is not a required GitHub Actions gate in BAT-006 and remains local evidence for visual/layout-sensitive dashboard changes.
• docs/runbooks/ci-smoke.md states scripts/streamlit_browser_evidence.py remains local evidence and is not a required GitHub Actions gate.

Result: PASS.

Command Evidence Summary

All commands were run from /root/projects/jarca-ledger-mcp.

Command	Result	Evidence
uv run pytest --ignore=tests/test_dashboard_app.py -q	PASS, exit code 0	............ss...............ss... [100%]
uv run pytest tests/test_dashboard_app.py -q	PASS, exit code 0	.... [100%]
uv run python scripts/ci_dashboard_http_smoke.py --host 127.0.0.1 --port 18501 --timeout 45	PASS, exit code 0	success: true, http_status: 200, cleanup: terminated
uv run python scripts/ci_db_safety_smoke.py --timeout 45	PASS, exit code 0	success: true, restored_integrity: ok, health_success: true, operating_db_used: false

Sensitive Evidence Review

• No command output observed in this verification exposed real account numbers, operating DB balances, source financial filenames, passwords, or raw_payload.
• CI fixture/smoke paths are synthetic/temp paths and refuse the default operating ledger DB path.
• Remote GitHub Actions logs were not available in this local handoff; jarcaledgerqc recorded this as a note rather than a blocker because local equivalent commands passed and the workflow wiring is present.

Blocking Issues

• None for local QC readiness.
• QC note: Remote GitHub Actions execution and branch protection required-check setup remain repository/remote configuration checks outside this local evidence update.