diff --git a/harness/agents/impl/skills/gitea-issue-rework/references/readback.md b/harness/agents/impl/skills/gitea-issue-rework/references/readback.md index 809b4ed..5235051 100644 --- a/harness/agents/impl/skills/gitea-issue-rework/references/readback.md +++ b/harness/agents/impl/skills/gitea-issue-rework/references/readback.md @@ -264,7 +264,7 @@ state deltas plus exact timestamps are not independent causal-event attestation. Run focused tests with `terminal`: ```sh -python3 -B -m unittest harness.tests.impl.test_rework_readback harness.tests.impl.test_rework_readback_transport harness.tests.impl.test_rework_comment -v +python3 -B -m unittest harness.tests.impl.rework.test_rework_readback_provider harness.tests.impl.rework.test_rework_readback_cache harness.tests.impl.rework.test_rework_readback_bridge harness.tests.impl.rework.test_rework_readback_transport harness.tests.impl.rework.test_rework_comment -v python3 -B harness/tests/run.py ``` diff --git a/harness/agents/impl/skills/gitea-issue-rework/references/workflow.md b/harness/agents/impl/skills/gitea-issue-rework/references/workflow.md index 5158f4b..9926c5d 100644 --- a/harness/agents/impl/skills/gitea-issue-rework/references/workflow.md +++ b/harness/agents/impl/skills/gitea-issue-rework/references/workflow.md @@ -325,7 +325,7 @@ def normalize_commit_files(rows): ``` -`harness/tests/impl/test_impl_architecture_docs.py`는 이 정본 block 자체를 추출·실행하여 순서 차이만 허용되는지, 실제 metadata/identity 변화·중복·malformed input을 거부하거나 비동등으로 판정하는지, 원본과 nested value가 불변인지 검증한다. 테스트에 comparator를 복제하지 않는다. 이 실행 회귀는 비교 예제의 local contract 검증이며 live provider capability, collection admission 또는 실제 전달 성공을 대신하지 않는다. +`harness/tests/impl/contracts/test_impl_architecture_comparator.py`는 이 정본 block 자체를 추출·실행하여 순서 차이만 허용되는지, 실제 metadata/identity 변화·중복·malformed input을 거부하거나 비동등으로 판정하는지, 원본과 nested value가 불변인지 검증한다. 테스트에 comparator를 복제하지 않는다. 이 실행 회귀는 비교 예제의 local contract 검증이며 live provider capability, collection admission 또는 실제 전달 성공을 대신하지 않는다. ### 7. 기존 PR branch 전달 diff --git a/harness/agents/review/skills/gitea-code-review/references/native-review.md b/harness/agents/review/skills/gitea-code-review/references/native-review.md index 3c443c2..4f3fac5 100644 --- a/harness/agents/review/skills/gitea-code-review/references/native-review.md +++ b/harness/agents/review/skills/gitea-code-review/references/native-review.md @@ -136,7 +136,7 @@ python3 "$HARNESS_ROOT/harness/agents/review/skills/gitea-code-review/scripts/na --plan "$RUN/plan.json" --baseline "$RUN/before.json" python3 "$HARNESS_ROOT/harness/agents/review/skills/gitea-code-review/scripts/native_review.py" verify \ --plan "$RUN/plan.json" --before "$RUN/before.json" --after "$RUN/after.json" --exact "$RUN/exact.json" -PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s harness/tests/review -p 'test_native_review.py' -v +PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s harness/tests/review/native -t . -p 'test_*.py' -v PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s harness/tests/spec -v PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s harness/tests/impl -v ``` diff --git a/harness/agents/spec/skills/gitea-implementation-spec-review/SKILL.md b/harness/agents/spec/skills/gitea-implementation-spec-review/SKILL.md index 73e2ca1..80e3737 100644 --- a/harness/agents/spec/skills/gitea-implementation-spec-review/SKILL.md +++ b/harness/agents/spec/skills/gitea-implementation-spec-review/SKILL.md @@ -237,7 +237,7 @@ terminal(command="python3 harness/agents/spec/skills/gitea-implementation-spec-r - 예시 provider-contract JSON: `{"version":{"version":"1.27.3"},"requested_reviewers_presentation_order":true}`. 기존 supersession/retouch/official 정책이 필요하면 각각 independently verified 값을 명시적으로 유지한다. 정본 계약 위치는 이 문서이며 기존 문서를 update-in-place한다; 회차별 artifact에서 별도 verifier나 snapshot normalization으로 우회하지 않는다. - `check-unchanged --before --after `는 read-only replay의 exact snapshot 동일성을 확인한다. Helper 성공은 호출자가 실제 write를 하지 않았다는 증명이나 최종 리뷰 완료를 단독으로 의미하지 않는다. - Helper는 credential을 발견하거나 자체 network client·외부 write를 제공하지 않고 acceptance 의미·evidence 유효성·verdict를 결정하지 않는다. Read-only collector가 호출하는 authenticated GET callback과 provider envelope 변환은 main의 기존 adapter 책임이다. 실제 secret 부재는 main이 별도로 확인하며 알려진 민감값은 Python API의 `sensitive_values`로 메모리 안에서만 전달한다. 입력 오류는 nonzero로 중단하고 입력 본문·credential을 error에 출력하지 않는다. -- 회귀 검증은 `terminal(command="PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s harness/tests/spec -p 'test_implementation_spec_review_support.py' -v")`로 실행한다. Schema/body/provider 처리 변경 시 같은 schema의 저장된 실제 wire artifact로 render·parse byte 일치와 publication·read-only replay도 검증한다. Helper 검증을 위해 새 외부 write를 하지 않으며 과거 artifact replay를 fresh authenticated readback으로 보고하지 않는다. +- 회귀 검증은 `terminal(command="PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s harness/tests/spec/support -t . -p 'test_*.py' -v")`로 실행한다. Schema/body/provider 처리 변경 시 같은 schema의 저장된 실제 wire artifact로 render·parse byte 일치와 publication·read-only replay도 검증한다. Helper 검증을 위해 새 외부 write를 하지 않으며 과거 artifact replay를 fresh authenticated readback으로 보고하지 않는다. ### Source Issue 자동 참조 projection @@ -249,7 +249,7 @@ terminal(command="python3 harness/agents/spec/skills/gitea-implementation-spec-r - Source Issue의 `updated_at`은 그 신규 reference event의 동일한 `created_at`/`updated_at`에 정확히 일치해야 하며 event는 review event보다 이전일 수 없다. 허용 timestamp는 Source Issue에만 적용한다. PR·PR Issue·historical review timestamp 규칙, Source Issue body·title·state·labels·assignees·milestone·normal comments와 기존 scope 보호를 완화하지 않는다. 초 단위 허용 오차나 timestamp 일괄 제외를 추가하지 않는다. - 독립 before/after `pr_issue`는 exact target·author·open/unmerged 상태를 확인하고, verified PR review event 시각의 `updated_at`을 제외한 모든 field를 보존한다. 일반 댓글 수인 `pr_issue.comments`는 before/after 모두 boolean이 아닌 0 이상의 integer이며 정확히 같아야 한다. Native top-level `CommentTypeReview`는 일반 댓글 수를 증가시키지 않는다(`models/issues/comment.go`의 `updateCommentInfos`: `UpdateIssueNumComments`는 `CommentTypeComment`/제출된 `CommentTypeCode` 경로에만 적용). 별도 PullRequest record의 `pr.review_comments`는 검증한 신규 review 하나에 대해 정확히 1 증가해야 하며, 이 검증은 유지한다. 두 카운터를 혼동하거나 일반 댓글 증가·감소·누락·잘못된 타입을 projection으로 허용하지 않는다. Source timeline의 기존 event는 ID·순서·type·author·body·ref_comment와 나머지 field를 보존한다. 단, provider가 참조 대상의 현재 Issue record를 serialize하므로 기존 event의 `ref_issue`만 달라질 수 있다. 이 nested projection은 old/new 값 전체가 독립 before/after `pr_issue`와 각각 exact-match할 때만 허용한다. 다른 Issue 참조, 임의 field 변경, baseline 제거·순서 변경·추가 경쟁 event는 차단한다. - `verify-publication`의 CLI 인자는 그대로이며 opt-in provider-contract와 위 9-field before/after snapshot을 전달한다. 성공 출력의 `source_issue_reference_ids`는 검증한 신규 Source event ID 하나다. Opt-in 없는 legacy snapshot은 기존 strict 규칙을 유지하고 확대 field를 자동 채택하지 않는다. Write 직전에는 확대 snapshot 전체를 `check-unchanged`로 baseline과 대조하고, 실패 후 추가 POST를 하지 않는다. -- 회귀 검증: `PYTHONDONTWRITEBYTECODE=1 python3 -m unittest harness.tests.spec.test_source_issue_reference_projection -v`. Collector → snapshot → CLI verifier의 로컬 검증과 saved-wire compatibility replay는 실제 새 게시 성공이 아니다. 과거 before evidence에 Source timeline 또는 독립 PR Issue가 없으면 재구성한 입력으로 과거 `partial`을 소급 PASS로 바꾸지 않는다. +- 회귀 검증: `PYTHONDONTWRITEBYTECODE=1 python3 -m unittest harness.tests.spec.projection.test_source_issue_reference -v`. Collector → snapshot → CLI verifier의 로컬 검증과 saved-wire compatibility replay는 실제 새 게시 성공이 아니다. 과거 before evidence에 Source timeline 또는 독립 PR Issue가 없으면 재구성한 입력으로 과거 `partial`을 소급 PASS로 바꾸지 않는다. ## 절차 diff --git a/harness/tests/README.md b/harness/tests/README.md index 610f8f4..d2a8691 100644 --- a/harness/tests/README.md +++ b/harness/tests/README.md @@ -72,16 +72,17 @@ unauthenticated-token rejection behavior. Rework readback contract: the impl role's canonical `rework_comment.py snapshot` and `verify-live` commands collect authenticated GET-only provider state through `rework_readback.py` and `rework_readback_cli.py`. Focused coverage lives in -`harness/tests/impl/test_rework_readback.py` and -`harness/tests/impl/test_rework_readback_transport.py`; run them with the existing -`harness/tests/impl/test_rework_comment.py` regressions. They exercise real +the provider/cache/bridge modules under `harness/tests/impl/rework/` and +`harness/tests/impl/rework/test_rework_readback_transport.py`; run them with the +`harness/tests/impl/rework/test_rework_comment.py` regressions. They exercise real renderer/verifier integration and a local HTTP fixture, not live Gitea access. Provider/baseline fingerprints, exact identity/endpoints, complete pagination, create/reuse deltas, metadata causality and failed-receipt preservation are fail-closed. The workflow contract and canonical runtime artifact paths are in `harness/agents/impl/skills/gitea-issue-rework/references/readback.md`. -Initial input-gate coverage is in `harness/tests/impl/test_rework_input_gate.py`. +Initial input-gate coverage is in the `test_rework_input_gate_*.py` modules +under `harness/tests/impl/rework/`. It exercises the real public CLI and canonical spec renderer, mixed-role/permutation and same-role supersession decisions, precise timestamp/tie handling, marker/state/ source binding, complete raw validation/drift, immutable capture and atomic output @@ -142,5 +143,220 @@ The regression tests use bounded subprocess timeouts and clean test-owned Run them with: ```sh -python3 -m unittest harness.tests.common.test_canonical_test_runner -v +python3 -B -m unittest harness.tests.common.runner.test_canonical_test_runner -v ``` + +## Domain ownership and verification levels + +The canonical output of this documentation is `harness/tests/README.md` in this +repository. Update this file in place rather than publishing a parallel test +inventory document. The canonical runner remains `harness/tests/run.py`; +there is no alternate discovery framework or hand-maintained source-hash list. + +- `common/runner`: source admission and exact discovery/exit-status contracts. +- `common/locking`: primitive flock and authenticated broker lifecycle remain + separate modules; neither substitutes for the other. +- `common/repository`: prefetch, runtime/verification roots, and Git source binding. +- `common/delegation`: evidence, adoption, and real-Git consumer integration. +- `common/evidence`: workflow evidence admission and conditional pytest recipe. +- `common/access`: local credential protocol recipe, never real credentials. +- `common/contracts`: cross-role documentation wiring, not live workflow proof. +- `impl/delivery`: coordinator, initial delivery, reconciliation, and response. +- `impl/rework`: intake, decision, rendering, provider readback, publication, + and separately authorized correction. +- `impl/evidence`: manifest/schema, filesystem/Git admission, and bounded + command/process-group execution (distinct verification boundaries). +- `impl/contracts`: documentation wiring and separately named comparator + execution extracted from the actual canonical documentation. +- `review/native`: native publication/plan, transport, lease/worktree admission, + and durable resume; GET/POST loopback fixtures are not live Gitea publication. +- `review/journal`: binding/presence safety, raw-byte legacy import, and actual + subprocess crash/locking/race tests. Archive import is not fresh readback. +- `review/evidence`: execution of the actual native evidence recipe. +- `review/contracts`: policy-specific classes consolidated in one documentation + module; string/link checks do not certify production behavior. +- `spec/support`: result codec, cache admission, publication/readback, and AC checker. +- `spec/projection`: source-Issue reference and requested-reviewer projection + contracts remain independent modules with their own strict input matrices. +- `spec/contracts`: documentation contracts plus the separately named + `test_actual_collector.py` executable collector regression. That regression + calls the real collector; it is not a local-model documentation example. + +### Documentation regression boundary + +General workflow guidance is not an exact-prose regression interface. Do not +pin advisory sentences or generic review/reporting guidance with substring +assertions. Keep executable behavior and explicit technical contracts: CLI +options, schema fields, canonical paths/links, recipe structure, and required +procedure ordering. In mixed tests, remove only the prose assertions rather +than deleting the executable or structural checks. + +The earlier `20261015T232331Z-final` traceability below is historical evidence +for that source snapshot, not an index of the current tree. Subsequent removal +runs under `logs/test-removal/` preserve their own baseline, exact dispositions, +source hashes and observed outcomes; never relabel old outcomes as current. +The general-guidance cleanup is recorded in +`logs/test-removal/general-guidance-20261016T054315Z/`. + +### Shared fixture boundaries + +`_support/` holds ordinary data/transport/resource builders only. It must not +contain or re-export TestCase classes, imported test bases, generated tests, or +`load_tests`. Root derivation is centralized in `_support/paths.py`; helpers do +not use `HARNESS_ROOT` or cwd to select canonical test sources. Resource builders +own explicit cleanup. Meaningful expected results, no-call/no-write/no-retry, +identity/source/stage and mode/inode/bytes assertions stay in the tests. Import +fixture helpers rather than importing another test module or manually invoking +another TestCase's `setUp()`/`tearDown()`. + +### Focused execution and evidence limits + +From the repository root, for example: + +```sh +HARNESS_ROOT=/opt/data/gitea-agent python3 -B -m unittest discover -s harness/tests/common -t . +HARNESS_ROOT=/opt/data/gitea-agent python3 -B -m unittest discover -s harness/tests/impl -t . +HARNESS_ROOT=/opt/data/gitea-agent python3 -B -m unittest discover -s harness/tests/review -t . +HARNESS_ROOT=/opt/data/gitea-agent python3 -B -m unittest discover -s harness/tests/spec -t . +``` + +These are focused runtime checks, not substitutes for the canonical admission +and full execution command above. Compilation is not test execution; +`--collect-only` is not runtime success. Source hashes and counts are emitted +from the actual current source tree, never manually adjusted to admit a move. + +- Documentation contracts read canonical documents (and required workflow + references through `_support/contracts.py`); they emit assertions, not a live + deployment/publication receipt. +- Unit tests call actual helpers with bounded synthetic inputs; they certify + their stated input/side-effect boundaries, not provider behavior in general. +- Git/HTTP integration tests create local temporary repositories or loopback + servers. They exercise real Git or transport code, not authenticated Gitea. +- Process tests exercise actual child processes/crash/locks and preserve durable + state assertions. In-process fakes or archive replay do not replace these. +- Evidence recipe tests execute the canonical recipe in test-owned temporary + roots. Their manifests/logs are local synthetic evidence, not product delivery. +- Two pytest recipe execution cases skip when this interpreter has no admitted + pytest installation (`admitted pytest interpreter required`). The documentation + case still runs. No dependency is installed, skip removed, or warning/failure + summary suppressed merely to obtain a green run. + +### Test-to-artifact scope + +Artifact scope is local and test-owned unless explicitly stated otherwise. +Temporary files, synthetic repositories, loopback traffic and child-process +journals are cleaned up by fixtures; they are not retained delivery evidence. +The retained canonical runtime log records each exact test ID and outcome. +The following groups specify what each module's tests may produce or inspect; +a passing group does not certify every stronger evidence layer listed elsewhere. + +- `common/access/test_git_credential_recipe.py`: local helper/askpass scripts, + permissions and protocol stdout; no stored or real provider credentials. +- `common/contracts/test_*.py`: canonical Markdown sections, workflow links and + documentation assertions; no publication receipt or workflow execution. +- `common/delegation/test_delegation_evidence*.py`: test-owned declarations, + source snapshots, bundle hashes and verifier decisions. Adoption tests add + fresh admission results; consumer unit tests add adapter-call records. + `test_delegation_consumers_integration.py` adds actual temporary Git evidence, + CLI stdout/stderr and fake consumer effects, not live Gitea writes. +- `common/evidence/test_workflow_guard.py`: local contract/test inventories, + snapshots, change decisions and fault inputs. `test_pytest_evidence_recipe.py` + inspects the canonical recipe and conditionally creates actual pytest + logs/summaries; skipped execution produces no recipe execution evidence. +- `common/locking/test_managed_lock.py`: local lock files/descriptors and + contention subprocess results. `test_managed_lock_broker.py`: private socket, + lease/token/lifecycle state and process results; not a remote Issue lock. +- `common/repository/test_repository_prefetch.py`: temporary local Git/cache + fixtures. Root tests produce local ownership/path/cleanup state; + `test_test_inventory_binding.py` produces source-bound inventories and binding + decisions, not semantic test-coverage proof. +- `common/runner/test_canonical_test_runner.py`: temporary source trees, + canonical collection JSON/diagnostics and subprocess exit codes, including + negative admission inputs. This is runner behavior, not product delivery. +- `impl/contracts/test_impl_architecture_wiring.py`: canonical document/link + assertions only. `test_impl_architecture_comparator.py`: comparator code + extracted from that document, synthetic inputs and comparator decisions. +- `impl/delivery/test_impl_operation.py`, `test_implementation_delivery.py` + and `test_implementation_operation_integration.py`: local operation plans, + evidence/adoption records and adapter call/effect ordering. + `test_delivery_response.py` and `test_delivery_reconciliation.py` add raised + HTTP errors or loopback response/reconciliation fixtures and completion + decisions; neither authorizes retrying an uncertain remote write. +- `impl/evidence/test_evidence_manifest_schema.py`: plan/manifest JSON, + command-admission diagnostics and bounded command outputs. + `test_evidence_manifest_filesystem_git.py`: real temporary Git identities, + artifact bytes/paths and manifest validation. + `test_evidence_command_deadlines.py`: actual child/process-group deadline + behavior and local command logs; separate from Git topology validation. +- `impl/rework/test_rework_input_gate_*.py`: schema inputs, CLI output files, + role/timestamp decisions and counter/publication-phase diagnostics using + canonical production entry points and synthetic provider data. +- `impl/rework/test_rework_readback_provider.py`: provider-shaped snapshots, + pagination and normalization decisions. Cache tests add strict-cache records; + bridge/transport tests add renderer/CLI or loopback HTTP artifacts. These are + not authenticated fresh Gitea readbacks. +- `impl/rework/test_rework_comment.py`, `test_rework_correction.py` and + `test_rework_publication.py`: local rendered comments, correction/publication + plans, synthetic readback/receipt and no-write/no-retry call records. +- `review/contracts/test_review_document_contracts.py`: actual policy document + assertions only; no native review publication or observability run receipt. +- `review/evidence/test_native_evidence_recipe.py`: actual recipe execution in + temporary roots, evidence plans/manifests/logs and archive-path rejection; + not a product delivery manifest or live provider verification. +- `review/journal/test_journal_safety.py` and + `test_journal_filesystem_safety.py`: private binding/presence records, + byte/mode/path integrity and corruption decisions. + `test_legacy_journal_contract.py`: source raw bytes, registration receipt, + imported legacy archive and zero-POST assertions; imported archives do not + establish fresh verification. `test_publication_processes.py`: actual child + crashes, lock contention and durable POST-counter/journal state. +- `review/native/test_native_publication.py`: synthetic plans/snapshots, + journal/receipt and fake API call counts. Lease tests add local worktree/lease + admission state; resume tests add durable pending-intent/readback failures; + transport tests add local HTTP request/response and adapter artifacts. +- `spec/contracts/test_priority_review_contracts.py`: actual canonical documents + and links, not execution of local-model examples. + `test_actual_collector.py`: actual collector callback arguments/results. +- `spec/projection/test_requested_reviewer.py` and + `test_source_issue_reference.py`: distinct snapshot/projection inputs, + production comparison results, CLI stdout/stderr and unchanged-file checks. +- `spec/support/test_codec.py`: rendered/parsed result bytes and CLI decisions; + `test_cache.py`: private cache/path/ownership admission; + `test_publication.py`: synthetic exact readback/publication comparisons; + `test_ac_checker.py`: actual classification CLI/input digest decisions and + canonical documentation assertions. None is live authenticated publication. + +Exact per-test IDs, baseline predecessors, source anchors, artifact scope, +verification layers and actual execution outcomes are retained in the run's +`traceability.json`; `inventory-summary.json` distinguishes documentation, +production behavior and overlapping process/integration evidence rather than +adding incompatible counts. For this cleanup the run directory is +`logs/regression-test-implementation/20261015T232331Z-final/`. +Rebuild that index after source/test-ID changes; fail closed on an unmapped ID, +missing outcome or source-hash mismatch. Primary classifications come from +source assertions/calls and fixture ownership. Detailed Git/HTTP/process/archive +labels are conservative source-confirmed subsets, not exhaustive dynamic +coverage counts; indirect helper calls may add work beyond those labels. Layer +counts overlap, and a declared execution layer is not executed evidence when +its recorded outcome is skipped. + +The final raw collection/runtime inputs are preserved in that run directory's +`acceptance-final/` subdirectory. After reviewing any classification changes, +rebuild the exact join with: + +```sh +python3 -B logs/regression-test-implementation/20261015T232331Z-final/refine_classification.py +python3 -B logs/regression-test-implementation/20261015T232331Z-final/build_traceability.py +``` + +These are run-specific evidence tools, not a replacement canonical runner. +Do not infer coverage from a filename, a successful collection or the mere +existence of an artifact. + +The cleanup's original-to-current exact ID dispositions and run-specific counts, +verification-level inventory, collection JSON, fault-injection controls and full +execution logs are preserved under +`logs/regression-test-implementation/20261015/`. The original 659-ID baseline +under `logs/regression-test-audit/20261015/` is immutable. Counts in run evidence +refer to actual methods (subTest branches are not additional tests); domain and +verification-level labels do not make one test a substitute for another. diff --git a/harness/tests/_support/__init__.py b/harness/tests/_support/__init__.py new file mode 100644 index 0000000..7a43330 --- /dev/null +++ b/harness/tests/_support/__init__.py @@ -0,0 +1 @@ +"""Static test helpers; never declare or re-export test cases here.""" diff --git a/harness/tests/common/impl_contract_docs.py b/harness/tests/_support/contracts.py similarity index 100% rename from harness/tests/common/impl_contract_docs.py rename to harness/tests/_support/contracts.py diff --git a/harness/tests/_support/data.py b/harness/tests/_support/data.py new file mode 100644 index 0000000..ec764af --- /dev/null +++ b/harness/tests/_support/data.py @@ -0,0 +1,7 @@ +"""Independent canonical JSON digests for synthetic evidence fixtures.""" +import hashlib +import json + + +def canonical_digest(value): + return hashlib.sha256(json.dumps(value, sort_keys=True, separators=(",", ":")).encode()).hexdigest() diff --git a/harness/tests/_support/delegation.py b/harness/tests/_support/delegation.py new file mode 100644 index 0000000..8ffed53 --- /dev/null +++ b/harness/tests/_support/delegation.py @@ -0,0 +1,65 @@ +"""Synthetic real-Git delegation evidence builder; no test cases.""" +import copy +import hashlib +import json +import subprocess +from harness.tests._support.data import canonical_digest + +class RealEvidence: + def __init__(self, root): + self.root = root + self.bundle = root / 'bundle' + (self.bundle / 'src').mkdir(parents=True) + (self.bundle / 'tests').mkdir() + (self.bundle / 'evidence').mkdir() + (self.bundle / 'src/a.py').write_text('VALUE = 1\n') + (self.bundle / 'tests/test_a.py').write_text('def test_value():\n assert 1 == 1\n') + (self.bundle / 'evidence/red.txt').write_text('red\n') + (self.bundle / 'evidence/green.txt').write_text('green\n') + subprocess.run(['/usr/bin/git', 'init', '-q', str(self.root / 'repo')], check=True) + repo = self.root / 'repo' + (repo / 'src').mkdir(); (repo / 'tests').mkdir() + (repo / 'src/a.py').write_text('VALUE = 1\n') + (repo / 'tests/test_a.py').write_text('def test_value():\n assert 1 == 1\n') + subprocess.run(['/usr/bin/git', '-C', str(repo), 'config', 'user.email', 'test@example.invalid'], check=True) + subprocess.run(['/usr/bin/git', '-C', str(repo), 'config', 'user.name', 'Integration'], check=True) + subprocess.run(['/usr/bin/git', '-C', str(repo), 'add', '.'], check=True) + subprocess.run(['/usr/bin/git', '-C', str(repo), 'commit', '-q', '-m', 'fixture'], check=True) + self.revision = subprocess.check_output(['/usr/bin/git', '-C', str(repo), 'rev-parse', 'HEAD'], text=True).strip() + self.git_dir = repo / '.git' + self.files = {p: hashlib.sha256((self.bundle / p).read_bytes()).hexdigest() for p in ('src/a.py', 'tests/test_a.py')} + self.bundle_sha = self.digest(self.files) + self.context = {'schema': 'gitea-agent.delegation-evidence-context/v1', + 'source': {'kind': 'git_tree', 'revision': self.revision}, + 'stage': 'spec-review', 'bundle_sha256': self.bundle_sha, + 'files': self.files, 'allowed_actions': ['static', 'execute'], + 'obligations': [{'id': 'static', 'kind': 'static'}, {'id': 'rg', 'kind': 'red_green'}], + 'executions': [self.execution('red', 1, 'assertion', 'evidence/red.txt'), + self.execution('green', 0, 'none', 'evidence/green.txt')]} + line_hash = hashlib.sha256(b'VALUE = 1\n').hexdigest() + self.result = {'schema': 'gitea-agent.delegation-evidence-result/v1', + 'source': copy.deepcopy(self.context['source']), 'stage': 'spec-review', + 'bundle_sha256': self.bundle_sha, + 'claims': [{'id': 'static', 'status': 'supported', 'anchors': [{'kind': 'lines', 'path': 'src/a.py', 'start_line': 1, 'end_line': 1, 'text_sha256': line_hash}], 'execution_ids': []}, + {'id': 'rg', 'status': 'supported', 'anchors': [{'kind': 'test', 'path': 'tests/test_a.py', 'symbol': 'test_value'}], 'execution_ids': ['red', 'green']}]} + self.context_path = root / 'context.json'; self.result_path = root / 'result.json' + self.write() + + digest = staticmethod(canonical_digest) + + def execution(self, eid, code, kind, path): + return {'id': eid, 'test_id': 'tests/test_a.py::test_value', 'bundle_sha256': self.bundle_sha, + 'test_file_sha256': self.files['tests/test_a.py'], 'argv': ['python', '-m', 'pytest'], + 'exit_code': code, 'failure_kind': kind, 'evidence_path': path, + 'evidence_sha256': hashlib.sha256((self.bundle / path).read_bytes()).hexdigest(), 'origin': 'direct'} + + def write(self): + self.context_path.write_text(json.dumps(self.context)) + self.result_path.write_text(json.dumps(self.result)) + + def declaration(self): + return {'mode': 'delegated', 'bundle_root': str(self.bundle), 'context_path': str(self.context_path), + 'result_path': str(self.result_path), 'git_dir': str(self.git_dir)} + + def set_stage(self, stage): + self.context['stage'] = stage; self.result['stage'] = stage; self.write() diff --git a/harness/tests/_support/evidence.py b/harness/tests/_support/evidence.py new file mode 100644 index 0000000..8c4d26e --- /dev/null +++ b/harness/tests/_support/evidence.py @@ -0,0 +1,72 @@ +"""Ordinary evidence fixture: owns synthetic Git checkout and artifacts.""" +import json +import os +import subprocess +import sys +import tempfile +from pathlib import Path +from harness.tests._support.paths import ROOT + +SCRIPT = ROOT / "harness/agents/impl/skills/gitea-issue-rework/scripts/evidence_manifest.py" + +def run_git(cwd, *args): + return subprocess.run(["git", *args], cwd=cwd, text=True, capture_output=True, check=True) + +class EvidenceManifestFixture: + def setup(self): + try: + self._setup() + except BaseException: + self.cleanup() + raise + return self + + def cleanup(self): + if hasattr(self, 'tmp'): + self.tmp.cleanup() + + def _setup(self): + self.tmp = tempfile.TemporaryDirectory(dir=os.environ.get('TMPDIR', '/opt/data/cache/scratch')) + self.root = Path(self.tmp.name) + self.contract = self.root / 'harness/operations/repository-cache.md' + self.contract.parent.mkdir(parents=True) + self.contract.write_text('# Repository cache 운영 지침\n') + self.repo = self.root / 'repositories/owner/project/worktrees/issue-1/checkout' + self.artifacts = self.repo.parent / 'verification/run-a' + self.repo.mkdir(parents=True) + self.artifacts.mkdir(parents=True) + for path in (self.root / 'repositories', *self.repo.parents): + if path == self.root: + break + path.chmod(448) + self.repo.chmod(448) + self.artifacts.parent.chmod(448) + self.artifacts.chmod(448) + run_git(self.repo, 'init', '-q') + run_git(self.repo, 'config', 'user.email', 'test@example.invalid') + run_git(self.repo, 'config', 'user.name', 'Test') + (self.repo / 'input.txt').write_text('hello\n') + run_git(self.repo, 'add', 'input.txt') + run_git(self.repo, 'commit', '-qm', 'initial') + + def invoke(self, *args, env=None): + op, *rest = args + common = [] if op == 'validate-legacy' else ['--contract-path', str(self.contract), '--artifact-root', str(self.artifacts)] + return subprocess.run([sys.executable, str(SCRIPT), str(op), *common, *map(str, rest)], cwd=self.repo, text=True, capture_output=True, env=env) + + def plan(self, commands=None, **extra): + if commands is None: + commands = [{'id': 'check', 'purpose': 'show input', 'argv': ['git', 'show', 'HEAD:input.txt'], 'expected_exit_code': 0, 'claims': ['output']}] + data = {'hash_paths': ['input.txt'], 'commands': [dict({'timeout_seconds': 5}, **command) for command in commands]} + data.update(extra) + p = self.artifacts / 'plan.json' + p.write_text(json.dumps(data)) + p.chmod(384) + return p + + def run_manifest(self, plan=None): + if plan is None: + plan = self.plan() + out = self.artifacts / 'manifest.json' + result = self.invoke('run', '--repo-root', self.repo, '--plan', plan.name, '--output', 'manifest.json') + return (result, out) diff --git a/harness/tests/_support/paths.py b/harness/tests/_support/paths.py new file mode 100644 index 0000000..c94f950 --- /dev/null +++ b/harness/tests/_support/paths.py @@ -0,0 +1,6 @@ +"""Canonical source-tree paths, independent of cwd and environment.""" +from pathlib import Path + +TESTS = Path(__file__).resolve().parents[1] +HARNESS = TESTS.parent +ROOT = HARNESS.parent diff --git a/harness/tests/_support/review.py b/harness/tests/_support/review.py new file mode 100644 index 0000000..28583b1 --- /dev/null +++ b/harness/tests/_support/review.py @@ -0,0 +1,112 @@ +"""Shared review fixtures; no unittest test cases or TestCase imports.""" +import copy +import importlib.util +from pathlib import Path + +from harness.tests._support.paths import ROOT + +SPEC_SCRIPT = ROOT / 'harness/agents/spec/skills/gitea-implementation-spec-review/scripts/review_support.py' + + +def _load(path, name): + spec = importlib.util.spec_from_file_location(name, path) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +support = _load(SPEC_SCRIPT, 'spec_support_fixture') + + +def plan_fixture(): + target = {'owner': 'acme', 'repository': 'repo', 'issue': 3, 'pull': 7, + 'head': 'a' * 40, 'base': 'c' * 40} + digest = __import__('hashlib').sha256(b'approved contract').hexdigest() + marker = ('') + return {'schema': 'gitea-agent.code-review-publication/v1', 'target': target, + 'contract_sha256': digest, 'reviewer': {'id': 42, 'login': 'gitea-review-agent'}, + 'spec_reviewer': {'id': 5, 'login': 'gitea-spec-agent'}, 'spec_review_id': 9, 'spec_paths': ['src/a.py'], + 'verdict': 'approve', 'body': marker + '\n- Verdict: `approve`\n검토 완료.\n', + 'provider_contract': {'supersede_same_author': True, 'retouch_dismissed': True, + 'unordered_commit_files': True, + 'evidence': ['saved observed Gitea projection and ordering']}} + + +def publication_fixture(): + p = plan_fixture(); t = p['target'] + spec_result = {'schema': support.SCHEMA, + 'target': {k: t[k] for k in ['owner', 'repository', 'issue', 'pull', 'head']}, + 'contract_sha256': p['contract_sha256'], 'verdict': 'approve', + 'criteria': [{'criterion': 'REQ-1', 'contract': '계약', + 'implementation_paths': ['src/a.py'], 'test_paths': [], + 'evidence': '직접 검증', 'result': 'satisfied', 'reason': '충족'}], + 'scope': {'missing': [], 'excess': [], 'spec_conflicts': []}, 'conclusion': '충족'} + url = 'https://gitea.jsfamily.quest/acme/repo/pulls/7' + old_time = '2026-10-04T00:00:00Z'; new_time = '2026-10-04T00:01:00Z' + def review(rid, user, body, head, state): + return {'id': rid, 'user': copy.deepcopy(user), 'body': body, 'commit_id': head, 'state': state, + 'official': True, 'dismissed': False, 'stale': head != t['head'], + 'submitted_at': old_time, 'updated_at': old_time, 'comments_count': 0, + 'pull_request_url': url} + sr = review(9, p['spec_reviewer'], support.render_review(spec_result, {'src/a.py'}, declaration={'mode': 'none'}), + t['head'], 'APPROVED') + previous = review(8, p['reviewer'], 'old review body', 'd' * 40, 'REQUEST_CHANGES') + s = {'identity': copy.deepcopy(p['reviewer']), 'spec_user': copy.deepcopy(p['spec_reviewer']), + 'repository': {'full_name': 'acme/repo', 'default_branch': 'main'}, + 'issue': {'number': 3, 'body': 'approved contract', 'state': 'open', + 'updated_at': old_time, 'repository': {'full_name': 'acme/repo'}}, + 'pr': {'number': 7, 'html_url': url, 'body': 'Closes #3', 'user': {'id': 99}, + 'state': 'open', 'merged': False, 'updated_at': old_time, + 'review_comments': 2, 'requested_reviewers': [], + 'base': {'sha': t['base'], 'ref': 'main', 'repo': {'full_name': 'acme/repo'}}, + 'head': {'sha': t['head'], 'ref': 'feature', 'repo': {'full_name': 'acme/repo'}}}, + 'files': [{'filename': 'src/a.py', 'status': 'modified'}], + 'commits': [{'sha': t['head'], 'files': [{'filename': 'src/a.py', 'status': 'modified'}, + {'filename': 'docs/x.md', 'status': 'added'}]}], + 'reviews': [previous, sr], 'review_comments': {'8': [], '9': []}, + 'issue_comments': [], 'pr_comments': [], 'statuses': [], 'pagination': {}} + exact = review(10, p['reviewer'], p['body'], t['head'], 'APPROVED') + exact['submitted_at'] = exact['updated_at'] = new_time + after = copy.deepcopy(s); after['reviews'].append(exact); after['review_comments']['10'] = [] + after['pr']['updated_at'] = new_time; after['pr']['review_comments'] += 1 + for snap in [s, after]: + for name in ['files', 'commits', 'reviews', 'issue_comments', 'pr_comments', 'statuses']: + snap['pagination'][name] = [{'page': 1, 'count': len(snap[name]), + 'declared_total': len(snap[name]), 'link': None}] + for rid, comments in snap['review_comments'].items(): + snap['pagination']['review_comments_' + rid] = [{'page': 1, 'count': len(comments), + 'declared_total': len(comments), 'link': None}] + return p, s, after, copy.deepcopy(exact) + + +class FakeAPI: + def __init__(self, plan, before, after, exact): + self.plan = plan; self.before = before; self.after = after; self.exact = exact + self.created = False; self.posts = 0; self.timeout = False; self.apply = True + self.events = [] + + def identity(self): + self.events.append('identity') + return copy.deepcopy(self.plan['reviewer']) + + def snapshot(self, target): + self.events.append('snapshot') + return copy.deepcopy(self.after if self.created else self.before) + + def create_review(self, target, payload): + self.events.append('create'); self.posts += 1 + if self.apply: self.created = True + if self.timeout: raise TimeoutError('secret transport detail must not escape') + return copy.deepcopy(self.exact) + + def read_review(self, target, rid): + self.events.append('exact') + return copy.deepcopy(self.exact) + + +class FakeLease: + def __init__(self): self.valid = True; self.checks = 0 + def check(self, plan): + self.checks += 1 + if not self.valid: raise support.ValidationError('expired lease') diff --git a/harness/tests/_support/rework.py b/harness/tests/_support/rework.py new file mode 100644 index 0000000..0e253f5 --- /dev/null +++ b/harness/tests/_support/rework.py @@ -0,0 +1,130 @@ +"""Provider-shaped rework readback fixture with explicit resource ownership.""" +import copy +import hashlib +import importlib.util +import json +import tempfile +from argparse import Namespace +from pathlib import Path +from urllib.parse import parse_qs, urlsplit + +from harness.tests._support.paths import ROOT + +DIRECTORY = ROOT / "harness/agents/impl/skills/gitea-issue-rework/scripts" +ORIGIN = "https://gitea.jsfamily.quest" +T0 = "2026-01-01T00:00:00Z" +T1 = "2026-01-02T00:00:00Z" + + +class ReworkProviderFixture: + """Own synthetic provider state and its temporary resources.""" + + def __init__(self): + self._temp = None + self._cleaned = False + + def setup(self): + if self._temp is not None: + raise RuntimeError("fixture already set up") + self._temp = tempfile.TemporaryDirectory() + root = Path(self._temp.name) + self.legacy = self.load("rework_comment") + data = { + "owner": "acme", "repository": "demo", "issue": 7, "pr": 9, "review": 301, + "reviewed_head": "a" * 40, "final_head": "b" * 40, "contract_sha256": "c" * 64, + "result": "partial", "findings": [{"id": "F-1", "status": "resolved", "classification": "valid_fix", + "changes": "bounded fix", "verification": "local checks pass", "reason": "regression"}], + "local_checks": "pass", "runtime_browser": "not applicable", + "commit_status": "success", "actions_runs": "없음", + } + (root / "input.json").write_text(json.dumps(data)) + self.legacy.do_render(Namespace(input=str(root / "input.json"), no_delegated_evidence=True, + delegation_adoption=None, comment=str(root / "comment.md"), expected=str(root / "expected.json"))) + self.expected = json.loads((root / "expected.json").read_text()) + self.contract = {"schema": "gitea-agent.rework-provider/v1", "version": "1.24.6", + "aggregate": {"pr": {"field": "comments", "components": ["top_level", "reviews"]}, + "pr_issue": {"field": "comments", "components": ["top_level", "reviews"]}}, + "inline": {"mode": "per_review", "field": "review_comments", "components": ["reviews", "inline"]}} + self.contract["aggregate"]["pr"]["components"] = ["top_level", "reviews", "inline"] + self.contract["aggregate"]["pr_issue"]["components"] = ["top_level", "reviews", "inline"] + self.repo = {"id": 3, "full_name": "acme/demo", "html_url": ORIGIN + "/acme/demo"} + self.source = {"id": 707, "number": 7, "html_url": ORIGIN + "/acme/demo/issues/7", + "body": "source contract", "title": "Source", "state": "open", "comments": 0, "updated_at": T0} + self.pr = {"id": 99, "number": 9, "html_url": ORIGIN + "/acme/demo/pulls/9", + "url": ORIGIN + "/acme/demo/pulls/9", "state": "open", + "head": {"sha": "b" * 40, "repo": {"id": 12, "full_name": "contributor/demo"}}, + "base": {"repo": copy.deepcopy(self.repo)}, "title": "PR", "body": "Closes #7", + "comments": 1, "review_comments": 1, "updated_at": T0} + self.underlying = {"id": 99, "number": 9, "html_url": self.pr["html_url"], + "pull_request": {"html_url": self.pr["html_url"], "merged": False}, + "title": "PR", "body": "Closes #7", "state": "open", "comments": 1, "updated_at": T0} + self.reviews = [{"id": 301, "commit_id": "a" * 40, "comments_count": 0, "state": "REQUEST_CHANGES", + "user": {"id": 33, "login": "gitea-review-agent"}}] + self.comments = [] + self.source_comments = [] + self.inline = [] + self.overrides = {} + self.calls = [] + self._request_override = None + for record in (self.source, self.pr, self.underlying): + record.update(created_at=T0, labels=[], assignees=[], milestone=None, + user={"id": 22, "login": "genichin"} if record is self.source + else {"id": 42, "login": "gitea-impl-agent"}) + self.pr["base"].update(ref="main", label="main", sha="d" * 40, repo_id=3) + self.pr["head"].update(ref="fix/issue-7", label="contributor:fix/issue-7", repo_id=12) + self.reviews[0].update(body="Fix F-1", submitted_at=T0, updated_at=T0, + stale=True, dismissed=False, html_url=self.pr["html_url"] + "#pullreview-301", + pull_request_url=self.pr["html_url"]) + return self + + def cleanup(self): + if self._temp is not None and not self._cleaned: + self._temp.cleanup() + self._cleaned = True + + def load(self, name): + path = DIRECTORY / (name + ".py") + if not path.is_file(): + raise AssertionError("canonical readback core missing") + spec = importlib.util.spec_from_file_location(name, path) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + def request(self, path): + self.calls.append(path) + if path in self.overrides: + return copy.deepcopy(self.overrides[path]) + parsed = urlsplit(path) + objects = {"/user": {"id": 42, "login": "gitea-impl-agent"}, "/version": {"version": "1.24.6"}, + "/repos/acme/demo": self.repo, "/repos/acme/demo/issues/7": self.source, + "/repos/acme/demo/pulls/9": self.pr, "/repos/acme/demo/issues/9": self.underlying} + if parsed.path in objects: + return copy.deepcopy(objects[parsed.path]), {}, 200 + collections = {"/repos/acme/demo/issues/7/comments": self.source_comments, + "/repos/acme/demo/issues/9/comments": self.comments, + "/repos/acme/demo/pulls/9/reviews": self.reviews, + "/repos/acme/demo/pulls/9/reviews/301/comments": self.inline} + if parsed.path not in collections: + raise AssertionError("unexpected GET endpoint") + rows = collections[parsed.path] + page = int(parse_qs(parsed.query)["page"][0]) + return copy.deepcopy(rows[(page - 1) * 50:page * 50]), {"X-Total-Count": str(len(rows))}, 200 + + def adapter(self): + module = self.load("rework_readback") + try: + return module.ReworkReadbackAdapter(self._request_override or self.request, origin=ORIGIN + "/api/v1", + credential_source="$HERMES_HOME/.env:GITEA_TOKEN", actor_id=42, + provider_contract=self.contract) + except Exception as error: + raise AssertionError("canonical CLI transport contract rejected: " + str(error)) from error + + def add_result(self): + self.comments.append({"id": 501, "html_url": self.pr["html_url"] + "#issuecomment-501", + "body": self.expected["body"], "created_at": T1, "updated_at": T1, + "user": {"id": 42, "login": "gitea-impl-agent"}}) + self.pr["comments"] += 1 + self.underlying["comments"] += 1 + self.pr["updated_at"] = self.underlying["updated_at"] = T1 diff --git a/harness/tests/_support/spec.py b/harness/tests/_support/spec.py new file mode 100644 index 0000000..46cf8a8 --- /dev/null +++ b/harness/tests/_support/spec.py @@ -0,0 +1,55 @@ +"""Shared implementation-spec review fixtures; no test cases are declared here.""" +import copy +import importlib.util + +from .paths import ROOT + +MODPATH = ROOT / "harness/agents/spec/skills/gitea-implementation-spec-review/scripts/review_support.py" +spec = importlib.util.spec_from_file_location("review_support", MODPATH) +mod = importlib.util.module_from_spec(spec) +spec.loader.exec_module(mod) + + +def result(verdict="approve"): + return { + "schema": "gitea-agent.implementation-spec-review-result/v1", + "target": {"owner": "acme", "repository": "repo", "issue": 3, "pull": 7, + "head": "a" * 40}, + "contract_sha256": "b" * 64, + "verdict": verdict, + "criteria": [{"criterion": "REQ-1", "contract": "계약", "implementation_paths": ["src/a.py"], + "test_paths": ["tests/test_a.py"], "evidence": "증거", "result": "satisfied", + "reason": "충족"}], + "scope": {"missing": [], "excess": [], "spec_conflicts": []}, + "conclusion": "모든 기준을 충족했다.", + } + + +def publication_fixture(): + r = result(); paths = {"src/a.py", "tests/test_a.py"} + body = mod.render_review(r, paths, declaration={"mode": "none"}) + url = "https://example.test/acme/repo/pulls/7" + user = {"id": 42, "login": "spec-agent"} + review = {"id": 11, "body": body, "commit_id": r["target"]["head"], "state": "APPROVED", + "user": user, "official": True, "stale": False, "dismissed": False, + "comments_count": 0, "submitted_at": "2026-10-03T01:24:00Z", + "updated_at": "2026-10-03T01:24:00Z", "pull_request_url": url} + before = {"issue": {"id": 3, "number": 3, "repository": {"full_name": "acme/repo"}, + "updated_at": "2026-10-03T00:00:00Z", "title": "contract"}, + "issue_comments": [], "pr": {"id": 7, "number": 7, "html_url": url, + "url": url, "base": {"repo": {"full_name": "acme/repo"}, "sha": "c" * 40}, + "head": {"sha": "a" * 40}, "user": {"id": 99}, "state": "open", "merged": False, + "review_comments": 0, "updated_at": "2026-10-03T00:00:00Z", "requested_reviewers": []}, + "pr_comments": [], "reviews": [], "timeline": [], + "files": [{"filename": "src/a.py", "status": "modified", "additions": 1}]} + after = copy.deepcopy(before); after["reviews"] = [review] + after["pr"]["updated_at"] = review["submitted_at"]; after["pr"]["review_comments"] = 1 + after["timeline"] = [{"id": 101, "review_id": 11, "user": user, "body": body, + "type": "review", "created_at": review["submitted_at"], + "updated_at": review["submitted_at"], "pull_request_url": url}] + return before, after, review, r, paths + + +def verify_fixture(before, after, review, r, paths, contract=None): + return mod.verify_publication(before, after, review, r, paths, 42, [], contract, + expected_pr_url="https://example.test/acme/repo/pulls/7") diff --git a/harness/tests/common/access/__init__.py b/harness/tests/common/access/__init__.py new file mode 100644 index 0000000..182440c --- /dev/null +++ b/harness/tests/common/access/__init__.py @@ -0,0 +1 @@ +"""access regression tests.""" diff --git a/harness/tests/common/test_git_credential_recipe.py b/harness/tests/common/access/test_git_credential_recipe.py similarity index 81% rename from harness/tests/common/test_git_credential_recipe.py rename to harness/tests/common/access/test_git_credential_recipe.py index bd6a1b3..1b81d2d 100644 --- a/harness/tests/common/test_git_credential_recipe.py +++ b/harness/tests/common/access/test_git_credential_recipe.py @@ -1,4 +1,5 @@ """Local credential protocol checks only; no real credentials or network.""" +from harness.tests._support.paths import ROOT as SOURCE_ROOT import os import secrets import shlex @@ -7,9 +8,9 @@ import subprocess import tempfile import unittest from pathlib import Path -from harness.tests.common.impl_contract_docs import read_contract +from harness.tests._support.contracts import read_contract -ROOT = Path(__file__).resolve().parents[3] +ROOT = SOURCE_ROOT POLICY = ROOT / "harness/operations/repository-cache.md" REWORK = ROOT / "harness/agents/impl/skills/gitea-issue-rework/SKILL.md" HEADING = "### Non-file Git credential helper" @@ -25,23 +26,21 @@ class GitCredentialRecipeTests(unittest.TestCase): def test_canonical_nonfile_recipe_preserves_permission_and_delivery_contract(self): section, command = self.recipe() - self.assertEqual(command[0], "git") - self.assertEqual(command[1:3], ["-c", "credential.helper="]) + self.assertEqual(command[0], 'git') + self.assertEqual(command[1:3], ['-c', 'credential.helper=']) helper = command[4] - self.assertTrue(helper.startswith("credential.helper=!")) - self.assertIn("GIT_ROLE_USERNAME", helper) - self.assertIn("GIT_ROLE_TOKEN", helper) - self.assertIn("credential.interactive=false", command) - self.assertIn("remote.origin.mirror=false", command) - self.assertEqual(command[-4:], ["push", "--dry-run", "$VERIFIED_REPOSITORY_URL", "$VERIFIED_REFSPEC"]) - for required in ("0600", "GIT_TERMINAL_PROMPT=0", "GIT_ASKPASS=/bin/false", "SSH_ASKPASS=/bin/false", - "same-UID", "store", "erase", "stdout", "readback"): + self.assertTrue(helper.startswith('credential.helper=!')) + self.assertIn('GIT_ROLE_USERNAME', helper) + self.assertIn('GIT_ROLE_TOKEN', helper) + self.assertIn('credential.interactive=false', command) + self.assertIn('remote.origin.mirror=false', command) + self.assertEqual(command[-4:], ['push', '--dry-run', '$VERIFIED_REPOSITORY_URL', '$VERIFIED_REFSPEC']) + for required in ('0600', 'GIT_TERMINAL_PROMPT=0', 'GIT_ASKPASS=/bin/false', 'SSH_ASKPASS=/bin/false', 'store', 'erase'): self.assertIn(required, section) - policy = POLICY.read_text(encoding="utf-8") - self.assertIn("Lock 파일과 임시 credential helper는 실행 role 소유와 mode `0600`을 요구한다", policy) - delivery = read_contract(REWORK).split("### 7. 기존 PR branch 전달", 1)[1].split("###", 1)[0] - self.assertIn("#non-file-git-credential-helper", delivery) - self.assertNotIn("필요한 temporary askpass helper", delivery) + policy = POLICY.read_text(encoding='utf-8') + self.assertIn('mode `0600`', policy) + delivery = read_contract(REWORK).split('### 7. 기존 PR branch 전달', 1)[1].split('###', 1)[0] + self.assertIn('#non-file-git-credential-helper', delivery) def test_nonfile_get_missing_variables_and_store_erase_do_not_persist_credentials(self): _, documented = self.recipe() diff --git a/harness/tests/common/contracts/__init__.py b/harness/tests/common/contracts/__init__.py new file mode 100644 index 0000000..0107a26 --- /dev/null +++ b/harness/tests/common/contracts/__init__.py @@ -0,0 +1 @@ +"""contracts regression tests.""" diff --git a/harness/tests/common/test_delegation_adoption_contract_docs.py b/harness/tests/common/contracts/test_delegation_adoption_contract_docs.py similarity index 78% rename from harness/tests/common/test_delegation_adoption_contract_docs.py rename to harness/tests/common/contracts/test_delegation_adoption_contract_docs.py index 81d195b..074d6e3 100644 --- a/harness/tests/common/test_delegation_adoption_contract_docs.py +++ b/harness/tests/common/contracts/test_delegation_adoption_contract_docs.py @@ -1,17 +1,15 @@ """Executable consumer declaration contract wiring.""" +from harness.tests._support.paths import ROOT as SOURCE_ROOT from pathlib import Path -from harness.tests.common.impl_contract_docs import read_contract +from harness.tests._support.contracts import read_contract import unittest -ROOT = Path(__file__).resolve().parents[3] +ROOT = SOURCE_ROOT class DelegationAdoptionContractTests(unittest.TestCase): def test_common_runtime_contract_is_explicit(self): text = (ROOT / 'harness/operations/repository-cache.md').read_text() - for token in ('require_adoption', 'expected_source', 'expected_stage', - '--no-delegated-evidence', '--delegation-adoption', - 'spec-review', 'code-review', 'implementation-delivery', - 'PASS receipt', 'sandbox'): + for token in ('require_adoption', 'expected_source', 'expected_stage', '--no-delegated-evidence', '--delegation-adoption', 'spec-review', 'code-review', 'implementation-delivery'): with self.subTest(token=token): self.assertIn(token, text) diff --git a/harness/tests/common/test_delegation_evidence_contract_docs.py b/harness/tests/common/contracts/test_delegation_evidence_contract_docs.py similarity index 55% rename from harness/tests/common/test_delegation_evidence_contract_docs.py rename to harness/tests/common/contracts/test_delegation_evidence_contract_docs.py index f557d28..a50aea1 100644 --- a/harness/tests/common/test_delegation_evidence_contract_docs.py +++ b/harness/tests/common/contracts/test_delegation_evidence_contract_docs.py @@ -1,9 +1,10 @@ """Procedural consumer wiring is not proof of live agent execution.""" +from harness.tests._support.paths import HARNESS as SOURCE_HARNESS from pathlib import Path -from harness.tests.common.impl_contract_docs import read_contract +from harness.tests._support.contracts import read_contract import unittest -ROOT = Path(__file__).resolve().parents[2] +ROOT = SOURCE_HARNESS ROLES = [ 'agents/impl/skills/gitea-issue-implementation/SKILL.md', 'agents/impl/skills/gitea-issue-rework/SKILL.md', @@ -17,10 +18,7 @@ class DelegationEvidenceContractTests(unittest.TestCase): for path in ROLES: with self.subTest(path=path): text = read_contract(ROOT / path) - for required in ['## Delegation evidence acceptance gate', - 'delegation_evidence.py', '위임 결과 채택 이전', - 'main이 독립적으로 고정한 context', - 'worker 자기 선언으로 context를 만들지 않는다']: + for required in ['## Delegation evidence acceptance gate', 'delegation_evidence.py', '위임 결과 채택 이전']: self.assertIn(required, text) def test_canonical_contract_names_cli_and_schema(self): @@ -33,18 +31,9 @@ class DelegationEvidenceContractTests(unittest.TestCase): def test_source_modes_and_coverage_limits_are_explicit(self): text = (ROOT / 'operations/repository-cache.md').read_text() - for required in ['captured_worktree', 'base_revision', 'git_tree', - 'bundle에 없다는 이유로 repository 부재로 판정하지 않는다', - 'named symbol 존재는 assertion 검출력을 인증하지 않는다', - '제품의 추가 evidence commit이나 전체 테스트 재실행 의무를 만들지 않는다']: + for required in ['captured_worktree', 'base_revision', 'git_tree']: self.assertIn(required, text) - def test_trusted_execution_catalog_and_failure_origin_limits(self): - text = (ROOT / 'operations/repository-cache.md').read_text() - for required in ['독립 실행 catalog', 'import·환경 오류는 RED로 인정하지 않는다', - '허용 작업', '실제 command/output/exit와 실패 원인', - 'gate는 context의 신뢰성을 스스로 인증하지 않는다']: - self.assertIn(required, text) if __name__ == '__main__': diff --git a/harness/tests/common/test_inventory_binding_contract_docs.py b/harness/tests/common/contracts/test_inventory_binding_contract_docs.py similarity index 53% rename from harness/tests/common/test_inventory_binding_contract_docs.py rename to harness/tests/common/contracts/test_inventory_binding_contract_docs.py index 83689a8..1156a7e 100644 --- a/harness/tests/common/test_inventory_binding_contract_docs.py +++ b/harness/tests/common/contracts/test_inventory_binding_contract_docs.py @@ -1,25 +1,20 @@ """Contract wiring for the read-only inventory source-binding preflight.""" +from harness.tests._support.paths import HARNESS as SOURCE_HARNESS import unittest from pathlib import Path -from harness.tests.common.impl_contract_docs import read_contract +from harness.tests._support.contracts import read_contract -ROOT = Path(__file__).resolve().parents[2] +ROOT = SOURCE_HARNESS class InventoryBindingDocumentationTests(unittest.TestCase): def test_all_consumer_roles_require_binding_before_check_tests(self): - paths = [ - 'agents/impl/skills/gitea-issue-implementation/SKILL.md', - 'agents/impl/skills/gitea-issue-rework/SKILL.md', - 'agents/spec/skills/gitea-implementation-spec-review/SKILL.md', - 'agents/review/skills/gitea-code-review/SKILL.md', - ] + paths = ['agents/impl/skills/gitea-issue-implementation/SKILL.md', 'agents/impl/skills/gitea-issue-rework/SKILL.md', 'agents/spec/skills/gitea-implementation-spec-review/SKILL.md', 'agents/review/skills/gitea-code-review/SKILL.md'] for path in paths: with self.subTest(path=path): text = read_contract(ROOT / path) self.assertIn('test_inventory_binding.py', text) self.assertIn('check-tests 이전', text) - self.assertIn('독립적으로 고정한 revision과 stage', text) def test_canonical_reference_defines_exact_cli_and_scope_digest(self): text = (ROOT / 'operations/repository-cache.md').read_text() @@ -30,11 +25,6 @@ class InventoryBindingDocumentationTests(unittest.TestCase): '/usr/bin/git', '/proc/self/fd/']: self.assertIn(value, text) - def test_binding_limits_do_not_create_product_or_execution_requirements(self): - text = (ROOT / 'operations/repository-cache.md').read_text() - self.assertIn('named symbol·parameter 존재나 실제 collection 실행을 인증하지 않는다', text) - self.assertIn('현재 collection을 baseline으로 재명명하지 않는다', text) - self.assertIn('별도 checkout이나 전체 테스트 재실행을 요구하지 않는다', text) if __name__ == '__main__': diff --git a/harness/tests/common/contracts/test_verification_contract_docs.py b/harness/tests/common/contracts/test_verification_contract_docs.py new file mode 100644 index 0000000..867da09 --- /dev/null +++ b/harness/tests/common/contracts/test_verification_contract_docs.py @@ -0,0 +1,182 @@ +from harness.tests._support.paths import ROOT as SOURCE_ROOT +import re +import subprocess +import sys +import unittest +from pathlib import Path +from harness.tests._support.contracts import read_contract + +ROOT = SOURCE_ROOT +SCRIPT = ROOT / "harness/agents/impl/skills/gitea-issue-rework/scripts/evidence_manifest.py" +PREPARE_SCRIPT = ROOT / "harness/lib/verification_root.py" +CACHE = ROOT / "harness/operations/repository-cache.md" +SKILLS = { + "implementation": ROOT / "harness/agents/impl/skills/gitea-issue-implementation/SKILL.md", + "rework": ROOT / "harness/agents/impl/skills/gitea-issue-rework/SKILL.md", + "review": ROOT / "harness/agents/spec/skills/gitea-implementation-spec-review/SKILL.md", +} +CANONICAL = "/repositories///worktrees/issue-/verification/" + + +class VerificationContractDocumentationTests(unittest.TestCase): + def test_runtime_policy_is_broker_owned_and_separate_from_verification(self): + text = CACHE.read_text(encoding='utf-8') + self.assertIn('runtime/', text) + self.assertIn('broker.handle', text) + self.assertIn('broker.sock', text) + self.assertIn('ready JSON', text) + self.assertIn('active resource', text) + self.assertIn('unknown', text) + for path in SKILLS.values(): + with self.subTest(path=path.name): + role = read_contract(path) + gate = role.split('## Managed-lock broker gate', 1)[1].split('\n## ', 1)[0] + self.assertIn('broker-generated', gate) + self.assertIn('ready JSON', gate) + self.assertNotIn('pre-created', gate) + self.assertNotIn('pre-create', gate) + + def test_role_runtime_startup_precedes_and_does_not_replace_admission(self): + roles = [path for path in (ROOT / 'harness/agents').rglob('SKILL.md') if 'broker-generated runtime' in read_contract(path)] + self.assertEqual(len(roles), 6) + for path in roles: + with self.subTest(role=path.parent.name): + gate = read_contract(path).split('## Managed-lock broker gate', 1)[1].split('\n## ', 1)[0] + self.assertIn('ready 이후 각 workflow의 진입 검증·승인 조건을 확인한다', gate) + self.assertIn('ready JSON의 `handle`/`socket`', gate) + + def test_evidence_plan_disables_python_bytecode_in_clean_checkout(self): + import json + template = ROOT / "harness/agents/impl/skills/gitea-issue-rework/templates/evidence-plan.json" + plan = json.loads(template.read_text()) + self.assertIn("-B", plan["commands"][0]["argv"]) + self.assertIn("PYTHONDONTWRITEBYTECODE", plan["command_env_allowlist"]) + + + def test_command_deadline_and_archive_policy_is_explicit(self): + import json + template = ROOT / 'harness/agents/impl/skills/gitea-issue-rework/templates/evidence-plan.json' + for command in json.loads(template.read_text())['commands']: + self.assertIsInstance(command['timeout_seconds'], (int, float)) + self.assertGreater(command['timeout_seconds'], 0) + self.assertLessEqual(command['timeout_seconds'], 3600) + for path in (CACHE, *SKILLS.values()): + text = read_contract(path) + for token in ('evidence-manifest/v3', 'timeout_seconds', 'process group', 'validate-archived-v2'): + with self.subTest(path=str(path), token=token): + self.assertIn(token, text) + help_result = subprocess.run([sys.executable, str(SCRIPT), 'validate-archived-v2', '--help'], capture_output=True, text=True, timeout=10) + self.assertEqual(help_result.returncode, 0, help_result.stderr) + for option in ('--contract-path', '--repo-root', '--artifact-root', '--manifest'): + self.assertIn(option, help_result.stdout) + + def test_all_roles_reference_canonical_external_artifacts(self): + for path in (CACHE, *SKILLS.values()): + with self.subTest(path=path.name): + text = read_contract(path) + self.assertIn(CANONICAL, text) + self.assertIn("evidence_gap", text) + self.assertIn("[REDACTED]", text) + self.assertIn("evidence-manifest/v3", text) + + def test_examples_match_real_helper_cli_not_unsupported_options(self): + for path in (SKILLS['implementation'], SKILLS['rework']): + section = read_contract(path).split('## Canonical external verification evidence', 1)[1] + match = re.search('```sh\\n(.*?)```', section, re.S) + assert match is not None + block = match.group(1).replace('\\\n', ' ') + commands = [line for line in block.splitlines() if line.startswith('python3 ')] + self.assertEqual(len(commands), 3) + for op, command in zip(('prepare', 'run', 'validate'), commands): + with self.subTest(skill=path.parent.name, op=op): + declared = set(re.findall('--[a-z-]+', command)) + if op == 'prepare': + self.assertIn('harness/lib/verification_root.py', command) + help_result = subprocess.run([sys.executable, str(PREPARE_SCRIPT), op, '--help'], text=True, capture_output=True) + self.assertEqual(help_result.returncode, 0, help_result.stderr) + supported = set(re.findall('--[a-z-]+', help_result.stdout)) + required = {'--root', '--owner', '--repository', '--issue', '--run-id', '--handle'} + self.assertEqual(declared, required) + self.assertTrue(declared <= supported) + else: + help_result = subprocess.run([sys.executable, str(SCRIPT), op, '--help'], text=True, capture_output=True) + self.assertEqual(help_result.returncode, 0) + supported = set(re.findall('--[a-z-]+', help_result.stdout)) + required = {'--contract-path', '--repo-root', '--artifact-root'} | ({'--plan', '--output'} if op == 'run' else {'--manifest'}) + self.assertEqual(declared, required) + self.assertTrue(declared <= supported) + self.assertIn('exact current HEAD', section) + self.assertIn('validate-legacy', section) + + def test_prepare_contract_binds_root_and_forbids_recursive_parent_creation(self): + for path in (CACHE, SKILLS['implementation'], SKILLS['rework']): + with self.subTest(path=path.name): + text = read_contract(path) + self.assertIn('exact harness root', text) + self.assertIn('Path.mkdir(parents=True, mode=0o700)', text) + + def test_checkout_cleanup_preserves_verification_without_recursive_delete(self): + text = CACHE.read_text() + cleanup = text.split('## Worktree 실패 처리와 제거', 1)[1].split('## Credential 안전', 1)[0] + self.assertIn('`verification/`이 남으면 Issue directory를 보존한다', cleanup) + self.assertIn('mode `0700`', text) + self.assertIn('mode `0600`', text) + + def test_spec_review_uses_immutable_source_and_not_moving_checkout(self): + section = SKILLS['review'].read_text().split('## External Issue verification evidence boundary', 1)[1] + self.assertIn('moving implementation checkout', section) + self.assertIn('authoritative blob evidence', section) + self.assertIn('trusted `harness/.runtime`', section) + + def test_workflow_guard_records_reuse_existing_verification_location(self): + text = CACHE.read_text() + guard = text.split('### Shared workflow guard gates', 1)[1].split('Issue별 검증 실행 기록의 정본', 1)[0] + self.assertIn('verification/', guard) + + + def test_workflow_guard_snapshot_protects_scope_before_delegation(self): + implementation = read_contract(SKILLS['implementation']) + self.assertIn('workflow_guard.py snapshot', implementation) + self.assertIn('compare-source --repo-root', implementation) + + def test_workflow_guard_inventory_is_checked_before_adopting_results(self): + rework = read_contract(SKILLS['rework']) + self.assertIn('workflow_guard.py check-tests', rework) + self.assertIn('test inventory', rework) + + def test_workflow_guard_contract_preservation_precedes_spec_verdict(self): + spec = SKILLS['review'].read_text() + self.assertIn('workflow_guard.py check-contracts', spec) + self.assertIn('baseline inventory', spec) + + def test_workflow_guard_nested_schema_is_actionable_and_claims_are_not_authenticated(self): + text = CACHE.read_text() + section = text.split('### Shared workflow guard gates', 1)[1].split('Issue별 검증 실행 기록의 정본', 1)[0] + for token in ('#### Nested input contracts', 'baseline_collection', 'inventory_sha256', 'baseline_status', 'expected_failure', 'replacement_ids', 'kind', 'delete', 'rename', 'status', 'review_evidence', 'scope_sha256', 'expected_exit_code', 'failure_kind', 'semantic_key', 'current_keys', 'observation', 'intentionally_changed', 'sort_keys=True', 'separators', 'UTF-8'): + with self.subTest(token=token): + self.assertIn(token, section) + import json + blocks = re.findall('```json\\n(.*?)```', section, re.S) + self.assertEqual(len(blocks), 2, 'provide runnable test and contract input examples') + self.assertEqual(set(json.loads(blocks[0])), {'baseline_ids', 'current_ids', 'baseline_status', 'current_status', 'baseline_collection', 'current_collection', 'authorized_changes', 'changed_tests', 'findings'}) + self.assertEqual(set(json.loads(blocks[1])), {'baseline', 'current', 'mappings', 'stages', 'precedence'}) + + def test_mapping_schema_types_use_raw_source_not_redacted_tool_rendering(self): + # Authorization-looking prose can be masked by read-tool transport. + # Inspect the approved public schema in raw source; never disable redaction. + declaration = 'authorization: string, verification: string' + masked = 'authorization: ' + '*' * 3 + ' verification' + helper = ROOT / 'harness/lib/workflow_guard.py' + for path in (helper, CACHE): + text = read_contract(path) + with self.subTest(path=path.name): + self.assertEqual(text.count(declaration), 1) + self.assertNotIn(masked, text) + + def test_rework_no_longer_excludes_repo_evidence_from_clean_tree(self): + text = read_contract(SKILLS['rework']) + self.assertIn('archived v1 read-only', text) + + +if __name__ == "__main__": + unittest.main() diff --git a/harness/tests/common/delegation/__init__.py b/harness/tests/common/delegation/__init__.py new file mode 100644 index 0000000..567611a --- /dev/null +++ b/harness/tests/common/delegation/__init__.py @@ -0,0 +1 @@ +"""delegation regression tests.""" diff --git a/harness/tests/common/test_delegation_adoption.py b/harness/tests/common/delegation/test_delegation_adoption.py similarity index 97% rename from harness/tests/common/test_delegation_adoption.py rename to harness/tests/common/delegation/test_delegation_adoption.py index 78276e6..28f21a1 100644 --- a/harness/tests/common/test_delegation_adoption.py +++ b/harness/tests/common/delegation/test_delegation_adoption.py @@ -1,4 +1,5 @@ import copy +from harness.tests._support.data import canonical_digest import hashlib import json import subprocess @@ -49,9 +50,7 @@ class DelegationAdoptionTests(unittest.TestCase): def tearDown(self): self.tmp.cleanup() - @staticmethod - def digest(value): - return hashlib.sha256(json.dumps(value, sort_keys=True, separators=(",", ":")).encode()).hexdigest() + digest = staticmethod(canonical_digest) def declaration(self): return { @@ -179,7 +178,7 @@ class DelegationAdoptionTests(unittest.TestCase): adoption.require_adoption(declaration, expected_source=self.context["source"], expected_stage="implementation") def test_all_git_declaration_paths_reject_symlinks_at_runtime(self): - from harness.tests.common.test_delegation_consumers_integration import RealEvidence + from harness.tests._support.delegation import RealEvidence evidence = RealEvidence(self.base / 'safe-path-fixture') declaration = evidence.declaration() for field in ('bundle_root', 'context_path', 'result_path', 'git_dir'): diff --git a/harness/tests/common/test_delegation_consumers.py b/harness/tests/common/delegation/test_delegation_consumers.py similarity index 96% rename from harness/tests/common/test_delegation_consumers.py rename to harness/tests/common/delegation/test_delegation_consumers.py index 37385c7..ec50852 100644 --- a/harness/tests/common/test_delegation_consumers.py +++ b/harness/tests/common/delegation/test_delegation_consumers.py @@ -1,9 +1,10 @@ +from harness.tests._support.paths import ROOT as SOURCE_ROOT import importlib.util import unittest from pathlib import Path from unittest.mock import patch -ROOT = Path(__file__).resolve().parents[3] +ROOT = SOURCE_ROOT PATH = ROOT / "harness/agents/impl/skills/gitea-issue-implementation/scripts/delivery_adoption.py" spec = importlib.util.spec_from_file_location("delivery_adoption", PATH) mod = importlib.util.module_from_spec(spec) diff --git a/harness/tests/common/test_delegation_consumers_integration.py b/harness/tests/common/delegation/test_delegation_consumers_integration.py similarity index 78% rename from harness/tests/common/test_delegation_consumers_integration.py rename to harness/tests/common/delegation/test_delegation_consumers_integration.py index f39a806..111ed6d 100644 --- a/harness/tests/common/test_delegation_consumers_integration.py +++ b/harness/tests/common/delegation/test_delegation_consumers_integration.py @@ -1,4 +1,6 @@ +from harness.tests._support.paths import ROOT as SOURCE_ROOT import copy +from harness.tests._support.data import canonical_digest import hashlib import importlib.util import json @@ -9,7 +11,7 @@ import tempfile import unittest from pathlib import Path -ROOT = Path(__file__).resolve().parents[3] +ROOT = SOURCE_ROOT SCRATCH = Path('/opt/data/cache/scratch/delegation-adoption-integration') SPEC = ROOT / 'harness/agents/spec/skills/gitea-implementation-spec-review/scripts/review_support.py' REWORK = ROOT / 'harness/agents/impl/skills/gitea-issue-rework/scripts/rework_comment.py' @@ -27,66 +29,7 @@ delivery = load(DELIVERY, 'integration_delivery') native = load(NATIVE, 'integration_native') rework = load(REWORK, 'integration_rework') -class RealEvidence: - def __init__(self, root): - self.root = root - self.bundle = root / 'bundle' - (self.bundle / 'src').mkdir(parents=True) - (self.bundle / 'tests').mkdir() - (self.bundle / 'evidence').mkdir() - (self.bundle / 'src/a.py').write_text('VALUE = 1\n') - (self.bundle / 'tests/test_a.py').write_text('def test_value():\n assert 1 == 1\n') - (self.bundle / 'evidence/red.txt').write_text('red\n') - (self.bundle / 'evidence/green.txt').write_text('green\n') - subprocess.run(['/usr/bin/git', 'init', '-q', str(self.root / 'repo')], check=True) - repo = self.root / 'repo' - (repo / 'src').mkdir(); (repo / 'tests').mkdir() - (repo / 'src/a.py').write_text('VALUE = 1\n') - (repo / 'tests/test_a.py').write_text('def test_value():\n assert 1 == 1\n') - subprocess.run(['/usr/bin/git', '-C', str(repo), 'config', 'user.email', 'test@example.invalid'], check=True) - subprocess.run(['/usr/bin/git', '-C', str(repo), 'config', 'user.name', 'Integration'], check=True) - subprocess.run(['/usr/bin/git', '-C', str(repo), 'add', '.'], check=True) - subprocess.run(['/usr/bin/git', '-C', str(repo), 'commit', '-q', '-m', 'fixture'], check=True) - self.revision = subprocess.check_output(['/usr/bin/git', '-C', str(repo), 'rev-parse', 'HEAD'], text=True).strip() - self.git_dir = repo / '.git' - self.files = {p: hashlib.sha256((self.bundle / p).read_bytes()).hexdigest() for p in ('src/a.py', 'tests/test_a.py')} - self.bundle_sha = self.digest(self.files) - self.context = {'schema': 'gitea-agent.delegation-evidence-context/v1', - 'source': {'kind': 'git_tree', 'revision': self.revision}, - 'stage': 'spec-review', 'bundle_sha256': self.bundle_sha, - 'files': self.files, 'allowed_actions': ['static', 'execute'], - 'obligations': [{'id': 'static', 'kind': 'static'}, {'id': 'rg', 'kind': 'red_green'}], - 'executions': [self.execution('red', 1, 'assertion', 'evidence/red.txt'), - self.execution('green', 0, 'none', 'evidence/green.txt')]} - line_hash = hashlib.sha256(b'VALUE = 1\n').hexdigest() - self.result = {'schema': 'gitea-agent.delegation-evidence-result/v1', - 'source': copy.deepcopy(self.context['source']), 'stage': 'spec-review', - 'bundle_sha256': self.bundle_sha, - 'claims': [{'id': 'static', 'status': 'supported', 'anchors': [{'kind': 'lines', 'path': 'src/a.py', 'start_line': 1, 'end_line': 1, 'text_sha256': line_hash}], 'execution_ids': []}, - {'id': 'rg', 'status': 'supported', 'anchors': [{'kind': 'test', 'path': 'tests/test_a.py', 'symbol': 'test_value'}], 'execution_ids': ['red', 'green']}]} - self.context_path = root / 'context.json'; self.result_path = root / 'result.json' - self.write() - - @staticmethod - def digest(value): - return hashlib.sha256(json.dumps(value, sort_keys=True, separators=(',', ':')).encode()).hexdigest() - - def execution(self, eid, code, kind, path): - return {'id': eid, 'test_id': 'tests/test_a.py::test_value', 'bundle_sha256': self.bundle_sha, - 'test_file_sha256': self.files['tests/test_a.py'], 'argv': ['python', '-m', 'pytest'], - 'exit_code': code, 'failure_kind': kind, 'evidence_path': path, - 'evidence_sha256': hashlib.sha256((self.bundle / path).read_bytes()).hexdigest(), 'origin': 'direct'} - - def write(self): - self.context_path.write_text(json.dumps(self.context)) - self.result_path.write_text(json.dumps(self.result)) - - def declaration(self): - return {'mode': 'delegated', 'bundle_root': str(self.bundle), 'context_path': str(self.context_path), - 'result_path': str(self.result_path), 'git_dir': str(self.git_dir)} - - def set_stage(self, stage): - self.context['stage'] = stage; self.result['stage'] = stage; self.write() +from harness.tests._support.delegation import RealEvidence class IntegrationTests(unittest.TestCase): def setUp(self): @@ -137,7 +80,7 @@ class IntegrationTests(unittest.TestCase): self.assertEqual(calls, []) def bound_publication_fixture(self): - from harness.tests.review import test_native_review as fixture + from harness.tests._support import review as fixture plan, before, after, exact = fixture.publication_fixture() old_head = plan['target']['head'] def replace(value): diff --git a/harness/tests/common/test_delegation_evidence.py b/harness/tests/common/delegation/test_delegation_evidence.py similarity index 98% rename from harness/tests/common/test_delegation_evidence.py rename to harness/tests/common/delegation/test_delegation_evidence.py index ff6fe3c..dcefe8d 100644 --- a/harness/tests/common/test_delegation_evidence.py +++ b/harness/tests/common/delegation/test_delegation_evidence.py @@ -1,4 +1,6 @@ +from harness.tests._support.paths import ROOT as SOURCE_ROOT import copy +from harness.tests._support.data import canonical_digest import hashlib import json import os @@ -10,7 +12,7 @@ from pathlib import Path from harness.lib import delegation_evidence as evidence -ROOT = Path(__file__).resolve().parents[3] +ROOT = SOURCE_ROOT SCHEMA_C = "gitea-agent.delegation-evidence-context/v1" SCHEMA_R = "gitea-agent.delegation-evidence-result/v1" @@ -63,9 +65,7 @@ class DelegationEvidenceTests(unittest.TestCase): def tearDown(self): self.tmp.cleanup() - @staticmethod - def digest(value): - return hashlib.sha256(json.dumps(value, sort_keys=True, separators=(",", ":")).encode()).hexdigest() + digest = staticmethod(canonical_digest) def line_hash(self, path, start, end): lines = (self.root / path).read_bytes().splitlines(keepends=True) diff --git a/harness/tests/common/test_delegation_evidence_safety.py b/harness/tests/common/delegation/test_delegation_evidence_safety.py similarity index 98% rename from harness/tests/common/test_delegation_evidence_safety.py rename to harness/tests/common/delegation/test_delegation_evidence_safety.py index 8903410..ee92819 100644 --- a/harness/tests/common/test_delegation_evidence_safety.py +++ b/harness/tests/common/delegation/test_delegation_evidence_safety.py @@ -1,6 +1,8 @@ """Independent safety/regression coverage for delegation evidence acceptance.""" +from harness.tests._support.paths import ROOT as SOURCE_ROOT import copy +from harness.tests._support.data import canonical_digest import hashlib import json import os @@ -13,7 +15,7 @@ from pathlib import Path from harness.lib import delegation_evidence as evidence -ROOT = Path(__file__).resolve().parents[3] +ROOT = SOURCE_ROOT HELPER = ROOT / "harness" / "lib" / "delegation_evidence.py" CONTEXT_SCHEMA = "gitea-agent.delegation-evidence-context/v1" RESULT_SCHEMA = "gitea-agent.delegation-evidence-result/v1" @@ -47,11 +49,7 @@ class DelegationEvidenceSafetyTests(unittest.TestCase): def tearDown(self): self.tmp.cleanup() - @staticmethod - def digest(value): - return hashlib.sha256( - json.dumps(value, sort_keys=True, separators=(",", ":")).encode() - ).hexdigest() + digest = staticmethod(canonical_digest) def make_context(self, *, actions=None, obligations=None, executions=None): return { diff --git a/harness/tests/common/evidence/__init__.py b/harness/tests/common/evidence/__init__.py new file mode 100644 index 0000000..839dec1 --- /dev/null +++ b/harness/tests/common/evidence/__init__.py @@ -0,0 +1 @@ +"""evidence regression tests.""" diff --git a/harness/tests/common/test_pytest_evidence_recipe.py b/harness/tests/common/evidence/test_pytest_evidence_recipe.py similarity index 92% rename from harness/tests/common/test_pytest_evidence_recipe.py rename to harness/tests/common/evidence/test_pytest_evidence_recipe.py index c8e3114..7d659f9 100644 --- a/harness/tests/common/test_pytest_evidence_recipe.py +++ b/harness/tests/common/evidence/test_pytest_evidence_recipe.py @@ -3,17 +3,18 @@ Run with an independently admitted pytest interpreter for execution coverage. No dependency installation occurs in this test or the production workflow. """ +from harness.tests._support.paths import ROOT as SOURCE_ROOT import importlib.util import json import os from pathlib import Path -from harness.tests.common.impl_contract_docs import read_contract +from harness.tests._support.contracts import read_contract import subprocess import sys import tempfile import unittest -ROOT = Path(__file__).resolve().parents[3] +ROOT = SOURCE_ROOT SKILL = ROOT / 'harness/agents/impl/skills/gitea-issue-rework/SKILL.md' HEADING = '### Conditional pytest evidence recipe' @@ -27,9 +28,7 @@ class PytestEvidenceRecipeTests(unittest.TestCase): def test_documentation_contract_limits_recipe_to_existing_pytest_projects(self): section, argv = self.recipe() - for rule in ['pytest를 이미 사용하는', 'PYTEST_ADDOPTS', 'plugin', - 'evidence_gap', '이전 실행', '0으로', 'schema', 'unittest', - 'warning', 'timeout', 'output_limit']: + for rule in ['PYTEST_ADDOPTS', 'plugin', 'evidence_gap', 'timeout', 'output_limit']: self.assertIn(rule, section) self.assertIsInstance(argv, list) self.assertEqual(argv[:4], ['{python}', '-B', '-m', 'pytest']) diff --git a/harness/tests/common/test_workflow_guard.py b/harness/tests/common/evidence/test_workflow_guard.py similarity index 100% rename from harness/tests/common/test_workflow_guard.py rename to harness/tests/common/evidence/test_workflow_guard.py diff --git a/harness/tests/common/locking/__init__.py b/harness/tests/common/locking/__init__.py new file mode 100644 index 0000000..501be73 --- /dev/null +++ b/harness/tests/common/locking/__init__.py @@ -0,0 +1 @@ +"""locking regression tests.""" diff --git a/harness/tests/common/test_managed_lock.py b/harness/tests/common/locking/test_managed_lock.py similarity index 98% rename from harness/tests/common/test_managed_lock.py rename to harness/tests/common/locking/test_managed_lock.py index 73abf69..0bfa6e7 100644 --- a/harness/tests/common/test_managed_lock.py +++ b/harness/tests/common/locking/test_managed_lock.py @@ -1,3 +1,4 @@ +from harness.tests._support.paths import ROOT as SOURCE_ROOT import fcntl import os import stat @@ -242,13 +243,13 @@ class ManagedLockTests(unittest.TestCase): self.assertEqual((before.st_ino, before.st_size, before.st_mode), (after.st_ino, after.st_size, after.st_mode)) def test_lock_contention_from_external_cwd_without_pythonpath(self): - root = Path(__file__).resolve().parents[3] + root = SOURCE_ROOT script = """ import sys import unittest sys.path.insert(0, sys.argv[1]) suite = unittest.defaultTestLoader.loadTestsFromName( - 'harness.tests.common.test_managed_lock.ManagedLockTests.' + 'harness.tests.common.locking.test_managed_lock.ManagedLockTests.' 'test_other_process_cannot_acquire_while_context_alive') result = unittest.TextTestRunner().run(suite) raise SystemExit(not result.wasSuccessful()) @@ -294,7 +295,7 @@ raise SystemExit(not result.wasSuccessful()) if not root.is_absolute() or not root.is_dir(): raise RuntimeError("HARNESS_ROOT must be an existing absolute directory") root = root.resolve() - if root != Path(__file__).resolve().parents[3]: + if root != SOURCE_ROOT: raise RuntimeError("HARNESS_ROOT does not match this test installation") with tempfile.TemporaryDirectory() as d: path = Path(d) / "process.lock" diff --git a/harness/tests/common/test_managed_lock_broker.py b/harness/tests/common/locking/test_managed_lock_broker.py similarity index 98% rename from harness/tests/common/test_managed_lock_broker.py rename to harness/tests/common/locking/test_managed_lock_broker.py index 92205c0..daf5449 100644 --- a/harness/tests/common/test_managed_lock_broker.py +++ b/harness/tests/common/locking/test_managed_lock_broker.py @@ -1,3 +1,4 @@ +from harness.tests._support.paths import HARNESS as SOURCE_HARNESS import json import os import secrets @@ -14,7 +15,7 @@ from unittest.mock import patch from harness.lib.managed_lock import ManagedLock, probe_free from harness.lib import managed_lock_broker as broker -BROKER = Path(__file__).parents[2] / "lib" / "managed_lock_broker.py" +BROKER = SOURCE_HARNESS / "lib" / "managed_lock_broker.py" def wait_ready(proc, timeout=3): @@ -86,7 +87,7 @@ class ManagedLockBrokerTests(unittest.TestCase): def test_operation_specific_broker_response_documentation(self): import ast - harness = Path(__file__).parents[2] + harness = SOURCE_HARNESS operations = (harness / "operations/repository-cache.md").read_text() reference = (harness / "agents/review/skills/gitea-code-review/references/native-review.md").read_text() self.assertIn("### Operation-specific broker responses", operations) @@ -196,13 +197,12 @@ class ManagedLockBrokerTests(unittest.TestCase): self.assertNotIn("Traceback", result.stderr) def test_canonical_role_values_and_example_are_documented(self): - doc = (Path(__file__).parents[2] / "operations/repository-cache.md").read_text() - section = doc.split("### Canonical command templates", 1)[1].split("## Lock 규칙", 1)[0] - for role in ("gitea-spec-agent", "gitea-impl-agent", "gitea-review-agent"): - self.assertIn(f"`{role}`", section) - self.assertIn("--role gitea-spec-agent --run-id", section) - self.assertIn("spec`/`impl`/`review", section) - self.assertIn("authorization", section) + doc = (SOURCE_HARNESS / 'operations/repository-cache.md').read_text() + section = doc.split('### Canonical command templates', 1)[1].split('## Lock 규칙', 1)[0] + for role in ('gitea-spec-agent', 'gitea-impl-agent', 'gitea-review-agent'): + self.assertIn(f'`{role}`', section) + self.assertIn('--role gitea-spec-agent --run-id', section) + self.assertIn('spec`/`impl`/`review', section) def test_constructor_rejects_lexical_root_and_handle_aliases(self): with tempfile.TemporaryDirectory() as d: diff --git a/harness/tests/common/repository/__init__.py b/harness/tests/common/repository/__init__.py new file mode 100644 index 0000000..65b7deb --- /dev/null +++ b/harness/tests/common/repository/__init__.py @@ -0,0 +1 @@ +"""repository regression tests.""" diff --git a/harness/tests/common/test_repository_prefetch.py b/harness/tests/common/repository/test_repository_prefetch.py similarity index 97% rename from harness/tests/common/test_repository_prefetch.py rename to harness/tests/common/repository/test_repository_prefetch.py index a41f1dc..2861b6e 100644 --- a/harness/tests/common/test_repository_prefetch.py +++ b/harness/tests/common/repository/test_repository_prefetch.py @@ -1,4 +1,5 @@ """Synthetic local Git pre-fetch gate checks; no provider/network operations.""" +from harness.tests._support.paths import ROOT as SOURCE_ROOT import json import os import subprocess @@ -6,7 +7,7 @@ import tempfile import unittest from pathlib import Path -ROOT = Path(__file__).resolve().parents[3] +ROOT = SOURCE_ROOT SCRIPT = ROOT / 'harness/lib/repository_prefetch.py' @@ -131,7 +132,7 @@ class RepositoryPrefetchTests(unittest.TestCase): section = text.split('## 기존 cache 갱신', 1)[1].split('\n## ', 1)[0] self.assertIn('repository_prefetch.py', section) self.assertLess(section.index('repository_prefetch.py'), section.index('prune/fetch')) - for rule in ['--git-dir', 'repository lock', '자동 repair', 'exit 0', 'exit 1', 'exit 2']: + for rule in ['--git-dir', 'repository lock', 'exit 0', 'exit 1', 'exit 2']: self.assertIn(rule, section) def fingerprint(self, *roots): diff --git a/harness/tests/common/test_runtime_root.py b/harness/tests/common/repository/test_runtime_root.py similarity index 99% rename from harness/tests/common/test_runtime_root.py rename to harness/tests/common/repository/test_runtime_root.py index 85a1b66..7dc3892 100644 --- a/harness/tests/common/test_runtime_root.py +++ b/harness/tests/common/repository/test_runtime_root.py @@ -1,3 +1,4 @@ +from harness.tests._support.paths import HARNESS as SOURCE_HARNESS import json import os import stat @@ -156,7 +157,7 @@ class RuntimeRootTests(unittest.TestCase): with tempfile.TemporaryDirectory() as tmp: root = Path(tmp); repo = self.cache(root) self.git('init', '--bare', repo / 'repository.git') - base = Path(__file__).resolve().parents[2] + base = SOURCE_HARNESS cmd = [sys.executable, '-B', str(base / 'lib' / 'managed_lock_broker.py'), 'serve', '--root', str(root), '--owner', 'owner', '--repository', 'repo', '--issue', '17', '--role', 'gitea-impl-agent', '--run-id', 'cli-workflow', '--ttl-seconds', '30'] diff --git a/harness/tests/common/test_test_inventory_binding.py b/harness/tests/common/repository/test_test_inventory_binding.py similarity index 99% rename from harness/tests/common/test_test_inventory_binding.py rename to harness/tests/common/repository/test_test_inventory_binding.py index adac9ed..00dba4f 100644 --- a/harness/tests/common/test_test_inventory_binding.py +++ b/harness/tests/common/repository/test_test_inventory_binding.py @@ -1,4 +1,5 @@ """Real-Git regressions; no pytest or product execution dependency.""" +from harness.tests._support.paths import ROOT as SOURCE_ROOT import copy import hashlib import json @@ -12,7 +13,7 @@ from unittest.mock import patch from harness.lib import test_inventory_binding as binding -ROOT = Path(__file__).resolve().parents[3] +ROOT = SOURCE_ROOT class InventoryBindingTests(unittest.TestCase): diff --git a/harness/tests/common/test_verification_root.py b/harness/tests/common/repository/test_verification_root.py similarity index 100% rename from harness/tests/common/test_verification_root.py rename to harness/tests/common/repository/test_verification_root.py diff --git a/harness/tests/common/runner/__init__.py b/harness/tests/common/runner/__init__.py new file mode 100644 index 0000000..12ecf9f --- /dev/null +++ b/harness/tests/common/runner/__init__.py @@ -0,0 +1 @@ +"""runner regression tests.""" diff --git a/harness/tests/common/test_canonical_test_runner.py b/harness/tests/common/runner/test_canonical_test_runner.py similarity index 88% rename from harness/tests/common/test_canonical_test_runner.py rename to harness/tests/common/runner/test_canonical_test_runner.py index 00b7991..db4dbff 100644 --- a/harness/tests/common/test_canonical_test_runner.py +++ b/harness/tests/common/runner/test_canonical_test_runner.py @@ -1,3 +1,4 @@ +from harness.tests._support.paths import ROOT as SOURCE_ROOT import json import os import subprocess @@ -8,7 +9,7 @@ from unittest import mock from pathlib import Path -REPO = Path(__file__).resolve().parents[3] +REPO = SOURCE_ROOT RUNNER = REPO / "harness" / "tests" / "run.py" @@ -76,17 +77,19 @@ class CanonicalRunnerTests(unittest.TestCase): root = self.make_suite() (root / "harness/tests/common/test_common.py").write_text("import unittest\n") result = self.execute(root, "--collect-only") - self.assertNotEqual(result.returncode, 0) + self.assertEqual(result.returncode, 2) self.assertIn("zero", result.stderr.lower()) - root = self.make_suite() - (root / "harness/tests/common/test_common.py").write_text('''import unittest -class X(unittest.TestCase): - def test_one(self): pass -def test_free(): pass -''') - result = self.execute(root, "--collect-only") - self.assertNotEqual(result.returncode, 0) - self.assertIn("free-function", result.stderr.lower()) + # Preserve both mixed TestCase/free-function and free-only inputs. + for source in ( + "import unittest\nclass X(unittest.TestCase):\n def test_one(self): pass\ndef test_free(): pass\n", + "def test_free(): pass\n", + ): + with self.subTest(source=source): + root = self.make_suite() + (root / "harness/tests/common/test_common.py").write_text(source) + result = self.execute(root, "--collect-only") + self.assertEqual(result.returncode, 2) + self.assertIn("free-function", result.stderr.lower()) def test_direct_case_does_not_short_circuit_cycle_or_unknown_base(self): for source, word in ( @@ -197,12 +200,21 @@ assert not Path(sys.argv[2]).exists() def test_collect_only_reports_exact_roles_ids_and_hashes(self): root = self.make_suite() + (root / "harness/tests/spec/test_spec.py").write_text( + "import unittest\nclass X(unittest.TestCase):\n def test_one(self): pass\n def test_two(self): pass\n") result = self.execute(root, "--collect-only") self.assertEqual(result.returncode, 0, result.stderr) data = json.loads(result.stdout) self.assertTrue(data["collection_only"]) - self.assertEqual(data["role_counts"], {"common": 1, "spec": 1, "impl": 1, "review": 1}) - self.assertEqual(len(data["ids"]), 4) + self.assertEqual(data["role_counts"], {"common": 1, "spec": 2, "impl": 1, "review": 1}) + self.assertEqual(data["ids"], sorted([ + "harness.tests.common.test_common.Example.test_one", + "harness.tests.spec.test_spec.X.test_one", "harness.tests.spec.test_spec.X.test_two", + "harness.tests.impl.test_impl.Example.test_one", "harness.tests.review.test_review.Example.test_one", + ])) + import hashlib + for path, digest in data["source_sha256"].items(): + self.assertEqual(digest, hashlib.sha256((root / path).read_bytes()).hexdigest()) self.assertEqual(set(data["source_sha256"]), {"harness/tests/common/test_common.py", "harness/tests/spec/test_spec.py", "harness/tests/impl/test_impl.py", "harness/tests/review/test_review.py"}) def test_missing_role_marker_duplicate_and_unexpected_file_fail_before_run(self): @@ -220,23 +232,8 @@ assert not Path(sys.argv[2]).exists() self.assertNotEqual(result.returncode, 0) self.assertIn("canonical", result.stderr.lower()) - def test_loader_omission_addition_zero_file_and_unsupported_function_fail(self): - cases = { - "omission": lambda root: (root / "harness/tests/common/test_common.py").write_text("import unittest\ndef load_tests(loader, tests, pattern): return unittest.TestSuite()\nclass X(unittest.TestCase):\n def test_one(self): pass\n"), - "zero": lambda root: (root / "harness/tests/review/test_review.py").write_text("import unittest\n"), - "free": lambda root: (root / "harness/tests/impl/test_impl.py").write_text("def test_free(): pass\n"), - } - for name, mutate in cases.items(): - with self.subTest(name=name): - root = self.make_suite(); mutate(root) - result = self.execute(root, "--collect-only") - self.assertNotEqual(result.returncode, 0) - root = self.make_suite() - (root / "harness/tests/spec/test_spec.py").write_text("import unittest\nclass X(unittest.TestCase):\n def test_one(self): pass\n def test_two(self): pass\n") - result = self.execute(root, "--collect-only") - self.assertEqual(result.returncode, 0, result.stderr) - def test_import_failure_source_drift_nested_omission_and_runtime_failure(self): + def test_import_failure_source_drift_and_nested_omission(self): root = self.make_suite() (root / "harness/tests/impl/test_impl.py").write_text("raise RuntimeError('broken import')\n") result = self.execute(root, "--collect-only") @@ -252,12 +249,6 @@ assert not Path(sys.argv[2]).exists() "class X(unittest.TestCase):\n def test_one(self): pass\n") result = self.execute(root, "--collect-only") self.assertNotEqual(result.returncode, 0) - root = self.make_suite() - (root / "harness/tests/common/test_common.py").write_text( - "import unittest\nclass X(unittest.TestCase):\n def test_one(self): self.fail('boom')\n") - result = self.execute(root) - self.assertNotEqual(result.returncode, 0) - self.assertIn("FAILED", result.stderr + result.stdout) def test_package_manifest_drift_and_extra_zero_file_fail_closed(self): cases = ( diff --git a/harness/tests/common/test_verification_contract_docs.py b/harness/tests/common/test_verification_contract_docs.py deleted file mode 100644 index 1a7e2ef..0000000 --- a/harness/tests/common/test_verification_contract_docs.py +++ /dev/null @@ -1,232 +0,0 @@ -import re -import subprocess -import sys -import unittest -from pathlib import Path -from harness.tests.common.impl_contract_docs import read_contract - -ROOT = Path(__file__).resolve().parents[3] -SCRIPT = ROOT / "harness/agents/impl/skills/gitea-issue-rework/scripts/evidence_manifest.py" -PREPARE_SCRIPT = ROOT / "harness/lib/verification_root.py" -CACHE = ROOT / "harness/operations/repository-cache.md" -SKILLS = { - "implementation": ROOT / "harness/agents/impl/skills/gitea-issue-implementation/SKILL.md", - "rework": ROOT / "harness/agents/impl/skills/gitea-issue-rework/SKILL.md", - "review": ROOT / "harness/agents/spec/skills/gitea-implementation-spec-review/SKILL.md", -} -CANONICAL = "/repositories///worktrees/issue-/verification/" - - -class VerificationContractDocumentationTests(unittest.TestCase): - def test_runtime_policy_is_broker_owned_and_separate_from_verification(self): - text = CACHE.read_text(encoding="utf-8") - self.assertIn("runtime/", text) - self.assertIn("broker.handle", text) - self.assertIn("broker.sock", text) - self.assertIn("ready JSON", text) - self.assertIn("runtime 생성은 승인이나 admission을 의미하지 않는다", text) - self.assertIn("active resource", text) - self.assertIn("unknown", text) - self.assertIn("성공 기록을 실패로 덮어쓰지 않는다", text) - for path in SKILLS.values(): - with self.subTest(path=path.name): - role = read_contract(path) - gate = role.split("## Managed-lock broker gate", 1)[1].split("\n## ", 1)[0] - self.assertIn("broker-generated", gate) - self.assertIn("ready JSON", gate) - self.assertNotIn("pre-created", gate) - self.assertNotIn("pre-create", gate) - - def test_role_runtime_startup_precedes_and_does_not_replace_admission(self): - roles = [path for path in (ROOT / 'harness/agents').rglob('SKILL.md') - if 'broker-generated runtime' in read_contract(path)] - self.assertEqual(len(roles), 6) - for path in roles: - with self.subTest(role=path.parent.name): - gate = read_contract(path).split('## Managed-lock broker gate', 1)[1].split('\n## ', 1)[0] - self.assertIn('ready 이후 각 workflow의 진입 검증·승인 조건을 확인한다', gate) - self.assertIn('Runtime 생성은 approval이나 worktree admission을 대신하지 않는다', gate) - self.assertIn('ready JSON의 `handle`/`socket`', gate) - - def test_evidence_plan_disables_python_bytecode_in_clean_checkout(self): - import json - template = ROOT / "harness/agents/impl/skills/gitea-issue-rework/templates/evidence-plan.json" - plan = json.loads(template.read_text()) - self.assertIn("-B", plan["commands"][0]["argv"]) - self.assertIn("PYTHONDONTWRITEBYTECODE", plan["command_env_allowlist"]) - - def test_full_suite_tracked_background_reconciles_before_retry(self): - text = read_contract(SKILLS['rework']) - heading = '### Tracked bounded background evidence execution' - self.assertIn(heading, text) - section = text.split(heading, 1)[1].split('\n### ', 1)[0] - for rule in ['foreground tool 시간창', 'tracked background', - 'process/session handle', 'run-id', 'source SHA', 'plan digest', - 'output path', 'timeout_seconds', '유한한 outer deadline', - 'manifest', 'validate', 'cleanup', 'pending/partial', - '중복 실행하지 않는다', '새 run-id', '별도 승인', - 'TDD RED', '세션 종료 이후의 지속성을 보장하지 않는다']: - with self.subTest(rule=rule): - self.assertIn(rule, section) - self.assertIn('자동 retry하지 않는다', section) - self.assertIn('기존 artifact를 덮어쓰지 않는다', section) - - def test_command_deadline_and_archive_policy_is_explicit(self): - import json - template = ROOT / "harness/agents/impl/skills/gitea-issue-rework/templates/evidence-plan.json" - for command in json.loads(template.read_text())["commands"]: - self.assertIsInstance(command["timeout_seconds"], (int, float)) - self.assertGreater(command["timeout_seconds"], 0) - self.assertLessEqual(command["timeout_seconds"], 3600) - for path in (CACHE, *SKILLS.values()): - text = read_contract(path) - for token in ("evidence-manifest/v3", "timeout_seconds", "process group", - "validate-archived-v2", "TDD RED", "자동 재실행"): - with self.subTest(path=str(path), token=token): - self.assertIn(token, text) - help_result = subprocess.run([sys.executable, str(SCRIPT), "validate-archived-v2", "--help"], - capture_output=True, text=True, timeout=10) - self.assertEqual(help_result.returncode, 0, help_result.stderr) - for option in ("--contract-path", "--repo-root", "--artifact-root", "--manifest"): - self.assertIn(option, help_result.stdout) - - def test_all_roles_reference_canonical_external_artifacts(self): - for path in (CACHE, *SKILLS.values()): - with self.subTest(path=path.name): - text = read_contract(path) - self.assertIn(CANONICAL, text) - self.assertIn("evidence_gap", text) - self.assertIn("[REDACTED]", text) - self.assertIn("evidence-manifest/v3", text) - - def test_examples_match_real_helper_cli_not_unsupported_options(self): - for path in (SKILLS["implementation"], SKILLS["rework"]): - section = read_contract(path).split("## Canonical external verification evidence", 1)[1] - match = re.search(r"```sh\n(.*?)```", section, re.S) - assert match is not None - block = match.group(1).replace("\\\n", " ") - commands = [line for line in block.splitlines() if line.startswith("python3 ")] - self.assertEqual(len(commands), 3) - for op, command in zip(("prepare", "run", "validate"), commands): - with self.subTest(skill=path.parent.name, op=op): - declared = set(re.findall(r"--[a-z-]+", command)) - if op == "prepare": - self.assertIn("harness/lib/verification_root.py", command) - help_result = subprocess.run( - [sys.executable, str(PREPARE_SCRIPT), op, "--help"], - text=True, - capture_output=True, - ) - self.assertEqual(help_result.returncode, 0, help_result.stderr) - supported = set(re.findall(r"--[a-z-]+", help_result.stdout)) - required = {"--root", "--owner", "--repository", "--issue", "--run-id", "--handle"} - self.assertEqual(declared, required) - self.assertTrue(declared <= supported) - else: - help_result = subprocess.run([sys.executable, str(SCRIPT), op, "--help"], text=True, capture_output=True) - self.assertEqual(help_result.returncode, 0) - supported = set(re.findall(r"--[a-z-]+", help_result.stdout)) - required = {"--contract-path", "--repo-root", "--artifact-root"} | ({"--plan", "--output"} if op == "run" else {"--manifest"}) - self.assertEqual(declared, required) - self.assertTrue(declared <= supported) - self.assertIn("exact current HEAD", section) - self.assertIn("validate-legacy", section) - self.assertIn("exclusion 없음", section) - - def test_prepare_contract_binds_root_and_forbids_recursive_parent_creation(self): - for path in (CACHE, SKILLS["implementation"], SKILLS["rework"]): - with self.subTest(path=path.name): - text = read_contract(path) - self.assertIn("exact harness root", text) - self.assertIn("Path.mkdir(parents=True, mode=0o700)", text) - self.assertIn("자동 chmod", text) - - def test_checkout_cleanup_preserves_verification_without_recursive_delete(self): - text = CACHE.read_text() - cleanup = text.split("## Worktree 실패 처리와 제거", 1)[1].split("## Credential 안전", 1)[0] - self.assertIn("`verification/`이 남으면 Issue directory를 보존한다", cleanup) - self.assertIn("Issue directory가 남는 것은 checkout 제거 실패가 아니다", cleanup) - self.assertIn("digest가 제거 전과 같음", cleanup) - self.assertIn("Artifact 폐기는 별도의 exact 경로 승인", text) - self.assertIn("mode `0700`", text) - self.assertIn("mode `0600`", text) - - def test_spec_review_uses_immutable_source_and_not_moving_checkout(self): - section = SKILLS["review"].read_text().split("## External Issue verification evidence boundary", 1)[1] - self.assertIn("제품 테스트를 실행하지 않", section) - self.assertIn("moving implementation checkout", section) - self.assertIn("authoritative blob evidence", section) - self.assertIn("trusted `harness/.runtime`", section) - self.assertIn("Producer evidence와 reviewer 실행을 구분", section) - - def test_workflow_guard_records_reuse_existing_verification_location(self): - text = CACHE.read_text() - guard = text.split('### Shared workflow guard gates', 1)[1].split('Issue별 검증 실행 기록의 정본', 1)[0] - self.assertNotIn('or verification root', guard) - for path in (SKILLS['implementation'], SKILLS['rework']): - self.assertNotIn('outside product checkout and verification root', read_contract(path)) - self.assertIn('verification/', guard) - - def test_workflow_guard_readonly_scope_does_not_ban_approved_worker_edits(self): - text = CACHE.read_text() - guard = text.split('### Shared workflow guard gates', 1)[1].split('Issue별 검증 실행 기록의 정본', 1)[0] - self.assertIn('읽기 전용', guard) - self.assertIn('쓰기 worker는 승인된 변경 allowlist', guard) - - def test_workflow_guard_snapshot_protects_scope_before_delegation(self): - implementation = read_contract(SKILLS["implementation"]) - self.assertIn("workflow_guard.py snapshot", implementation) - self.assertIn("compare-source --repo-root", implementation) - self.assertIn("immutable protected scope", implementation) - - def test_workflow_guard_inventory_is_checked_before_adopting_results(self): - rework = read_contract(SKILLS["rework"]) - self.assertIn("workflow_guard.py check-tests", rework) - self.assertIn("test inventory", rework) - self.assertIn("RED/GREEN records must be named", rework) - - def test_workflow_guard_contract_preservation_precedes_spec_verdict(self): - spec = SKILLS["review"].read_text() - self.assertIn("workflow_guard.py check-contracts", spec) - self.assertIn("baseline inventory", spec) - self.assertIn("Gate failure invalidates workflow evidence", spec) - - def test_workflow_guard_nested_schema_is_actionable_and_claims_are_not_authenticated(self): - text = CACHE.read_text() - section = text.split('### Shared workflow guard gates', 1)[1].split('Issue별 검증 실행 기록의 정본', 1)[0] - for token in ('#### Nested input contracts', 'baseline_collection', 'inventory_sha256', - 'baseline_status', 'expected_failure', 'replacement_ids', 'kind', - 'delete', 'rename', 'status', 'review_evidence', 'scope_sha256', - 'expected_exit_code', 'failure_kind', 'semantic_key', 'current_keys', - 'observation', 'intentionally_changed', 'sort_keys=True', - 'separators', 'UTF-8', 'shape-only', 'one row per baseline semantic key'): - with self.subTest(token=token): - self.assertIn(token, section) - import json - blocks = re.findall(r'```json\n(.*?)```', section, re.S) - self.assertEqual(len(blocks), 2, 'provide runnable test and contract input examples') - self.assertEqual(set(json.loads(blocks[0])), {'baseline_ids', 'current_ids', 'baseline_status', - 'current_status', 'baseline_collection', 'current_collection', 'authorized_changes', 'changed_tests', 'findings'}) - self.assertEqual(set(json.loads(blocks[1])), {'baseline', 'current', 'mappings', 'stages', 'precedence'}) - - def test_mapping_schema_types_use_raw_source_not_redacted_tool_rendering(self): - # Authorization-looking prose can be masked by read-tool transport. - # Inspect the approved public schema in raw source; never disable redaction. - declaration = 'authorization: string, verification: string' - masked = 'authorization: ' + '*' * 3 + ' verification' - helper = ROOT / 'harness/lib/workflow_guard.py' - for path in (helper, CACHE): - text = read_contract(path) - with self.subTest(path=path.name): - self.assertEqual(text.count(declaration), 1) - self.assertNotIn(masked, text) - - def test_rework_no_longer_excludes_repo_evidence_from_clean_tree(self): - text = read_contract(SKILLS["rework"]) - self.assertNotIn("plan과 manifest를 제외한 clean tree", text) - self.assertIn("새 evidence는 checkout 밖", text) - self.assertIn("archived v1 read-only", text) - - -if __name__ == "__main__": - unittest.main() diff --git a/harness/tests/impl/contracts/__init__.py b/harness/tests/impl/contracts/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/harness/tests/impl/contracts/test_impl_architecture_comparator.py b/harness/tests/impl/contracts/test_impl_architecture_comparator.py new file mode 100644 index 0000000..af8005b --- /dev/null +++ b/harness/tests/impl/contracts/test_impl_architecture_comparator.py @@ -0,0 +1,147 @@ +import copy +import json +import re +import unittest +from harness.tests._support.contracts import read_contract, REQUIRED +from harness.tests._support.paths import ROOT + +COMMIT_FILES_START = "" +COMMIT_FILES_END = "" + + +def _execute_documented_commit_files_comparator(): + """Load the single trusted Python example from the canonical workflow.""" + path = ROOT / "harness/agents/impl/skills/gitea-issue-rework/references/workflow.md" + text = path.read_text() + start = text.count(COMMIT_FILES_START) + end = text.count(COMMIT_FILES_END) + if start != 1 or end != 1: + raise AssertionError("workflow must contain exactly one commit-files example boundary pair") + section = text.split(COMMIT_FILES_START, 1)[1].split(COMMIT_FILES_END, 1)[0] + snippets = re.findall(r"```python\s*\n(.*?)\n```", section, re.DOTALL) + if len(snippets) != 1: + raise AssertionError("commit-files example must contain exactly one Python fenced block") + namespace = {} + exec(compile(snippets[0], str(path) + "#commit-files-example", "exec"), namespace) + comparator = namespace.get("normalize_commit_files") + if not callable(comparator): + raise AssertionError("Python example must define normalize_commit_files(rows)") + heading = "#### commit.files 순서 비교 계약\n" + if text.count(heading) != 1: + raise AssertionError("workflow must contain one scoped commit-files comparison contract") + contract = text.split(heading, 1)[1].split("\n### 7.", 1)[0] + return comparator, contract + + +def _commit(sha, message, parent, files): + return { + "sha": sha, + "commit": {"message": message, "author": {"name": "Test Author", "email": "author@example.invalid"}}, + "url": "https://example.invalid/commit/" + sha, + "parents": [{"sha": parent, "url": "https://example.invalid/commit/" + parent}], + "verification": {"verified": True, "reason": "valid", "arbitrary": [1, {"keep": True}]}, + "files": files, + "provider_extension": {"retained": ["opaque", 7]}, + } + + +def _file(filename, status, additions): + return { + "filename": filename, + "status": status, + "additions": additions, + "deletions": 2, + "changes": additions + 2, + "patch": "@@ -1 +1 @@\n-old\n+new", + "provider_extension": {"nested": [filename, {"opaque": True}]}, + } + +class DocumentedCommitFilesComparatorTests(unittest.TestCase): + SHA_A = '0123456789abcdef0123456789abcdef01234567' + SHA_B = '89abcdef0123456789abcdef0123456789abcdef' + PARENT_A = '1111111111111111111111111111111111111111' + PARENT_B = '2222222222222222222222222222222222222222' + + def setUp(self): + self.normalize, self.workflow = _execute_documented_commit_files_comparator() + + def fixture(self): + return [_commit(self.SHA_A, 'first message', self.PARENT_A, [_file('src/zeta.py', 'modified', 3), _file('docs/alpha.md', 'added', 5)]), _commit(self.SHA_B, 'second message', self.PARENT_B, [_file('tests/test_beta.py', 'modified', 7), _file('README.md', 'added', 11)])] + + def test_inner_file_order_is_ignored_but_outer_commit_order_is_preserved(self): + first = self.fixture() + second = copy.deepcopy(first) + for commit in second: + commit['files'].reverse() + self.assertEqual(self.normalize(first), self.normalize(second)) + self.assertNotEqual(self.normalize(first), self.normalize(list(reversed(second)))) + + def test_identity_and_all_file_metadata_remain_comparison_bound(self): + baseline = self.normalize(self.fixture()) + mutations = (lambda rows: rows[0]['files'][0].update(filename='src/renamed.py'), lambda rows: rows[0]['files'][0].update(status='added'), lambda rows: rows[0]['files'][0].update(provider_extension={'changed': True}), lambda rows: rows[0]['files'].append(_file('new.txt', 'added', 1)), lambda rows: rows[0]['files'].pop(), lambda rows: rows[0].update(commit={'message': 'changed message', 'author': {'name': 'Test Author', 'email': 'author@example.invalid'}}), lambda rows: rows[0]['parents'].__setitem__(0, {'sha': self.SHA_B, 'url': 'https://example.invalid/commit/' + self.SHA_B}), lambda rows: rows[0].update(sha='3333333333333333333333333333333333333333')) + for mutate in mutations: + with self.subTest(mutation=mutate): + changed = self.fixture() + mutate(changed) + self.assertNotEqual(baseline, self.normalize(changed)) + + def test_normalization_preserves_input_and_returns_independent_deep_copy(self): + rows = self.fixture() + before = json.dumps(rows, sort_keys=True, separators=(',', ':')) + normalized = self.normalize(rows) + self.assertEqual(before, json.dumps(rows, sort_keys=True, separators=(',', ':'))) + self.assertEqual(rows, self.fixture()) + normalized[0]['files'][0]['provider_extension']['nested'][1]['opaque'] = False + normalized[0]['provider_extension']['retained'].append('output-only') + self.assertTrue(rows[0]['files'][0]['provider_extension']['nested'][1]['opaque']) + self.assertNotIn('output-only', rows[0]['provider_extension']['retained']) + + def test_empty_file_list_is_valid_and_order_is_only_normalized_within_commits(self): + rows = self.fixture() + rows[0]['files'] = [] + result = self.normalize(rows) + self.assertEqual([], result[0]['files']) + self.assertEqual([self.SHA_A, self.SHA_B], [row['sha'] for row in result]) + self.assertEqual(sorted((file['filename'] for file in rows[1]['files'])), [file['filename'] for file in result[1]['files']]) + + def test_invalid_collection_commit_and_file_shapes_raise_value_error(self): + valid = self.fixture() + invalid_cases = [] + for value in (None, {}, 'rows', (valid,)): + invalid_cases.append(value) + invalid_cases.extend(([None], ['commit'], [{'sha': self.SHA_A}], [{'sha': self.SHA_A, 'files': None}], [{'sha': self.SHA_A, 'files': {}}])) + for bad in invalid_cases: + with self.subTest(rows=bad): + with self.assertRaises(ValueError): + self.normalize(bad) + for bad_sha in ('', 'A' * 40, 'a' * 39, 'g' * 40, None, 7): + with self.subTest(sha=bad_sha): + rows = self.fixture() + rows[0]['sha'] = bad_sha + with self.assertRaises(ValueError): + self.normalize(rows) + duplicate_sha = self.fixture() + duplicate_sha[1]['sha'] = duplicate_sha[0]['sha'] + with self.assertRaises(ValueError): + self.normalize(duplicate_sha) + for bad_file in (None, 'file', {}, {'filename': 'x'}): + with self.subTest(file=bad_file): + rows = self.fixture() + rows[0]['files'] = [bad_file] + with self.assertRaises(ValueError): + self.normalize(rows) + for key in ('filename', 'status'): + for value in (None, 1, ''): + with self.subTest(key=key, value=value): + rows = self.fixture() + rows[0]['files'][0][key] = value + with self.assertRaises(ValueError): + self.normalize(rows) + duplicate_file = self.fixture() + duplicate_file[0]['files'][1]['filename'] = duplicate_file[0]['files'][0]['filename'] + with self.assertRaises(ValueError): + self.normalize(duplicate_file) + + def test_minimal_file_record_and_literal_filename_are_retained(self): + rows = [{'sha': self.SHA_A, 'files': [{'filename': ' dir/../literal name.txt ', 'status': 'modified'}]}] + self.assertEqual(rows, self.normalize(rows)) diff --git a/harness/tests/impl/contracts/test_impl_architecture_wiring.py b/harness/tests/impl/contracts/test_impl_architecture_wiring.py new file mode 100644 index 0000000..2bf51a0 --- /dev/null +++ b/harness/tests/impl/contracts/test_impl_architecture_wiring.py @@ -0,0 +1,40 @@ +"""Canonical impl contracts stay reachable without duplicated skill bodies.""" +import re +import unittest +from pathlib import Path +from harness.tests._support.contracts import read_contract, REQUIRED +from harness.tests._support.paths import ROOT + +class ImplArchitectureDocsTests(unittest.TestCase): + + def test_skills_require_single_detailed_workflow_source(self): + for name in ('gitea-issue-implementation', 'gitea-issue-rework'): + path = ROOT / 'harness/agents/impl/skills' / name / 'SKILL.md' + text = path.read_text() + reference = path.parent / 'references/workflow.md' + with self.subTest(skill=name): + self.assertIn(REQUIRED, text) + self.assertLess(len(text.splitlines()), 100) + self.assertEqual(text.count('name: ' + name), 1) + self.assertNotIn('## Managed-lock broker gate', text) + self.assertFalse(reference.read_text().startswith('---')) + contract = read_contract(path) + self.assertEqual(contract.count('## Managed-lock broker gate'), 1) + for gate in ('## Delegation evidence acceptance gate', '## Test inventory source-binding gate', '## Canonical external verification evidence'): + self.assertEqual(contract.count(gate), 1) + + def test_all_canonical_impl_markdown_links_resolve(self): + for name in ('gitea-issue-implementation', 'gitea-issue-rework'): + directory = ROOT / 'harness/agents/impl/skills' / name + for path in [directory / 'SKILL.md', directory / 'references/workflow.md']: + for link in re.findall('\\[[^\\]]+\\]\\(([^)]+)\\)', path.read_text()): + if link.startswith(('https://', 'http://', '#')): + continue + target = link.split('#', 1)[0] + with self.subTest(path=path.name, link=link): + self.assertTrue((path.parent / target).resolve().exists()) + + def test_correction_default_and_exception_remain_distinct(self): + text = (ROOT / 'harness/agents/impl/skills/gitea-issue-rework/references/readback.md').read_text() + for token in ('atomic_client_precondition=false',): + self.assertIn(token.lower(), text.lower()) diff --git a/harness/tests/impl/delivery/__init__.py b/harness/tests/impl/delivery/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/harness/tests/impl/test_delivery_reconciliation.py b/harness/tests/impl/delivery/test_delivery_reconciliation.py similarity index 98% rename from harness/tests/impl/test_delivery_reconciliation.py rename to harness/tests/impl/delivery/test_delivery_reconciliation.py index faa6b26..77f4e0c 100644 --- a/harness/tests/impl/test_delivery_reconciliation.py +++ b/harness/tests/impl/delivery/test_delivery_reconciliation.py @@ -3,7 +3,9 @@ import importlib.util from pathlib import Path import unittest -PATH = Path(__file__).resolve().parents[3] / 'harness/agents/impl/skills/gitea-issue-implementation/scripts/implementation_delivery.py' +from harness.tests._support.paths import ROOT + +PATH = ROOT / 'harness/agents/impl/skills/gitea-issue-implementation/scripts/implementation_delivery.py' class ReadOnlyAdapter: diff --git a/harness/tests/impl/test_delivery_response.py b/harness/tests/impl/delivery/test_delivery_response.py similarity index 90% rename from harness/tests/impl/test_delivery_response.py rename to harness/tests/impl/delivery/test_delivery_response.py index 543e36c..cfac4ea 100644 --- a/harness/tests/impl/test_delivery_response.py +++ b/harness/tests/impl/delivery/test_delivery_response.py @@ -7,7 +7,8 @@ from http.server import BaseHTTPRequestHandler, HTTPServer from urllib.error import HTTPError from urllib.request import Request, urlopen -sys.path.insert(0, str(Path(__file__).resolve().parents[2] / 'agents/impl/skills/gitea-issue-implementation/scripts')) +from harness.tests._support.paths import ROOT +sys.path.insert(0, str(ROOT / 'harness/agents/impl/skills/gitea-issue-implementation/scripts')) import delivery_response as dr @@ -64,8 +65,11 @@ class DeliveryResponseTests(unittest.TestCase): def test_endpoint_specific_success_statuses(self): for method, status in [('GET', 200), ('POST', 201), ('PATCH', 201)]: - FakeHandler.status = status - self.assertEqual(self.request(method, status), {'ok': True}) + with self.subTest(method=method, status=status): + FakeHandler.status = status + calls_before = FakeHandler.calls + self.assertEqual(self.request(method, status), {'ok': True}) + self.assertEqual(FakeHandler.calls - calls_before, 1) def test_rejects_invalid_expected_status_before_send(self): calls = [] @@ -81,14 +85,6 @@ class DeliveryResponseTests(unittest.TestCase): self.request('POST', 201) self.assertEqual(FakeHandler.calls, 1) - def test_reconstructed_method_only_heuristic_rejects_valid_patch_ack(self): - # Reconstructed historical predicate, NOT a prior runtime RED run. - old_expected = 201 if 'PATCH' == 'POST' else 200 - self.assertNotEqual(201, old_expected) - FakeHandler.status = 201 - self.assertEqual(self.request('PATCH', 201), {'ok': True}) - self.assertEqual(FakeHandler.calls, 1) - def test_failure_status_cannot_be_configured_as_success(self): calls = [] for status in (100, 199, 300, 400, 500, 599): diff --git a/harness/tests/impl/test_impl_operation.py b/harness/tests/impl/delivery/test_impl_operation.py similarity index 99% rename from harness/tests/impl/test_impl_operation.py rename to harness/tests/impl/delivery/test_impl_operation.py index d1c15fb..30a9973 100644 --- a/harness/tests/impl/test_impl_operation.py +++ b/harness/tests/impl/delivery/test_impl_operation.py @@ -2,7 +2,7 @@ import sys import unittest from pathlib import Path -ROOT = Path(__file__).resolve().parents[3] +from harness.tests._support.paths import ROOT sys.path.insert(0, str(ROOT / "harness" / "lib")) import impl_operation as op diff --git a/harness/tests/impl/test_implementation_delivery.py b/harness/tests/impl/delivery/test_implementation_delivery.py similarity index 97% rename from harness/tests/impl/test_implementation_delivery.py rename to harness/tests/impl/delivery/test_implementation_delivery.py index 28c826e..48b25b3 100644 --- a/harness/tests/impl/test_implementation_delivery.py +++ b/harness/tests/impl/delivery/test_implementation_delivery.py @@ -4,9 +4,8 @@ from pathlib import Path from tempfile import TemporaryDirectory import unittest -ROOT = Path(__file__).resolve().parents[3] +from harness.tests._support.paths import ROOT PATH = ROOT / "harness/agents/impl/skills/gitea-issue-implementation/scripts/implementation_delivery.py" -INTEGRATION = ROOT / "harness/tests/common/test_delegation_consumers_integration.py" def load(path, name): @@ -51,7 +50,8 @@ class ImplementationDeliveryTests(unittest.TestCase): @classmethod def setUpClass(cls): cls.delivery = load(PATH, "implementation_delivery_under_test") - cls.evidence = load(INTEGRATION, "implementation_delivery_evidence") + from harness.tests._support import delegation + cls.evidence = delegation def setUp(self): self.tmp = TemporaryDirectory(dir="/opt/data/cache/scratch") diff --git a/harness/tests/impl/test_implementation_operation_integration.py b/harness/tests/impl/delivery/test_implementation_operation_integration.py similarity index 99% rename from harness/tests/impl/test_implementation_operation_integration.py rename to harness/tests/impl/delivery/test_implementation_operation_integration.py index fb9120b..098a716 100644 --- a/harness/tests/impl/test_implementation_operation_integration.py +++ b/harness/tests/impl/delivery/test_implementation_operation_integration.py @@ -4,7 +4,7 @@ import sys import unittest from pathlib import Path -ROOT = Path(__file__).resolve().parents[3] +from harness.tests._support.paths import ROOT SCRIPT = ROOT / "harness/agents/impl/skills/gitea-issue-implementation/scripts/implementation_delivery.py" sys.path.insert(0, str(ROOT)) spec = importlib.util.spec_from_file_location("implementation_delivery", SCRIPT) diff --git a/harness/tests/impl/evidence/__init__.py b/harness/tests/impl/evidence/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/harness/tests/impl/test_evidence_command_deadlines.py b/harness/tests/impl/evidence/test_evidence_command_deadlines.py similarity index 99% rename from harness/tests/impl/test_evidence_command_deadlines.py rename to harness/tests/impl/evidence/test_evidence_command_deadlines.py index a11c9d0..38d9d72 100644 --- a/harness/tests/impl/test_evidence_command_deadlines.py +++ b/harness/tests/impl/evidence/test_evidence_command_deadlines.py @@ -9,7 +9,9 @@ import time import unittest from pathlib import Path -SCRIPT = Path(__file__).resolve().parents[2] / "agents/impl/skills/gitea-issue-rework/scripts/evidence_manifest.py" +from harness.tests._support.paths import ROOT + +SCRIPT = ROOT / "harness/agents/impl/skills/gitea-issue-rework/scripts/evidence_manifest.py" SCRATCH = Path("/opt/data/cache/scratch/evidence-deadline-regressions") diff --git a/harness/tests/impl/evidence/test_evidence_manifest_filesystem_git.py b/harness/tests/impl/evidence/test_evidence_manifest_filesystem_git.py new file mode 100644 index 0000000..3f9b8b8 --- /dev/null +++ b/harness/tests/impl/evidence/test_evidence_manifest_filesystem_git.py @@ -0,0 +1,245 @@ +import hashlib +import json +import os +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path + +from harness.tests._support.paths import ROOT + +SCRIPT = ROOT / "harness/agents/impl/skills/gitea-issue-rework/scripts/evidence_manifest.py" + + +def run_git(cwd, *args): + return subprocess.run(["git", *args], cwd=cwd, text=True, capture_output=True, check=True) + +class ExternalEvidenceTests(unittest.TestCase): + + def setUp(self): + self.tmp = tempfile.TemporaryDirectory(dir=os.environ.get('TMPDIR', '/opt/data/cache/scratch')) + self.root = Path(self.tmp.name) + self.contract = self.root / 'harness/operations/repository-cache.md' + self.contract.parent.mkdir(parents=True) + self.contract.write_text('# Repository cache 운영 지침\n', encoding='utf-8') + self.repo = self.root / 'repositories/owner/project/worktrees/issue-1/checkout' + self.artifacts = self.repo.parent / 'verification/run-a' + self.repo.mkdir(parents=True) + self.artifacts.mkdir(parents=True) + for path in (self.root / 'repositories', *self.repo.parents): + if path == self.root: + break + path.chmod(448) + self.repo.chmod(448) + self.artifacts.parent.chmod(448) + self.artifacts.chmod(448) + run_git(self.repo, 'init', '-q') + run_git(self.repo, 'config', 'user.email', 'test@example.invalid') + run_git(self.repo, 'config', 'user.name', 'Test') + (self.repo / 'input.txt').write_text('hello\n') + run_git(self.repo, 'add', 'input.txt') + run_git(self.repo, 'commit', '-qm', 'initial') + self.plan = self.artifacts / 'evidence-plan.json' + self.plan.write_text(json.dumps({'hash_paths': ['input.txt'], 'commands': [{'id': 'check', 'argv': ['git', 'show', 'HEAD:input.txt'], 'timeout_seconds': 5, 'claims': ['output']}]})) + self.plan.chmod(384) + self.manifest = self.artifacts / 'evidence-manifest.json' + + def tearDown(self): + self.tmp.cleanup() + + def invoke(self, op, *args): + return subprocess.run([sys.executable, str(SCRIPT), op, '--contract-path', str(self.contract), '--repo-root', str(self.repo), '--artifact-root', str(self.artifacts), *map(str, args)], text=True, capture_output=True) + + def test_ignored_source_input_is_rejected_for_run_and_validate(self): + (self.repo / '.gitignore').write_text('ignored.txt\n') + run_git(self.repo, 'add', '.gitignore') + run_git(self.repo, 'commit', '-qm', 'ignore fixture') + ignored = self.repo / 'ignored.txt' + ignored.write_text('unbound input\n') + result = self.invoke('run', '--plan', self.plan.name, '--output', self.manifest.name) + self.assertEqual(result.returncode, 2, result.stderr) + self.assertIn('not clean', result.stderr) + self.assertFalse(self.manifest.exists()) + ignored.unlink() + self.assertEqual(self.invoke('run', '--plan', self.plan.name, '--output', self.manifest.name).returncode, 0) + prior = self.manifest.read_bytes() + ignored.write_text('unbound input again\n') + result = self.invoke('validate', '--manifest', self.manifest.name) + self.assertEqual(result.returncode, 2, result.stderr) + self.assertIn('not clean', result.stderr) + self.assertEqual(self.manifest.read_bytes(), prior) + + def test_canonical_control_directory_modes_match_installed_harness(self): + self.root.chmod(504) + self.contract.parent.parent.chmod(511) + result = self.invoke('run', '--plan', self.plan.name, '--output', self.manifest.name) + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(self.invoke('validate', '--manifest', self.manifest.name).returncode, 0) + self.assertEqual(self.root.stat().st_mode & 511, 504) + self.assertEqual(self.contract.parent.parent.stat().st_mode & 511, 511) + + def test_same_filename_in_source_and_artifacts_is_not_a_collision(self): + result = self.invoke('run', '--plan', self.plan.name, '--output', 'input.txt') + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual((self.repo / 'input.txt').read_text(), 'hello\n') + self.assertEqual(self.invoke('validate', '--manifest', 'input.txt').returncode, 0) + + def test_real_linked_detached_worktree_preserves_git_registration(self): + seed = self.root / 'seed' + self.repo.rename(seed) + bare = self.repo.parents[2] / 'repository.git' + subprocess.run(['git', 'clone', '--bare', str(seed), str(bare)], check=True, capture_output=True) + subprocess.run(['git', '--git-dir', str(bare), 'worktree', 'add', '--detach', str(self.repo), 'HEAD'], check=True, capture_output=True) + subprocess.run(['git', '--git-dir', str(bare), 'worktree', 'lock', str(self.repo)], check=True, capture_output=True) + before = run_git(self.repo, 'worktree', 'list', '--porcelain').stdout + result = self.invoke('run', '--plan', self.plan.name, '--output', self.manifest.name) + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(self.invoke('validate', '--manifest', self.manifest.name).returncode, 0) + self.assertEqual(run_git(self.repo, 'worktree', 'list', '--porcelain').stdout, before) + self.assertNotEqual(run_git(self.repo, 'rev-parse', '--git-common-dir').stdout.strip(), '.git') + self.assertEqual(run_git(self.repo, 'status', '--porcelain').stdout, '') + + def test_canonical_root_identity_and_traversal_denied_before_execution(self): + targets = (self.repo / 'docs/verification/run-a', self.root / 'scratch/run-a', self.repo.parent.parent / 'issue-2/verification/run-a', self.artifacts.parent / '../run-a') + for target in targets: + with self.subTest(target=str(target)): + result = self.invoke('run', '--artifact-root', target, '--plan', self.plan.name, '--output', self.manifest.name) + self.assertEqual(result.returncode, 2) + self.assertFalse(self.manifest.exists()) + for filename in ('../escape.json', '/absolute.json', 'a/../escape.json', './plan.json', 'plan.json/'): + with self.subTest(filename=filename): + result = self.invoke('run', '--plan', filename, '--output', self.manifest.name) + self.assertEqual(result.returncode, 2) + result = self.invoke('run', '--contract-path', self.root / 'wrong.md', '--plan', self.plan.name, '--output', self.manifest.name) + self.assertEqual(result.returncode, 2) + + def test_artifact_file_and_directory_symlinks_denied(self): + alias = self.artifacts / 'alias.json' + alias.symlink_to(self.plan.name) + result = self.invoke('run', '--plan', alias.name, '--output', self.manifest.name) + self.assertEqual(result.returncode, 2) + self.assertIn('unsafe path', result.stderr) + outside = self.root / 'outside' + self.artifacts.rename(outside) + self.artifacts.symlink_to(outside, target_is_directory=True) + result = self.invoke('run', '--plan', self.plan.name, '--output', self.manifest.name) + self.assertEqual(result.returncode, 2) + self.assertIn('symlink', result.stderr) + self.assertFalse((outside / self.manifest.name).exists()) + + def test_private_permissions_fail_closed_without_repair(self): + self.plan.chmod(420) + result = self.invoke('run', '--plan', self.plan.name, '--output', self.manifest.name) + self.assertEqual(result.returncode, 2) + self.assertEqual(self.plan.stat().st_mode & 511, 420) + self.plan.chmod(384) + self.artifacts.chmod(493) + result = self.invoke('run', '--plan', self.plan.name, '--output', self.manifest.name) + self.assertEqual(result.returncode, 2) + self.assertEqual(self.artifacts.stat().st_mode & 511, 493) + self.assertFalse(self.manifest.exists()) + + def test_wrong_uid_is_denied(self): + import importlib.util + from unittest.mock import patch + spec = importlib.util.spec_from_file_location('evidence_manifest_uid_test', SCRIPT) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + with patch.object(module.os, 'getuid', return_value=self.artifacts.stat().st_uid + 1): + with self.assertRaisesRegex(ValueError, 'ownership'): + module.safe_entry(self.artifacts, directory=True, mode=448) + + def test_group_writable_cache_parent_requires_exclusive_role_membership(self): + import importlib.util + from types import SimpleNamespace + from unittest.mock import patch + spec = importlib.util.spec_from_file_location('evidence_manifest_group_test', SCRIPT) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + parent = self.root / 'repositories' + parent.chmod(509) + uid = parent.stat().st_uid + gid = parent.stat().st_gid + role = SimpleNamespace(pw_name='role', pw_uid=uid, pw_gid=gid) + foreign = SimpleNamespace(pw_name='foreign', pw_uid=uid + 1, pw_gid=gid) + with patch.object(module.grp, 'getgrgid', return_value=SimpleNamespace(gr_mem=[])), patch.object(module.pwd, 'getpwall', return_value=[role]), patch.object(module.pwd, 'getpwnam', return_value=role): + module.safe_repositories(parent) + with patch.object(module.grp, 'getgrgid', return_value=SimpleNamespace(gr_mem=['foreign'])), patch.object(module.pwd, 'getpwall', return_value=[role, foreign]), patch.object(module.pwd, 'getpwnam', side_effect=lambda name: role if name == 'role' else foreign): + with self.assertRaisesRegex(ValueError, 'exclusive'): + module.safe_repositories(parent) + with patch.object(module.grp, 'getgrgid', side_effect=KeyError('unresolved')): + with self.assertRaisesRegex(ValueError, 'unverifiable'): + module.safe_repositories(parent) + self.assertEqual(parent.stat().st_mode & 511, 509) + + def test_no_clobber_rejects_rerun_before_command_execution(self): + self.assertEqual(self.invoke('run', '--plan', self.plan.name, '--output', self.manifest.name).returncode, 0) + prior = self.manifest.read_bytes() + obj = json.loads(self.plan.read_text()) + marker = self.root / 'rerun-executed' + obj['commands'][0]['argv'] = [sys.executable, '-c', 'from pathlib import Path; Path(' + repr(str(marker)) + ").write_text('ran')"] + self.plan.write_text(json.dumps(obj)) + result = self.invoke('run', '--plan', self.plan.name, '--output', self.manifest.name) + self.assertEqual(result.returncode, 2) + self.assertIn('already exists', result.stderr) + self.assertEqual(self.manifest.read_bytes(), prior) + self.assertFalse(marker.exists()) + + def test_v2_rejects_forged_issue_binding_and_v1_normal_validation(self): + self.assertEqual(self.invoke('run', '--plan', self.plan.name, '--output', self.manifest.name).returncode, 0) + original = json.loads(self.manifest.read_text()) + for replacement in (2, True, '1'): + obj = json.loads(json.dumps(original)) + obj['artifact']['issue'] = replacement + self.manifest.write_text(json.dumps(obj, sort_keys=True, separators=(',', ':')) + '\n') + self.assertEqual(self.invoke('validate', '--manifest', self.manifest.name).returncode, 2) + original['schema'] = 'evidence-manifest/v1' + self.manifest.write_text(json.dumps(original, sort_keys=True, separators=(',', ':')) + '\n') + self.assertEqual(self.invoke('validate', '--manifest', self.manifest.name).returncode, 2) + + def test_artifact_boundary_mutation_by_command_rejected_before_write(self): + obj = json.loads(self.plan.read_text()) + obj['commands'][0]['argv'] = [sys.executable, '-c', 'from pathlib import Path; Path(' + repr(str(self.artifacts)) + ').chmod(0o755)'] + self.plan.write_text(json.dumps(obj)) + result = self.invoke('run', '--plan', self.plan.name, '--output', self.manifest.name) + self.assertEqual(result.returncode, 2, result.stderr) + self.assertIn('permission', result.stderr) + self.assertFalse(self.manifest.exists()) + + def test_v2_stale_head_rejected_even_when_bound_file_is_unchanged(self): + self.assertEqual(self.invoke('run', '--plan', self.plan.name, '--output', self.manifest.name).returncode, 0) + (self.repo / 'other.txt').write_text('new source revision\n') + run_git(self.repo, 'add', 'other.txt') + run_git(self.repo, 'commit', '-qm', 'advance head') + result = self.invoke('validate', '--manifest', self.manifest.name) + self.assertEqual(result.returncode, 2, result.stderr) + self.assertIn('source revision is not current HEAD', result.stderr) + + def test_v2_rejects_source_symlink_alias(self): + (self.repo / 'alias.txt').symlink_to('input.txt') + run_git(self.repo, 'add', 'alias.txt') + run_git(self.repo, 'commit', '-qm', 'source symlink fixture') + obj = json.loads(self.plan.read_text()) + obj['hash_paths'] = ['alias.txt'] + self.plan.write_text(json.dumps(obj)) + result = self.invoke('run', '--plan', self.plan.name, '--output', self.manifest.name) + self.assertEqual(result.returncode, 2, result.stderr) + self.assertIn('symlink', result.stderr) + self.assertFalse(self.manifest.exists()) + + def test_external_artifacts_leave_checkout_and_head_unchanged(self): + head = run_git(self.repo, 'rev-parse', 'HEAD').stdout.strip() + result = self.invoke('run', '--plan', self.plan.name, '--output', self.manifest.name) + self.assertEqual(result.returncode, 0, result.stderr) + data = json.loads(self.manifest.read_text()) + self.assertEqual(data['schema'], 'evidence-manifest/v3') + self.assertEqual(data['source_revision'], head) + self.assertEqual(data['artifact']['run_id'], 'run-a') + self.assertEqual(data['artifact']['issue'], 1) + self.assertEqual(run_git(self.repo, 'status', '--porcelain').stdout, '') + self.assertEqual(run_git(self.repo, 'rev-parse', 'HEAD').stdout.strip(), head) + self.assertEqual(self.manifest.stat().st_mode & 511, 384) + self.assertEqual(self.invoke('validate', '--manifest', self.manifest.name).returncode, 0) diff --git a/harness/tests/impl/evidence/test_evidence_manifest_schema.py b/harness/tests/impl/evidence/test_evidence_manifest_schema.py new file mode 100644 index 0000000..91e9c32 --- /dev/null +++ b/harness/tests/impl/evidence/test_evidence_manifest_schema.py @@ -0,0 +1,206 @@ +import hashlib +import json +import os +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path + +from harness.tests._support.paths import ROOT +from harness.tests._support.evidence import EvidenceManifestFixture + +SCRIPT = ROOT / "harness/agents/impl/skills/gitea-issue-rework/scripts/evidence_manifest.py" + + +def run_git(cwd, *args): + return subprocess.run(["git", *args], cwd=cwd, text=True, capture_output=True, check=True) + +class EvidenceManifestTests(unittest.TestCase): + + def setUp(self): + self.fixture = EvidenceManifestFixture().setup() + self.addCleanup(self.fixture.cleanup) + self.tmp = self.fixture.tmp + self.root = self.fixture.root + self.contract = self.fixture.contract + self.repo = self.fixture.repo + self.artifacts = self.fixture.artifacts + + def invoke(self, *args, **kwargs): + return self.fixture.invoke(*args, **kwargs) + + def plan(self, *args, **kwargs): + return self.fixture.plan(*args, **kwargs) + + def run_manifest(self, *args, **kwargs): + return self.fixture.run_manifest(*args, **kwargs) + + def test_run_captures_success_claim_and_hash(self): + result, out = self.run_manifest() + self.assertEqual(result.returncode, 0, result.stderr) + manifest = json.loads(out.read_text()) + self.assertEqual(manifest['commands'][0]['argv'], ['git', 'show', 'HEAD:input.txt']) + self.assertEqual(manifest['commands'][0]['exit_code'], 0) + self.assertEqual(manifest['commands'][0]['claim_ids'], ['output']) + self.assertEqual(manifest['claims']['output'], ['/commands/0/argv', '/commands/0/exit_code', '/commands/0/expected_exit_code', '/commands/0/output', '/commands/0/started_at_utc', '/commands/0/finished_at_utc', '/commands/0/timeout_seconds', '/commands/0/execution_status', '/commands/0/termination']) + self.assertEqual(manifest['files'][0]['sha256'], hashlib.sha256(b'hello\n').hexdigest()) + self.assertEqual(self.invoke('validate', '--repo-root', self.repo, '--manifest', 'manifest.json').returncode, 0) + + def test_unexpected_exit_is_recorded_and_nonzero(self): + result, out = self.run_manifest(self.plan([{'argv': [sys.executable, '-c', "print('no'); raise SystemExit(3)"], 'expected_exit_code': 0, 'claims': ['result']}])) + self.assertNotEqual(result.returncode, 0) + self.assertEqual(json.loads(out.read_text())['commands'][0]['exit_code'], 3) + + def test_failed_run_preserves_prior_manifest(self): + result, out = self.run_manifest() + self.assertEqual(result.returncode, 0, result.stderr) + prior = out.read_bytes() + bad = self.plan([{'argv': [sys.executable, '-c', "print('Authorization: Bearer ***')"]}]) + result = self.invoke('run', '--repo-root', self.repo, '--plan', bad, '--output', out) + self.assertNotEqual(result.returncode, 0) + self.assertEqual(out.read_bytes(), prior) + + def test_command_environment_is_allowlisted_and_output_is_bounded(self): + env = os.environ.copy() + env['UNLISTED_SECRET'] = 'must-not-reach-child' + command = [{'argv': [sys.executable, '-c', "import os; print(os.getenv('UNLISTED_SECRET'))"], 'claims': ['environment']}] + plan = self.plan(command) + out = self.artifacts / 'manifest.json' + result = self.invoke('run', '--repo-root', self.repo, '--plan', plan.name, '--output', out.name, env=env) + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(json.loads(out.read_text())['commands'][0]['output'], 'None\n') + env['LEAK'] = 'SECRET_VALUE' + plan = self.plan(command, command_env_allowlist=['LEAK']) + result = self.invoke('run', '--repo-root', self.repo, '--plan', plan.name, '--output', out.name, env=env) + self.assertNotEqual(result.returncode, 0) + self.assertNotIn('SECRET_VALUE', out.read_text()) + prior = out.read_bytes() + plan = self.plan([{'argv': [sys.executable, '-c', "print('x' * 100000)"], 'max_output_bytes': 16, 'claims': ['bounded']}]) + result = self.invoke('run', '--repo-root', self.repo, '--plan', plan.name, '--output', 'overflow.json') + self.assertEqual(result.returncode, 1) + overflow = json.loads((self.artifacts / 'overflow.json').read_text()) + self.assertEqual(overflow['result'], 'failure') + self.assertEqual(overflow['commands'][0]['execution_status'], 'output_limit') + self.assertLessEqual(len(overflow['commands'][0]['output'].encode()), 16) + self.assertEqual(out.read_bytes(), prior) + + def test_output_collision_and_password_argument_are_rejected(self): + plan = self.plan() + result = self.invoke('run', '--repo-root', self.repo, '--plan', plan.name, '--output', '../checkout/input.txt') + self.assertNotEqual(result.returncode, 0) + self.assertEqual((self.repo / 'input.txt').read_text(), 'hello\n') + result, out = self.run_manifest(self.plan([{'argv': [sys.executable, '-c', "from pathlib import Path; Path('ran.txt').write_text('ran')", '--password', 'visible-value'], 'claims': ['password-leak']}])) + self.assertNotEqual(result.returncode, 0) + self.assertFalse(out.exists()) + self.assertFalse((self.repo / 'ran.txt').exists()) + + def test_argv_only_and_traversal_rejected(self): + result, out = self.run_manifest(self.plan([{'argv': ['echo', 'shell=true']}])) + self.assertNotEqual(result.returncode, 0) + self.assertFalse(out.exists()) + result, out = self.run_manifest(self.plan(hash_paths=['../outside'])) + self.assertNotEqual(result.returncode, 0) + self.assertFalse(out.exists()) + result, out = self.run_manifest(self.plan(hash_paths=['nested/../input.txt'])) + self.assertNotEqual(result.returncode, 0) + self.assertFalse(out.exists()) + + def test_empty_plan_and_malformed_command_record_are_rejected(self): + result, out = self.run_manifest(self.plan([])) + self.assertNotEqual(result.returncode, 0) + self.assertFalse(out.exists()) + result, out = self.run_manifest() + self.assertEqual(result.returncode, 0, result.stderr) + data = json.loads(out.read_text()) + data['commands'][0]['duration_seconds'] = -1 + out.write_text(json.dumps(data, sort_keys=True, separators=(',', ':')) + '\n') + self.assertNotEqual(self.invoke('validate', '--repo-root', self.repo, '--manifest', out.name).returncode, 0) + + def test_secret_or_header_leak_fails_closed(self): + env = os.environ.copy() + env['TEST_SECRET'] = 'never-write-this' + p = self.plan([{'argv': [sys.executable, '-c', "import os; print(os.environ['TEST_SECRET'])"]}], secret_env_names=['TEST_SECRET']) + result = self.invoke('run', '--repo-root', self.repo, '--plan', 'plan.json', '--output', 'manifest.json', env=env) + self.assertNotEqual(result.returncode, 0) + self.assertFalse((self.artifacts / 'manifest.json').exists()) + + def test_source_drift_hash_mismatch_rejected(self): + result, out = self.run_manifest() + self.assertEqual(result.returncode, 0) + (self.repo / 'input.txt').write_text('changed\n') + self.assertNotEqual(self.invoke('validate', '--repo-root', self.repo, '--manifest', out.name).returncode, 0) + + def test_command_source_mutation_is_rejected_before_manifest_write(self): + command = [{'argv': [sys.executable, '-c', "from pathlib import Path; Path('input.txt').write_text('changed\\n')"], 'claims': ['mutation']}] + result, out = self.run_manifest(self.plan(command)) + self.assertNotEqual(result.returncode, 0) + self.assertFalse(out.exists()) + + def test_command_commit_drift_is_rejected_before_manifest_write(self): + script = "from pathlib import Path; import subprocess; Path('other.txt').write_text('committed\\n'); subprocess.run(['git','add','other.txt'],check=True); subprocess.run(['git','commit','-qm','drift'],check=True)" + result, out = self.run_manifest(self.plan([{'argv': [sys.executable, '-c', script], 'claims': ['commit-drift']}])) + self.assertNotEqual(result.returncode, 0) + self.assertFalse(out.exists()) + + def test_binary_source_hash_is_byte_accurate(self): + payload = b'\xff\x00\x80binary\n' + (self.repo / 'binary.bin').write_bytes(payload) + run_git(self.repo, 'add', 'binary.bin') + run_git(self.repo, 'commit', '-qm', 'binary') + result, out = self.run_manifest(self.plan(hash_paths=['binary.bin'])) + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(json.loads(out.read_text())['files'][0]['sha256'], hashlib.sha256(payload).hexdigest()) + + def test_invalid_claim_pointer_rejected(self): + result, out = self.run_manifest(self.plan([{'argv': ['true'], 'claims': ['bad']}])) + self.assertEqual(result.returncode, 0) + data = json.loads(out.read_text()) + data['claims']['bad'] = '/missing' + out.write_text(json.dumps(data, sort_keys=True, separators=(',', ':')) + '\n') + self.assertNotEqual(self.invoke('validate', '--repo-root', self.repo, '--manifest', out.name).returncode, 0) + + def test_v2_validate_rejects_forged_plan_exclusion(self): + result, out = self.run_manifest() + self.assertEqual(result.returncode, 0, result.stderr) + (self.artifacts / 'plan.json').unlink() + (self.repo / 'dirty.txt').write_text('dirty\n') + data = json.loads(out.read_text()) + data['excluded_paths'] = ['dirty.txt', 'manifest.json'] + out.write_text(json.dumps(data, sort_keys=True, separators=(',', ':')) + '\n') + self.assertNotEqual(self.invoke('validate', '--repo-root', self.repo, '--manifest', out.name).returncode, 0) + + def test_validate_rejects_empty_or_shell_argv(self): + result, out = self.run_manifest() + self.assertEqual(result.returncode, 0, result.stderr) + data = json.loads(out.read_text()) + data['commands'][0]['argv'] = [] + out.write_text(json.dumps(data, sort_keys=True, separators=(',', ':')) + '\n') + self.assertNotEqual(self.invoke('validate', '--repo-root', self.repo, '--manifest', out.name).returncode, 0) + data['commands'][0]['argv'] = ['bash', '-c', 'true'] + out.write_text(json.dumps(data, sort_keys=True, separators=(',', ':')) + '\n') + self.assertNotEqual(self.invoke('validate', '--repo-root', self.repo, '--manifest', out.name).returncode, 0) + + def test_archived_v1_descendant_validates_read_only(self): + result, out = self.run_manifest() + self.assertEqual(result.returncode, 0) + obj = json.loads(out.read_text()) + obj['schema'] = 'evidence-manifest/v1' + del obj['artifact'] + obj['excluded_paths'] = ['plan.json', 'manifest.json'] + archived_plan = json.loads((self.artifacts / 'plan.json').read_text()) + archived_plan['commands'][0].pop('timeout_seconds', None) + (self.repo / 'plan.json').write_text(json.dumps(archived_plan)) + obj['plan']['sha256'] = hashlib.sha256((self.repo / 'plan.json').read_bytes()).hexdigest() + for key in ('timeout_seconds', 'execution_status', 'termination'): + obj['commands'][0].pop(key, None) + obj['claims'] = {key: pointers[:6] for key, pointers in obj['claims'].items()} + archived = self.repo / 'manifest.json' + archived.write_text(json.dumps(obj, sort_keys=True, separators=(',', ':'), ensure_ascii=False) + '\n') + run_git(self.repo, 'add', 'plan.json', 'manifest.json') + run_git(self.repo, 'commit', '-qm', 'historical evidence fixture') + before = archived.read_bytes() + archived_result = self.invoke('validate-legacy', '--repo-root', self.repo, '--manifest', 'manifest.json') + self.assertEqual(archived_result.returncode, 0, archived_result.stderr) + self.assertEqual(archived.read_bytes(), before) + self.assertEqual(run_git(self.repo, 'status', '--porcelain').stdout, '') diff --git a/harness/tests/impl/rework/__init__.py b/harness/tests/impl/rework/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/harness/tests/impl/rework/test_rework_comment.py b/harness/tests/impl/rework/test_rework_comment.py new file mode 100644 index 0000000..568ab5a --- /dev/null +++ b/harness/tests/impl/rework/test_rework_comment.py @@ -0,0 +1,259 @@ +import hashlib, json, subprocess, sys, tempfile, unittest +from pathlib import Path + +from harness.tests._support.paths import ROOT + +SCRIPT = ROOT / "harness/agents/impl/skills/gitea-issue-rework/scripts/rework_comment.py" + +class ReworkCommentTests(unittest.TestCase): + + def setUp(self): + self.t = tempfile.TemporaryDirectory() + self.d = Path(self.t.name) + self.data = {'owner': 'acme', 'repository': 'widget', 'issue': 7, 'pr': 12, 'review': 99, 'reviewed_head': 'a' * 40, 'final_head': 'b' * 40, 'contract_sha256': 'c' * 64, 'result': 'partial', 'findings': [{'id': 'F-1', 'status': 'resolved', 'classification': 'valid_fix', 'changes': 'src/main.py', 'verification': 'pytest -q (0), pass', 'reason': 'bounded fix'}], 'local_checks': 'pytest -q (0), pass', 'runtime_browser': '해당 없음·미검증', 'commit_status': 'success', 'actions_runs': '없음'} + self.write('input.json', self.data) + self.invoke('render', '--no-delegated-evidence', '--input', self.p('input.json'), '--comment', self.p('comment.md'), '--expected', self.p('expected.json')) + self.expected = json.loads(self.d.joinpath('expected.json').read_text()) + + def tearDown(self): + self.t.cleanup() + + def p(self, n): + return str(self.d / n) + + def write(self, n, x): + self.d.joinpath(n).write_text(json.dumps(x, ensure_ascii=False) if not isinstance(x, str) else x) + + def invoke(self, *args): + return subprocess.run([sys.executable, str(SCRIPT), *args], text=True, capture_output=True) + + def read(self, n): + return json.loads(self.d.joinpath(n).read_text()) + + def readback(self, comments=None, head=None): + self.write('pr.json', {'object_kind': 'pull_request', 'owner': 'acme', 'repository': 'widget', 'number': 12, 'state': 'open', 'head': {'sha': head or self.data['final_head']}, 'html_url': 'https://gitea/acme/widget/pulls/12', 'metadata_binding_verified': True}) + self.write('identity.json', {'id': 42, 'login': 'bot', 'role': 'gitea-impl-agent', 'authenticated': True, 'stable_account_verified': True}) + items = comments if comments is not None else [{'id': 501, 'html_url': 'https://gitea/x/501', 'created_at': '2026-01-01T00:00:00Z', 'updated_at': '2026-01-01T00:00:00Z', 'body': self.expected['body'], 'user': {'id': 42, 'login': 'bot'}}] + self.write('comments.json', {'object_kind': 'pull_request_top_level_comments', 'owner': 'acme', 'repository': 'widget', 'pr': 12, 'complete': True, 'endpoint_binding_verified': True, 'source_issue_unchanged': True, 'review_collection_unchanged': True, 'pr_metadata_unchanged': True, 'page_count': 1, 'total_count': len(items), 'provider_counts': {'top_level': len(items), 'reviews': 2, 'inline': 3, 'aggregate_comments': len(items), 'aggregate_components': ['top_level']}, 'comments': items}) + + def verify(self, action='create', baseline=0): + return self.invoke('verify', '--expected', self.p('expected.json'), '--pr-readback', self.p('pr.json'), '--comments-readback', self.p('comments.json'), '--identity-readback', self.p('identity.json'), '--action', action, '--baseline-count', str(baseline), '--output', self.p('receipt.json')) + + def test_render_digest_and_correlation(self): + self.assertEqual(hashlib.sha256(self.expected['marker_free_body'].encode()).hexdigest(), self.expected['result_sha256']) + parts = ['acme', 'widget', '7', '12', '99', 'a' * 40, 'b' * 40, 'c' * 64] + self.assertEqual(hashlib.sha256('\x00'.join(parts).encode()).hexdigest(), self.expected['correlation']) + self.assertTrue(self.d.joinpath('comment.md').read_text().endswith('-->\n')) + + def test_no_comment_and_marker_body_mismatch(self): + self.readback([]) + self.assertNotEqual(self.verify().returncode, 0) + self.readback([{'id': 1, 'body': self.expected['body'].replace('correlation=', 'correlation=x'), 'user': {'id': 42, 'login': 'bot'}}]) + self.assertNotEqual(self.verify().returncode, 0) + + def test_wrong_head_and_author(self): + self.readback(head='d' * 40) + self.assertNotEqual(self.verify().returncode, 0) + self.readback() + c = self.read('comments.json') + c['comments'][0]['user'] = {'id': 43, 'login': 'bot'} + self.write('comments.json', c) + self.assertNotEqual(self.verify().returncode, 0) + + def test_digest_and_correlation_tampering_duplicate(self): + self.expected['result_sha256'] = '0' * 64 + self.write('expected.json', self.expected) + self.readback() + self.assertNotEqual(self.verify().returncode, 0) + self.setUp_render_again() + self.expected['correlation'] = '0' * 64 + self.write('expected.json', self.expected) + self.readback() + self.assertNotEqual(self.verify().returncode, 0) + self.setUp_render_again() + self.readback() + c = self.read('comments.json') + c['comments'].append(dict(c['comments'][0], id=502)) + c['total_count'] = 2 + self.write('comments.json', c) + self.assertNotEqual(self.verify('reuse', 1).returncode, 0) + + def setUp_render_again(self): + self.invoke('render', '--no-delegated-evidence', '--input', self.p('input.json'), '--comment', self.p('comment.md'), '--expected', self.p('expected.json')) + self.expected = self.read('expected.json') + + def test_create_reuse_cardinality(self): + self.readback() + self.assertEqual(self.verify('create', 0).returncode, 0) + self.assertEqual(self.read('receipt.json')['status'], 'verified') + original = self.read('comments.json')['comments'][0] + self.readback([original, {'id': 2, 'html_url': 'u', 'created_at': 't', 'updated_at': 't', 'body': 'other', 'user': {'id': 42, 'login': 'bot'}}]) + self.assertEqual(self.verify('reuse', 2).returncode, 0) + + def test_exact_success_and_receipt_preserved(self): + self.readback() + self.assertEqual(self.verify().returncode, 0) + old = self.d.joinpath('receipt.json').read_bytes() + self.readback(head='e' * 40) + self.assertNotEqual(self.verify().returncode, 0) + self.assertEqual(old, self.d.joinpath('receipt.json').read_bytes()) + + def test_unresolved_delivery_and_incomplete_or_wrong_scope_readback_fail(self): + bad = dict(self.data) + bad['result'] = 'rework_delivered' + bad['findings'] = [dict(self.data['findings'][0], status='unresolved')] + self.write('bad-input.json', bad) + self.assertNotEqual(self.invoke('render', '--no-delegated-evidence', '--input', self.p('bad-input.json'), '--comment', self.p('bad.md'), '--expected', self.p('bad.json')).returncode, 0) + self.readback() + comments = self.read('comments.json') + comments['complete'] = False + self.write('comments.json', comments) + self.assertNotEqual(self.verify().returncode, 0) + self.readback() + pr = self.read('pr.json') + pr['repository'] = 'other' + self.write('pr.json', pr) + self.assertNotEqual(self.verify().returncode, 0) + + def test_impl_role_and_provider_aggregate_binding_are_required(self): + self.readback() + identity = self.read('identity.json') + identity['role'] = 'other' + self.write('identity.json', identity) + self.assertNotEqual(self.verify().returncode, 0) + self.readback() + comments = self.read('comments.json') + comments['provider_counts']['aggregate_comments'] += 1 + self.write('comments.json', comments) + self.assertNotEqual(self.verify().returncode, 0) + + def test_render_visible_identifier_lines_do_not_emit_issue_references(self): + body = self.expected['marker_free_body'] + self.assertIn('- Source Issue: 7', body) + self.assertIn('- Pull Request: 12', body) + self.assertIn('- Source Review: 99', body) + self.assertNotIn('- Source Issue: #7', body) + self.assertNotIn('- Pull Request: #12', body) + self.assertNotIn('- Source Review: #99', body) + + def test_render_rejects_reference_hazards_in_every_safe_human_field(self): + hazards = ('#123', 'otherrepo#123', '\\\\#123', '\\#123', 'review #99') + fields = ('changes', 'verification', 'reason', 'local_checks', 'runtime_browser') + for field in fields: + for snippet in hazards: + with self.subTest(field=field, snippet=snippet): + data = json.loads(json.dumps(self.data)) + if field in ('changes', 'verification', 'reason'): + data['findings'][0][field] = 'unsafe payload ' + snippet + else: + data[field] = 'unsafe payload ' + snippet + self.write('hazard.json', data) + result = self.invoke('render', '--no-delegated-evidence', '--input', self.p('hazard.json'), '--comment', self.p('hazard.md'), '--expected', self.p('hazard-expected.json')) + self.assertNotEqual(result.returncode, 0) + + def test_offline_verifier_rejects_reference_in_expected_body(self): + self.expected['marker_free_body'] += '\n- unsafe: otherrepo#123' + before_hash = self.expected['result_sha256'] + self.expected['result_sha256'] = hashlib.sha256(self.expected['marker_free_body'].encode()).hexdigest() + self.expected['marker'] = self.expected['marker'].replace(before_hash, self.expected['result_sha256']) + self.expected['body'] = self.expected['marker_free_body'] + self.expected['marker'] + self.write('expected.json', self.expected) + self.readback() + self.assertNotEqual(self.verify().returncode, 0) + + def _reseal(self, value, body): + value = json.loads(json.dumps(value)) + old = value['result_sha256'] + value['marker_free_body'] = body + value['result_sha256'] = hashlib.sha256(body.encode()).hexdigest() + value['marker'] = value['marker'].replace(old, value['result_sha256']) + value['body'] = body + value['marker'] + return value + + def _legacy_expected(self): + old = json.loads(json.dumps(self.expected)) + body = old['marker_free_body'] + body = body.replace('- Source Issue: #7', '- Source Issue: 7').replace('- Pull Request: #12', '- Pull Request: 12').replace('- Source Review: #99', '- Source Review: 99') + body = body.replace('- Source Issue: 7', '- Source Issue: #7').replace('- Pull Request: 12', '- Pull Request: #12').replace('- Source Review: 99', '- Source Review: #99') + old['marker_free_body'] = body + old['result_sha256'] = hashlib.sha256(body.encode()).hexdigest() + old['marker'] = old['marker'].replace(self.expected['result_sha256'], old['result_sha256']) + old['body'] = body + old['marker'] + return old + + def test_prepare_correction_returns_bounded_verified_legacy_migration_plan(self): + import importlib.util + spec = importlib.util.spec_from_file_location('rework_comment_correction_test', SCRIPT) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + self.assertTrue(callable(getattr(module, 'prepare_correction', None)), 'approved correction API missing') + plan = module.prepare_correction(self._legacy_expected(), self.expected, comment_id=501, actor_id=42) + self.assertEqual(plan['schema'], 'gitea-rework-correction-plan/v1') + self.assertEqual(plan['target'], self.expected['target']) + self.assertEqual(plan['comment_id'], 501) + self.assertEqual(plan['actor_id'], 42) + self.assertEqual(plan['old_body'], self._legacy_expected()['body']) + self.assertEqual(plan['new_body'], self.expected['body']) + self.assertEqual(plan['old_body_sha256'], hashlib.sha256(plan['old_body'].encode()).hexdigest()) + self.assertEqual(plan['new_body_sha256'], hashlib.sha256(plan['new_body'].encode()).hexdigest()) + self.assertIs(plan['prior_handoff_verified'], False) + self.assertTrue(plan['immutable_source_required']) + + def test_prepare_correction_rejects_tampered_or_unbounded_changes(self): + import importlib.util + spec = importlib.util.spec_from_file_location('rework_comment_correction_reject_test', SCRIPT) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + self.assertTrue(callable(getattr(module, 'prepare_correction', None)), 'approved correction API missing') + old = self._legacy_expected() + cases = [] + bad = json.loads(json.dumps(old)) + bad['result_sha256'] = '0' * 64 + cases.append((bad, self.expected)) + bad = json.loads(json.dumps(old)) + bad['body'] = bad['body'].replace('#7', '#8', 1) + cases.append((bad, self.expected)) + bad = json.loads(json.dumps(old)) + bad['marker_free_body'] = bad['marker_free_body'].replace('bounded fix', 'unsafe old text #123') + cases.append((bad, self.expected)) + bad = json.loads(json.dumps(self.expected)) + bad['marker_free_body'] = bad['marker_free_body'].replace('bounded fix', 'different extra text') + cases.append((old, bad)) + bad = self._reseal(self.expected, self.expected['marker_free_body'].replace('bounded fix', 'different extra text')) + cases.append((old, bad)) + unsafe_old = self._reseal(old, old['marker_free_body'].replace('bounded fix', 'same unsafe #123')) + unsafe_new = self._reseal(self.expected, self.expected['marker_free_body'].replace('bounded fix', 'same unsafe #123')) + cases.append((unsafe_old, unsafe_new)) + bad = json.loads(json.dumps(self.expected)) + bad['head'] = 'd' * 40 + cases.append((old, bad)) + bad = json.loads(json.dumps(self.expected)) + bad['correlation'] = 'e' * 64 + cases.append((old, bad)) + for previous, new in cases: + with self.subTest(previous=previous['result_sha256'], new=new['correlation']): + with self.assertRaises(module.Bad): + module.prepare_correction(previous, new, comment_id=501, actor_id=42) + + def test_prepare_correction_cli_emits_plan_not_approval_or_receipt(self): + self.write('previous.json', self._legacy_expected()) + r = self.invoke('prepare-correction', '--previous-expected', self.p('previous.json'), '--expected', self.p('expected.json'), '--comment-id', '501', '--actor-id', '42', '--output', self.p('plan.json')) + self.assertEqual(r.returncode, 0, r.stderr) + plan = self.read('plan.json') + self.assertEqual(plan['schema'], 'gitea-rework-correction-plan/v1') + self.assertIs(plan['approval_required'], True) + self.assertIs(plan['prior_handoff_verified'], False) + self.assertNotEqual(plan.get('status'), 'verified') + self.assertEqual(self.d.joinpath('plan.json').stat().st_mode & 511, 384) + + def test_prepare_correction_cli_refuses_input_alias_and_preserves_file(self): + self.write('previous.json', self._legacy_expected()) + before = self.d.joinpath('expected.json').read_bytes() + r = self.invoke('prepare-correction', '--previous-expected', self.p('previous.json'), '--expected', self.p('expected.json'), '--comment-id', '501', '--actor-id', '42', '--output', self.p('expected.json')) + self.assertNotEqual(r.returncode, 0) + self.assertEqual(self.d.joinpath('expected.json').read_bytes(), before) + + def test_different_body_same_correlation_is_not_default_reuse(self): + old = self._legacy_expected() + self.readback([{'id': 501, 'html_url': 'https://gitea/x/501', 'created_at': '2026-01-01T00:00:00Z', 'updated_at': '2026-01-01T00:00:00Z', 'body': old['body'], 'user': {'id': 42, 'login': 'bot'}}]) + self.assertNotEqual(self.verify('reuse', 1).returncode, 0) diff --git a/harness/tests/impl/rework/test_rework_correction.py b/harness/tests/impl/rework/test_rework_correction.py new file mode 100644 index 0000000..5f4bfbe --- /dev/null +++ b/harness/tests/impl/rework/test_rework_correction.py @@ -0,0 +1,136 @@ +"""Functional correction verification with provider-shaped synthetic readbacks.""" +import copy +import hashlib +import importlib.util +import json +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path +from harness.tests._support.rework import ReworkProviderFixture, ORIGIN, T1 + +from harness.tests._support.paths import ROOT + +DIRECTORY = ROOT / 'harness/agents/impl/skills/gitea-issue-rework/scripts' +spec = importlib.util.spec_from_file_location('correction_verifier', DIRECTORY/'rework_correction.py') +assert spec is not None and spec.loader is not None +mod = importlib.util.module_from_spec(spec) +spec.loader.exec_module(mod) + +class CorrectionVerifierTests(unittest.TestCase): + + def setUp(self): + fixture = ReworkProviderFixture().setup() + fixture.contract['version'] = '1.27.3' + fixture.overrides['/version'] = ({'version': '1.27.3'}, {}, 200) + self.addCleanup(fixture.cleanup) + self.new = copy.deepcopy(fixture.expected) + self.old = copy.deepcopy(self.new) + free = self.new['marker_free_body'] + for label in ('Source Issue', 'Pull Request', 'Source Review'): + free = free.replace('- ' + label + ': ', '- ' + label + ': #') + digest = hashlib.sha256(free.encode()).hexdigest() + marker = self.new['marker'].replace(self.new['result_sha256'], digest) + self.old.update(body=free + marker, marker=marker, marker_free_body=free, result_sha256=digest) + self.plan = fixture.legacy.prepare_correction(self.old, self.new, comment_id=501, actor_id=42) + fixture.add_result() + fixture.comments[0]['body'] = self.old['body'] + self.before = fixture.adapter().snapshot(self.new) + fixture.comments[0]['body'] = self.new['body'] + fixture.comments[0]['updated_at'] = '2026-01-03T00:00:00Z' + self.after = fixture.adapter().snapshot(self.new) + event = {'id': 601, 'type': 'comment_ref', 'user': {'id': 42, 'login': 'gitea-impl-agent'}, 'created_at': T1, 'updated_at': T1, 'ref_action': 'none', 'issue_url': ORIGIN + '/acme/demo/issues/7', 'ref_issue': {'number': 9, 'html_url': ORIGIN + '/acme/demo/pulls/9'}, 'ref_comment': copy.deepcopy(self.before['raw']['comments']['rows'][0])} + self.before['source_issue_timeline'] = self.timeline(event) + changed = copy.deepcopy(event) + changed.update(ref_action='neutered', updated_at='2026-01-03T00:00:00Z', ref_comment=copy.deepcopy(self.after['raw']['comments']['rows'][0])) + self.after['source_issue_timeline'] = self.timeline(changed) + self.approval = {'approved': True, 'operation': 'same_comment_correction', 'historical_failure_baseline_sha256': '1' * 64, 'old_expected_document': self.old, 'new_expected_document': self.new, 'timeline_profile': 'observed-comment-ref-neutralization', 'ordinary_patch': True, 'provider_version': self.before['provider']['version']} + + def timeline(self, event): + return {'complete': True, 'rows': [event], 'pages': [{'page': 1, 'count': 1, 'headers': {'X-Total-Count': '1'}}]} + + def verify(self, profile='observed-comment-ref-neutralization'): + return mod.verify_correction(self.plan, self.before, self.after, self.old['body'], self.new['body'], self.approval, approval_sha256=mod.sha(mod.canonical(self.approval)), historical_failure_sha256='1' * 64, profile=profile) + + def test_exact_approved_reference_neutralization_passes(self): + result = self.verify() + self.assertEqual(result['status'], 'verified') + self.assertEqual(result['changed_timeline_ids'], [601]) + self.assertEqual(result['historical_handoff_status'], 'partial_unchanged') + + def test_strict_unchanged_timeline_passes(self): + self.after['source_issue_timeline'] = copy.deepcopy(self.before['source_issue_timeline']) + self.approval['timeline_profile'] = 'strict' + self.assertEqual(self.verify('strict')['status'], 'verified') + + def test_reference_profile_requires_separate_approval(self): + self.approval['timeline_profile'] = 'strict' + with self.assertRaises(mod.Bad): + self.verify() + + def test_source_and_cache_deltas_fail(self): + for name, field in [('source_issue', 'title'), ('pr', 'comments'), ('pr_issue', 'comments')]: + original = copy.deepcopy(self.after) + self.after['raw'][name][field] = 99 if field == 'comments' else 'changed' + with self.subTest(name=name), self.assertRaises(mod.Bad): + self.verify() + self.after = original + + def test_unrelated_timeline_field_change_fails(self): + self.after['source_issue_timeline']['rows'][0]['body'] = 'unexpected' + with self.assertRaises(mod.Bad): + self.verify() + + def test_unrelated_null_timeline_field_is_still_a_mutation(self): + self.after['source_issue_timeline']['rows'][0]['unexpected_field'] = None + with self.assertRaises(mod.Bad): + self.verify() + + def test_duplicate_and_missing_timeline_rows_fail(self): + original = copy.deepcopy(self.after) + for rows in [[], self.after['source_issue_timeline']['rows'] * 2]: + self.after = copy.deepcopy(original) + self.after['source_issue_timeline']['rows'] = rows + with self.subTest(rows=len(rows)), self.assertRaises(mod.Bad): + self.verify() + + def test_wrong_actor_head_and_extra_body_change_fail(self): + for field, value in [('actor_id', 99), ('head', 'd' * 40), ('new_body', 'arbitrary')]: + original = copy.deepcopy(self.plan) + self.plan[field] = value + with self.subTest(field=field), self.assertRaises(mod.Bad): + self.verify() + self.plan = original + + def test_reference_timestamp_cannot_move_backwards(self): + self.after['source_issue_timeline']['rows'][0]['updated_at'] = '2025-01-01T00:00:00Z' + with self.assertRaises(mod.Bad): + self.verify() + + def test_real_cli_valid_inputs_and_no_overwrite(self): + with tempfile.TemporaryDirectory() as td: + root = Path(td) + args = [sys.executable, '-B', str(DIRECTORY / 'rework_correction.py'), 'verify'] + values = {'plan': self.plan, 'before': self.before, 'after': self.after, 'old-comment': {'body': self.old['body']}, 'new-comment': {'body': self.new['body']}, 'approval': self.approval} + for name, value in values.items(): + raw = json.dumps(value).encode() + path = root / (name + '.json') + path.write_bytes(raw) + path.chmod(384) + args.extend(['--' + name, str(path), '--' + name + '-sha256', hashlib.sha256(raw).hexdigest()]) + output = root / 'receipt.json' + args.extend(['--historical-failure-sha256', '1' * 64, '--profile', 'observed-comment-ref-neutralization', '--output', str(output)]) + result = subprocess.run(args, capture_output=True, text=True, timeout=10) + self.assertEqual(result.returncode, 0, result.stderr) + saved = output.read_bytes() + again = subprocess.run(args, capture_output=True, text=True, timeout=10) + self.assertNotEqual(again.returncode, 0) + self.assertEqual(output.read_bytes(), saved) + + def test_digest_tampering_is_rejected(self): + with tempfile.TemporaryDirectory() as td: + path = Path(td) / 'data.json' + path.write_text('{}') + with self.assertRaises(mod.Bad): + mod.load(path, '0' * 64) diff --git a/harness/tests/impl/rework/test_rework_input_gate_cli.py b/harness/tests/impl/rework/test_rework_input_gate_cli.py new file mode 100644 index 0000000..ad4c63d --- /dev/null +++ b/harness/tests/impl/rework/test_rework_input_gate_cli.py @@ -0,0 +1,214 @@ +import importlib.util +import os +import tempfile +import unittest +from pathlib import Path +from harness.tests._support.paths import ROOT + +SCRIPT = ROOT / 'harness/agents/impl/skills/gitea-issue-rework/scripts/rework_input_gate.py' + +class OrderedDecisionCliTests(unittest.TestCase): + + def setUp(self): + import copy, hashlib + from harness.tests._support.rework import ReworkProviderFixture + from harness.tests._support import spec as spec_tests + self.rb = ReworkProviderFixture().setup() + self.addCleanup(self.rb.cleanup) + self.rb.repo['default_branch'] = 'main' + self.rb.source['body'] = '\nSynthetic approved source contract.\n' + self.inp = {'schema': 'gitea-agent.rework-input/v1', 'target': {'owner': 'acme', 'repository': 'demo', 'issue': 7, 'pr': 9}, 'head': 'b' * 40, 'contract_sha256': hashlib.sha256(self.rb.source['body'].encode()).hexdigest()} + self.rows = [] + for role, rid, uid in [('gitea-spec-agent', 301, 33), ('gitea-review-agent', 302, 34)]: + row = copy.deepcopy(self.rb.reviews[0]) + row.update(id=rid, commit_id=self.inp['head'], state='APPROVED', user={'id': uid, 'login': role}, stale=False, dismissed=False, html_url=self.rb.pr['html_url'] + '#pullreview-' + str(rid)) + if role == 'gitea-spec-agent': + result = spec_tests.result() + result['target'] = dict(self.inp['target']) + result['target']['pull'] = result['target'].pop('pr') + result['target']['head'] = self.inp['head'] + result['contract_sha256'] = self.inp['contract_sha256'] + row['body'] = spec_tests.mod.render_review(result, {'src/a.py', 'tests/test_a.py'}, declaration={'mode': 'none'}) + else: + row['body'] = self.code_body('approve') + self.rows.append(row) + self.set_rows(self.rows) + original = self.rb.request + + def request(path): + from urllib.parse import urlsplit + import re + if re.fullmatch('/repos/acme/demo/pulls/9/reviews/[1-9][0-9]*/comments', urlsplit(path).path): + self.rb.calls.append(path) + return ([], {'X-Total-Count': '0'}, 200) + return original(path) + self.rb.request = request + + def code_body(self, verdict): + return '\n- Verdict: `' + verdict + '`\nBounded native code-review fixture.\n' + + def set_rows(self, rows): + import copy + self.rb.reviews = copy.deepcopy(rows) + self.rb.pr['comments'] = self.rb.underlying['comments'] = self.rb.pr['review_comments'] = len(rows) + + def snapshot(self): + return self.rb.adapter().input_snapshot(self.inp) + + def spec_body(self, verdict, head=None): + from harness.tests._support import spec as spec_tests + result = spec_tests.result(verdict) + result['target'] = {'owner': 'acme', 'repository': 'demo', 'issue': 7, 'pull': 9, 'head': head or self.inp['head']} + result['contract_sha256'] = self.inp['contract_sha256'] + if verdict == 'request_changes': + result['criteria'][0].update(result='unsatisfied', reason='Bounded regression finding.') + result['conclusion'] = 'Bounded correction required.' + return spec_tests.mod.render_review(result, {'src/a.py', 'tests/test_a.py'}, declaration={'mode': 'none'}) + + def test_legacy_output_budget_is_unchanged(self): + cli_path = SCRIPT.with_name('rework_readback_cli.py') + spec = importlib.util.spec_from_file_location('legacy_budget_probe', cli_path) + cli = importlib.util.module_from_spec(spec) + spec.loader.exec_module(cli) + with tempfile.TemporaryDirectory() as td: + output = Path(td) / 'legacy.json' + try: + with cli.OutputFile(output) as writer: + writer.write({'data': 'x' * 1000001}) + except ValueError as exc: + self.fail('legacy output below original 16MB budget must remain accepted: ' + str(exc)) + self.assertGreater(output.stat().st_size, 1000000) + + def test_real_public_cli_runs_gate_without_selected_review(self): + import hashlib, json, sys + from unittest.mock import patch as mock + + def load(name, path): + spec = importlib.util.spec_from_file_location(name, path) + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + return mod + cli = load('gate_cli_test', SCRIPT.with_name('rework_readback_cli.py')) + command = load('gate_command_test', SCRIPT.with_name('rework_comment.py')) + with tempfile.TemporaryDirectory() as td: + root = Path(td) + target = root / 'target.json' + provider = root / 'provider.json' + output = root / 'decision.json' + target.write_text(json.dumps(self.inp)) + provider.write_text(json.dumps(self.rb.contract)) + target.chmod(384) + provider.chmod(384) + args = ['rework_comment.py', 'input-gate', '--target', str(target), '--target-sha256', hashlib.sha256(target.read_bytes()).hexdigest(), '--provider-contract', str(provider), '--provider-contract-sha256', hashlib.sha256(provider.read_bytes()).hexdigest(), '--actor-id', '42', '--output', str(output)] + with mock.dict(sys.modules, {'rework_readback_cli': cli}), mock.object(sys, 'argv', args), mock.dict(os.environ, {'HERMES_HOME': td}), mock.object(cli, 'load_role_token', return_value='PUBLIC' + '-fixture-value') as loader, mock.object(cli, 'RestTransport', return_value=self.rb.request): + self.assertEqual(command.main(), 0) + self.assertTrue(loader.called) + self.assertEqual(json.loads(output.read_text())['status'], 'no_op') + self.assertEqual(output.stat().st_mode & 511, 384) + import copy + rows = copy.deepcopy(self.rows) + rows[1].update(state='REQUEST_CHANGES', body=self.code_body('request_changes')) + self.set_rows(list(reversed(rows))) + output = root / 'rework-decision.json' + args[-1] = str(output) + with mock.dict(sys.modules, {'rework_readback_cli': cli}), mock.object(sys, 'argv', args), mock.dict(os.environ, {'HERMES_HOME': td}), mock.object(cli, 'load_role_token', return_value='PUBLIC' + '-fixture-value'), mock.object(cli, 'RestTransport', return_value=self.rb.request): + self.assertEqual(command.main(), 0) + actual = json.loads(output.read_text()) + self.assertEqual(actual['status'], 'rework') + self.assertEqual(actual['decision']['rework_review_ids'], [302]) + saved = output.read_bytes() + with mock.dict(sys.modules, {'rework_readback_cli': cli}), mock.object(sys, 'argv', args), mock.object(cli, 'load_role_token') as loader, mock.object(cli, 'RestTransport') as transport: + self.assertEqual(command.main(), 2) + loader.assert_not_called() + transport.assert_not_called() + self.assertEqual(output.read_bytes(), saved) + + def test_initial_output_atomic_publish_preserves_race_created_file(self): + from unittest.mock import patch as mock + spec = importlib.util.spec_from_file_location('no_clobber_test', SCRIPT.with_name('rework_readback_cli.py')) + cli = importlib.util.module_from_spec(spec) + spec.loader.exec_module(cli) + with tempfile.TemporaryDirectory() as td: + output = Path(td) / 'decision.json' + original_link = cli.os.link + with cli.OutputFile(output, initial_only=True) as writer: + + def race_link(*args, **kwargs): + output.write_bytes(b'raced output') + output.chmod(384) + return original_link(*args, **kwargs) + with mock.object(cli.os, 'link', side_effect=race_link), self.assertRaisesRegex(ValueError, 'created during capture'): + writer.write({'decision': 'bounded'}) + self.assertEqual(output.read_bytes(), b'raced output') + self.assertEqual(output.stat().st_mode & 511, 384) + self.assertEqual([x.name for x in Path(td).iterdir()], ['decision.json']) + + def test_initial_output_total_byte_budget_rejects_without_artifact(self): + spec = importlib.util.spec_from_file_location('size_gate_test', SCRIPT.with_name('rework_readback_cli.py')) + cli = importlib.util.module_from_spec(spec) + spec.loader.exec_module(cli) + with tempfile.TemporaryDirectory() as td: + output = Path(td) / 'decision.json' + with cli.OutputFile(output, initial_only=True) as writer, self.assertRaisesRegex(ValueError, 'output size'): + writer.write({'raw': 'x' * 1000001}) + self.assertFalse(output.exists()) + + def test_cli_preflight_rejects_bool_target_before_credentials(self): + import hashlib, json, sys, copy + from unittest.mock import patch as mock + spec = importlib.util.spec_from_file_location('cli_preflight_test', SCRIPT.with_name('rework_readback_cli.py')) + cli = importlib.util.module_from_spec(spec) + spec.loader.exec_module(cli) + spec = importlib.util.spec_from_file_location('cmd_preflight_test', SCRIPT.with_name('rework_comment.py')) + cmd = importlib.util.module_from_spec(spec) + spec.loader.exec_module(cmd) + with tempfile.TemporaryDirectory() as td: + root = Path(td) + target = root / 'target.json' + provider = root / 'provider.json' + output = root / 'decision.json' + bad = copy.deepcopy(self.inp) + bad['target']['issue'] = True + target.write_text(json.dumps(bad)) + provider.write_text(json.dumps(self.rb.contract)) + target.chmod(384) + provider.chmod(384) + args = ['command', 'input-gate', '--target', str(target), '--target-sha256', hashlib.sha256(target.read_bytes()).hexdigest(), '--provider-contract', str(provider), '--provider-contract-sha256', hashlib.sha256(provider.read_bytes()).hexdigest(), '--actor-id', '42', '--output', str(output)] + with mock.dict(sys.modules, {'rework_readback_cli': cli}), mock.object(sys, 'argv', args), mock.object(cli, 'load_role_token') as loader, mock.object(cli, 'RestTransport') as transport: + self.assertEqual(cmd.main(), 2) + loader.assert_not_called() + transport.assert_not_called() + self.assertFalse(output.exists()) + + def test_cli_capture_race_preserves_foreign_output(self): + import hashlib, json, sys + from unittest.mock import patch as mock + spec = importlib.util.spec_from_file_location('cli_race_test', SCRIPT.with_name('rework_readback_cli.py')) + cli = importlib.util.module_from_spec(spec) + spec.loader.exec_module(cli) + spec = importlib.util.spec_from_file_location('cmd_race_test', SCRIPT.with_name('rework_comment.py')) + cmd = importlib.util.module_from_spec(spec) + spec.loader.exec_module(cmd) + with tempfile.TemporaryDirectory() as td: + root = Path(td) + target = root / 'target.json' + provider = root / 'provider.json' + output = root / 'decision.json' + target.write_text(json.dumps(self.inp)) + provider.write_text(json.dumps(self.rb.contract)) + target.chmod(384) + provider.chmod(384) + args = ['command', 'input-gate', '--target', str(target), '--target-sha256', hashlib.sha256(target.read_bytes()).hexdigest(), '--provider-contract', str(provider), '--provider-contract-sha256', hashlib.sha256(provider.read_bytes()).hexdigest(), '--actor-id', '42', '--output', str(output)] + requests = [] + + def transport(path): + requests.append(path) + if len(requests) == 1: + output.write_bytes(b'created concurrently') + output.chmod(384) + return self.rb.request(path) + with mock.dict(sys.modules, {'rework_readback_cli': cli}), mock.object(sys, 'argv', args), mock.dict(os.environ, {'HERMES_HOME': td}), mock.object(cli, 'load_role_token', return_value='PUBLIC' + '-fixture-value'), mock.object(cli, 'RestTransport', return_value=transport): + self.assertEqual(cmd.main(), 2) + self.assertGreater(len(requests), 1) + self.assertEqual(output.read_bytes(), b'created concurrently') + self.assertEqual(output.stat().st_mode & 511, 384) diff --git a/harness/tests/impl/rework/test_rework_input_gate_decision.py b/harness/tests/impl/rework/test_rework_input_gate_decision.py new file mode 100644 index 0000000..59f9b89 --- /dev/null +++ b/harness/tests/impl/rework/test_rework_input_gate_decision.py @@ -0,0 +1,380 @@ +import importlib.util +import os +import tempfile +import unittest +from pathlib import Path +from harness.tests._support.paths import ROOT + +SCRIPT = ROOT / 'harness/agents/impl/skills/gitea-issue-rework/scripts/rework_input_gate.py' + +class OrderedDecisionTests(unittest.TestCase): + + def setUp(self): + import copy, hashlib + from harness.tests._support.rework import ReworkProviderFixture + from harness.tests._support import spec as spec_tests + self.rb = ReworkProviderFixture().setup() + self.addCleanup(self.rb.cleanup) + self.rb.repo['default_branch'] = 'main' + self.rb.source['body'] = '\nSynthetic approved source contract.\n' + self.inp = {'schema': 'gitea-agent.rework-input/v1', 'target': {'owner': 'acme', 'repository': 'demo', 'issue': 7, 'pr': 9}, 'head': 'b' * 40, 'contract_sha256': hashlib.sha256(self.rb.source['body'].encode()).hexdigest()} + self.rows = [] + for role, rid, uid in [('gitea-spec-agent', 301, 33), ('gitea-review-agent', 302, 34)]: + row = copy.deepcopy(self.rb.reviews[0]) + row.update(id=rid, commit_id=self.inp['head'], state='APPROVED', user={'id': uid, 'login': role}, stale=False, dismissed=False, html_url=self.rb.pr['html_url'] + '#pullreview-' + str(rid)) + if role == 'gitea-spec-agent': + result = spec_tests.result() + result['target'] = dict(self.inp['target']) + result['target']['pull'] = result['target'].pop('pr') + result['target']['head'] = self.inp['head'] + result['contract_sha256'] = self.inp['contract_sha256'] + row['body'] = spec_tests.mod.render_review(result, {'src/a.py', 'tests/test_a.py'}, declaration={'mode': 'none'}) + else: + row['body'] = self.code_body('approve') + self.rows.append(row) + self.set_rows(self.rows) + original = self.rb.request + + def request(path): + from urllib.parse import urlsplit + import re + if re.fullmatch('/repos/acme/demo/pulls/9/reviews/[1-9][0-9]*/comments', urlsplit(path).path): + self.rb.calls.append(path) + return ([], {'X-Total-Count': '0'}, 200) + return original(path) + self.rb.request = request + + def code_body(self, verdict): + return '\n- Verdict: `' + verdict + '`\nBounded native code-review fixture.\n' + + def set_rows(self, rows): + import copy + self.rb.reviews = copy.deepcopy(rows) + self.rb.pr['comments'] = self.rb.underlying['comments'] = self.rb.pr['review_comments'] = len(rows) + + def snapshot(self): + return self.rb.adapter().input_snapshot(self.inp) + + def spec_body(self, verdict, head=None): + from harness.tests._support import spec as spec_tests + result = spec_tests.result(verdict) + result['target'] = {'owner': 'acme', 'repository': 'demo', 'issue': 7, 'pull': 9, 'head': head or self.inp['head']} + result['contract_sha256'] = self.inp['contract_sha256'] + if verdict == 'request_changes': + result['criteria'][0].update(result='unsatisfied', reason='Bounded regression finding.') + result['conclusion'] = 'Bounded correction required.' + return spec_tests.mod.render_review(result, {'src/a.py', 'tests/test_a.py'}, declaration={'mode': 'none'}) + + def test_fresh_source_contract_digest_is_enforced(self): + self.assertEqual(self.snapshot()['status'], 'no_op') + self.rb.source['body'] += 'Unexpected contract mutation.\n' + with self.assertRaisesRegex(ValueError, 'contract|digest'): + self.snapshot() + + def test_native_state_marker_contradiction_is_blocked(self): + import copy + rows = copy.deepcopy(self.rows) + code = next((row for row in rows if row['user']['login'] == 'gitea-review-agent')) + code['body'] = self.code_body('request_changes') + code['state'] = 'APPROVED' + self.set_rows(rows) + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'blocked') + self.assertIn('state_marker_contradiction', receipt['reason_codes']) + + def test_mixed_roles_permutations_require_rework_without_raw_mutation(self): + import copy + rows = copy.deepcopy(self.rows) + rows[1].update(state='REQUEST_CHANGES', body=self.code_body('request_changes')) + for ordered in (rows, list(reversed(rows))): + self.set_rows(ordered) + before = copy.deepcopy(self.rb.reviews) + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'rework', receipt['reason_codes']) + self.assertEqual(receipt['decision']['selected_review_ids'], {'gitea-spec-agent': 301, 'gitea-review-agent': 302}) + self.assertEqual(self.rb.reviews, before) + self.assertEqual(receipt['raw_input_snapshot']['reviews']['rows'], before) + self.assertEqual([x['id'] for x in receipt['decision']['timeline']], [301, 302]) + + def test_only_same_role_later_approval_supersedes_request(self): + import copy + rows = copy.deepcopy(self.rows) + rows[1].update(state='REQUEST_CHANGES', body=self.code_body('request_changes')) + self.set_rows(rows) + self.assertEqual(self.snapshot()['status'], 'rework') + later = copy.deepcopy(self.rows[1]) + later.update(id=303, submitted_at='2026-01-02T00:00:00Z', updated_at='2026-01-02T00:00:00Z', html_url=self.rb.pr['html_url'] + '#pullreview-303') + self.set_rows([later, *rows]) + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'no_op', receipt['reason_codes']) + self.assertEqual(receipt['decision']['selected_review_ids']['gitea-review-agent'], 303) + + def test_missing_stale_and_dismissed_approvals_are_not_noop_eligible(self): + import copy + self.assertEqual(self.snapshot()['status'], 'no_op') + for rows in ([self.rows[0]], [self.rows[1]], []): + self.set_rows(rows) + self.assertEqual(self.snapshot()['status'], 'blocked') + for field in ('stale', 'dismissed'): + rows = copy.deepcopy(self.rows) + rows[1][field] = True + self.set_rows(rows) + self.assertEqual(self.snapshot()['status'], 'blocked') + + def test_current_pending_and_manual_comments_never_allow_noop(self): + import copy + self.assertEqual(self.snapshot()['status'], 'no_op') + for state in ('PENDING', 'COMMENT'): + rows = copy.deepcopy(self.rows) + rows[1]['state'] = state + self.set_rows(rows) + self.assertEqual(self.snapshot()['status'], 'blocked') + + def test_blocked_spec_comment_dominates_code_request_changes(self): + import copy + from harness.tests._support import spec as spec_tests + result = spec_tests.result('blocked_spec') + result['target'] = {'owner': 'acme', 'repository': 'demo', 'issue': 7, 'pull': 9, 'head': self.inp['head']} + result['contract_sha256'] = self.inp['contract_sha256'] + result['criteria'][0]['result'] = 'indeterminate' + result['scope']['spec_conflicts'] = ['src/a.py'] + rows = copy.deepcopy(self.rows) + rows[0].update(state='COMMENT', body=spec_tests.mod.render_review(result, {'src/a.py', 'tests/test_a.py'}, declaration={'mode': 'none'})) + rows[1].update(state='REQUEST_CHANGES', body=self.code_body('request_changes')) + self.set_rows(rows) + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'blocked') + self.assertIn('blocked_spec', receipt['reason_codes']) + + def test_marker_contract_and_target_defects_block_rework(self): + import copy + self.assertEqual(self.snapshot()['status'], 'no_op') + for body in ('missing marker', self.code_body('approve').replace('issue=7', 'issue=8'), self.code_body('approve').replace(self.inp['contract_sha256'], 'f' * 64)): + rows = copy.deepcopy(self.rows) + rows[1]['body'] = body + self.set_rows(rows) + self.assertEqual(self.snapshot()['status'], 'blocked') + + def test_unknown_role_cannot_replace_authoritative_code_review(self): + import copy + self.assertEqual(self.snapshot()['status'], 'no_op') + unknown = copy.deepcopy(self.rows[1]) + unknown['user'] = {'id': 99, 'login': 'reader'} + self.set_rows([self.rows[0], unknown]) + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'blocked') + self.assertIn('unknown_role', receipt['decision']['timeline'][1]['reason_codes']) + + def test_native_rows_source_marker_and_default_branch_fail_closed(self): + import copy + self.assertEqual(self.snapshot()['status'], 'no_op') + for field, bad in [('id', True), ('state', 'UNRECOGNIZED'), ('stale', 'false'), ('dismissed', None), ('submitted_at', 'not-a-timestamp')]: + rows = copy.deepcopy(self.rows) + rows[1][field] = bad + self.set_rows(rows) + with self.subTest(field=field), self.assertRaises(ValueError): + self.snapshot() + self.set_rows(self.rows) + original = self.rb.source['body'] + self.rb.source['body'] = original.replace(' -->', ' --> suffix', 1) + import hashlib + self.inp['contract_sha256'] = hashlib.sha256(self.rb.source['body'].encode()).hexdigest() + with self.assertRaisesRegex(ValueError, 'discovery'): + self.snapshot() + self.rb.source['body'] = original + self.inp['contract_sha256'] = hashlib.sha256(original.encode()).hexdigest() + self.rb.pr['base']['ref'] = 'other' + with self.assertRaisesRegex(ValueError, 'default branch'): + self.snapshot() + + def test_duplicate_review_ids_and_capture_drift_fail_closed(self): + import copy + self.assertEqual(self.snapshot()['status'], 'no_op') + self.set_rows([*self.rows, copy.deepcopy(self.rows[1])]) + with self.assertRaisesRegex(ValueError, 'duplicate'): + self.snapshot() + self.set_rows(self.rows) + old = self.rb.request + users = [0] + + def request(path): + if path == '/user': + users[0] += 1 + if users[0] == 2: + self.rb.source['title'] = 'Changed during second capture' + return old(path) + self.rb.request = request + with self.assertRaisesRegex(ValueError, 'concurrent provider drift'): + self.snapshot() + self.assertEqual(users[0], 2) + + def test_timezone_and_nanosecond_order_is_exact(self): + import copy + rows = copy.deepcopy(self.rows) + rows[1].update(state='REQUEST_CHANGES', body=self.code_body('request_changes'), submitted_at='2026-01-01T00:00:00.000000001Z', updated_at='2026-01-01T00:00:00.000000001Z') + later = copy.deepcopy(self.rows[1]) + later.update(id=303, submitted_at='2026-01-01T01:00:00.000000002+01:00', updated_at='2026-01-01T01:00:00.000000002+01:00', html_url=self.rb.pr['html_url'] + '#pullreview-303') + self.set_rows([later, *rows]) + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'no_op') + self.assertEqual(receipt['decision']['selected_review_ids']['gitea-review-agent'], 303) + + def test_old_head_state_contradiction_is_history_only(self): + import copy + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'no_op', receipt['reason_codes']) + old = copy.deepcopy(self.rows[1]) + old.update(id=303, commit_id='a' * 40, body=self.code_body('request_changes').replace(self.inp['head'], 'a' * 40), html_url=self.rb.pr['html_url'] + '#pullreview-303') + self.set_rows([old, *self.rows]) + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'no_op', receipt['reason_codes']) + history = next((x for x in receipt['decision']['timeline'] if x['id'] == 303)) + self.assertIn('old_head', history['reason_codes']) + self.assertFalse(history['eligible']) + + def test_same_instant_body_conflict_has_no_selected_review(self): + import copy + self.assertEqual(self.snapshot()['status'], 'no_op') + other = copy.deepcopy(self.rows[1]) + other.update(id=303, body=other['body'] + 'Different finding text.\n', html_url=self.rb.pr['html_url'] + '#pullreview-303') + self.set_rows([other, *self.rows]) + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'blocked') + self.assertIn('conflicting_same_time_reviews', receipt['reason_codes']) + self.assertIsNone(receipt['decision']['selected_review_ids']['gitea-review-agent']) + + def test_ambiguous_code_marker_or_verdict_lines_block(self): + import copy + self.assertEqual(self.snapshot()['status'], 'no_op') + for suffix in ('- Verdict: malformed\n', 'Quoted marker: ' + self.code_body('approve').splitlines()[0] + '\n'): + rows = copy.deepcopy(self.rows) + rows[1]['body'] += suffix + self.set_rows(rows) + with self.subTest(suffix=suffix): + self.assertEqual(self.snapshot()['status'], 'blocked') + + def test_invalid_timezone_minute_is_rejected(self): + import copy + self.assertEqual(self.snapshot()['status'], 'no_op') + rows = copy.deepcopy(self.rows) + rows[1]['submitted_at'] = '2026-01-01T00:00:00+00:99' + self.set_rows(rows) + with self.assertRaisesRegex(ValueError, 'timestamp|submitted_at'): + self.snapshot() + + def test_rework_targets_exclude_spec_approvals(self): + import copy + self.assertEqual(self.snapshot()['status'], 'no_op') + rows = copy.deepcopy(self.rows) + rows[1].update(state='REQUEST_CHANGES', body=self.code_body('request_changes')) + self.set_rows(rows) + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'rework') + self.assertEqual(receipt['decision'].get('rework_review_ids'), [302]) + + def test_lone_spec_request_changes_allows_rework(self): + rows = [dict(self.rows[0], state='REQUEST_CHANGES', body=self.spec_body('request_changes'))] + self.set_rows(rows) + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'rework', receipt['reason_codes']) + self.assertEqual(receipt['decision']['selected_review_ids']['gitea-spec-agent'], 301) + self.assertIsNone(receipt['decision']['selected_review_ids']['gitea-review-agent']) + self.assertEqual(receipt['decision']['rework_review_ids'], [301]) + self.assertIn('missing_current_review', receipt['reason_codes']) + + def test_lone_code_request_changes_allows_rework(self): + import copy + row = copy.deepcopy(self.rows[1]) + row.update(state='REQUEST_CHANGES', body=self.code_body('request_changes')) + self.set_rows([row]) + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'rework', receipt['reason_codes']) + self.assertIsNone(receipt['decision']['selected_review_ids']['gitea-spec-agent']) + self.assertEqual(receipt['decision']['selected_review_ids']['gitea-review-agent'], 302) + self.assertEqual(receipt['decision']['rework_review_ids'], [302]) + self.assertIn('missing_current_review', receipt['reason_codes']) + + def test_valid_request_changes_survives_invalid_counterpart_history(self): + import copy + for request_index in (0, 1): + for excluded in ('stale', 'dismissed', 'old_head'): + with self.subTest(request_role=request_index, excluded=excluded): + rows = copy.deepcopy(self.rows) + request = rows[request_index] + request.update(state='REQUEST_CHANGES', body=self.spec_body('request_changes') if request_index == 0 else self.code_body('request_changes')) + other = rows[1 - request_index] + if excluded == 'old_head': + other.update(commit_id='a' * 40, body=self.spec_body('approve', head='a' * 40) if request_index == 1 else self.code_body('approve').replace(self.inp['head'], 'a' * 40)) + else: + other[excluded] = True + self.set_rows(rows) + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'rework', receipt['reason_codes']) + self.assertEqual(receipt['decision']['rework_review_ids'], [request['id']]) + self.assertIsNone(receipt['decision']['selected_review_ids'][other['user']['login']]) + self.assertIn('missing_current_review', receipt['reason_codes']) + + def test_excluded_lone_requests_cannot_authorize_rework(self): + import copy + for request_index in (0, 1): + for excluded in ('stale', 'dismissed', 'old_head'): + with self.subTest(request_role=request_index, excluded=excluded): + request = copy.deepcopy(self.rows[request_index]) + head = 'a' * 40 if excluded == 'old_head' else self.inp['head'] + request.update(state='REQUEST_CHANGES', commit_id=head, body=self.spec_body('request_changes', head=head) if request_index == 0 else self.code_body('request_changes').replace(self.inp['head'], head)) + if excluded != 'old_head': + request[excluded] = True + self.set_rows([request]) + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'blocked') + self.assertEqual(receipt['decision']['rework_review_ids'], []) + + def test_superseded_request_with_missing_counterpart_stays_blocked(self): + import copy + request = copy.deepcopy(self.rows[1]) + request.update(state='REQUEST_CHANGES', body=self.code_body('request_changes')) + approval = copy.deepcopy(self.rows[1]) + approval.update(id=303, submitted_at='2026-01-02T00:00:00Z', updated_at='2026-01-02T00:00:00Z', html_url=self.rb.pr['html_url'] + '#pullreview-303') + self.set_rows([request, approval]) + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'blocked', receipt['reason_codes']) + self.assertNotEqual(receipt['status'], 'rework') + self.assertNotEqual(receipt['status'], 'no_op') + self.assertEqual(receipt['decision']['rework_review_ids'], []) + + def test_true_blockers_override_valid_request_changes(self): + import copy + request = copy.deepcopy(self.rows[1]) + request.update(state='REQUEST_CHANGES', body=self.code_body('request_changes')) + for state in ('COMMENT', 'PENDING'): + with self.subTest(state=state): + rows = copy.deepcopy(self.rows) + rows[1] = copy.deepcopy(request) + rows[0]['state'] = state + self.set_rows(rows) + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'blocked') + self.assertIn('comment_manual_classification' if state == 'COMMENT' else 'pending_authoritative_review', receipt['reason_codes']) + rows = copy.deepcopy(self.rows) + rows[1] = request + rows[0]['body'] = 'missing marker' + self.set_rows(rows) + self.assertEqual(self.snapshot()['status'], 'blocked') + rows = copy.deepcopy(self.rows) + rows[1] = request + rows[0]['body'] = rows[0]['body'].replace(self.inp['contract_sha256'], 'f' * 64) + self.set_rows(rows) + self.assertEqual(self.snapshot()['status'], 'blocked') + rows = copy.deepcopy(self.rows) + from harness.tests._support import spec as result + spec_result = result.result('blocked_spec') + spec_result['target'] = {'owner': 'acme', 'repository': 'demo', 'issue': 7, 'pull': 9, 'head': self.inp['head']} + spec_result['contract_sha256'] = self.inp['contract_sha256'] + spec_result['criteria'][0]['result'] = 'indeterminate' + spec_result['scope']['spec_conflicts'] = ['src/a.py'] + rows[0].update(state='COMMENT', body=result.mod.render_review(spec_result, {'src/a.py', 'tests/test_a.py'}, declaration={'mode': 'none'})) + rows[1] = request + self.set_rows(rows) + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'blocked') + self.assertIn('blocked_spec', receipt['reason_codes']) diff --git a/harness/tests/impl/rework/test_rework_input_gate_diagnostics.py b/harness/tests/impl/rework/test_rework_input_gate_diagnostics.py new file mode 100644 index 0000000..9833d8a --- /dev/null +++ b/harness/tests/impl/rework/test_rework_input_gate_diagnostics.py @@ -0,0 +1,205 @@ +import importlib.util +import os +import tempfile +import unittest +from pathlib import Path +from harness.tests._support.paths import ROOT + +SCRIPT = ROOT / 'harness/agents/impl/skills/gitea-issue-rework/scripts/rework_input_gate.py' + +class OrderedDecisionDiagnosticsTests(unittest.TestCase): + + def setUp(self): + import copy, hashlib + from harness.tests._support.rework import ReworkProviderFixture + from harness.tests._support import spec as spec_tests + self.rb = ReworkProviderFixture().setup() + self.addCleanup(self.rb.cleanup) + self.rb.repo['default_branch'] = 'main' + self.rb.source['body'] = '\nSynthetic approved source contract.\n' + self.inp = {'schema': 'gitea-agent.rework-input/v1', 'target': {'owner': 'acme', 'repository': 'demo', 'issue': 7, 'pr': 9}, 'head': 'b' * 40, 'contract_sha256': hashlib.sha256(self.rb.source['body'].encode()).hexdigest()} + self.rows = [] + for role, rid, uid in [('gitea-spec-agent', 301, 33), ('gitea-review-agent', 302, 34)]: + row = copy.deepcopy(self.rb.reviews[0]) + row.update(id=rid, commit_id=self.inp['head'], state='APPROVED', user={'id': uid, 'login': role}, stale=False, dismissed=False, html_url=self.rb.pr['html_url'] + '#pullreview-' + str(rid)) + if role == 'gitea-spec-agent': + result = spec_tests.result() + result['target'] = dict(self.inp['target']) + result['target']['pull'] = result['target'].pop('pr') + result['target']['head'] = self.inp['head'] + result['contract_sha256'] = self.inp['contract_sha256'] + row['body'] = spec_tests.mod.render_review(result, {'src/a.py', 'tests/test_a.py'}, declaration={'mode': 'none'}) + else: + row['body'] = self.code_body('approve') + self.rows.append(row) + self.set_rows(self.rows) + original = self.rb.request + + def request(path): + from urllib.parse import urlsplit + import re + if re.fullmatch('/repos/acme/demo/pulls/9/reviews/[1-9][0-9]*/comments', urlsplit(path).path): + self.rb.calls.append(path) + return ([], {'X-Total-Count': '0'}, 200) + return original(path) + self.rb.request = request + + def code_body(self, verdict): + return '\n- Verdict: `' + verdict + '`\nBounded native code-review fixture.\n' + + def set_rows(self, rows): + import copy + self.rb.reviews = copy.deepcopy(rows) + self.rb.pr['comments'] = self.rb.underlying['comments'] = self.rb.pr['review_comments'] = len(rows) + + def snapshot(self): + return self.rb.adapter().input_snapshot(self.inp) + + def spec_body(self, verdict, head=None): + from harness.tests._support import spec as spec_tests + result = spec_tests.result(verdict) + result['target'] = {'owner': 'acme', 'repository': 'demo', 'issue': 7, 'pull': 9, 'head': head or self.inp['head']} + result['contract_sha256'] = self.inp['contract_sha256'] + if verdict == 'request_changes': + result['criteria'][0].update(result='unsatisfied', reason='Bounded regression finding.') + result['conclusion'] = 'Bounded correction required.' + return spec_tests.mod.render_review(result, {'src/a.py', 'tests/test_a.py'}, declaration={'mode': 'none'}) + + def test_pr_comment_counter_mismatch_preserves_no_op_and_diagnostics(self): + self.assertEqual(self.snapshot()['status'], 'no_op') + self.rb.pr['comments'] += 4 + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'no_op') + diagnostics = receipt['counter_diagnostics'] + self.assertEqual(set(diagnostics), {'inline', 'pr', 'pr_issue'}) + self.assertEqual(diagnostics['pr']['status'], 'mismatch') + + def test_review_counter_mismatch_preserves_rework(self): + import copy + rows = copy.deepcopy(self.rows) + rows[1].update(state='REQUEST_CHANGES', body=self.code_body('request_changes')) + self.set_rows(rows) + self.rb.pr['review_comments'] += 3 + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'rework') + self.assertEqual(receipt['counter_diagnostics']['inline']['status'], 'mismatch') + self.assertEqual(receipt['decision']['rework_review_ids'], [302]) + + def test_malformed_pr_counters_are_unverified_diagnostics(self): + self.rb.pr['comments'] = 'not-a-count' + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'no_op') + self.assertEqual(receipt['counter_diagnostics']['pr']['status'], 'unverified') + self.assertIn('field', receipt['counter_diagnostics']['pr']) + + def test_unavailable_inline_endpoint_is_rejected_even_with_pr_count_fallback(self): + from urllib.parse import urlsplit + original = self.rb.request + + def unavailable(path): + if '/pulls/9/reviews/' in urlsplit(path).path and urlsplit(path).path.endswith('/comments'): + self.rb.calls.append(path) + return ({'message': 'not available'}, {}, 404) + return original(path) + self.rb.request = unavailable + self.rb.contract['inline']['mode'] = 'per_review_or_pr_count' + with self.assertRaisesRegex(ValueError, 'inline collection unavailable'): + self.snapshot() + + def test_inline_endpoint_405_is_rejected_even_with_pr_count_fallback(self): + from urllib.parse import urlsplit + original = self.rb.request + + def unavailable(path): + if '/pulls/9/reviews/' in urlsplit(path).path and urlsplit(path).path.endswith('/comments'): + self.rb.calls.append(path) + return ({'message': 'method not allowed'}, {}, 405) + return original(path) + self.rb.request = unavailable + self.rb.contract['inline']['mode'] = 'per_review_or_pr_count' + with self.assertRaisesRegex(ValueError, 'inline collection unavailable'): + self.snapshot() + + def test_publication_snapshot_remains_strict_on_pr_counter_mismatch(self): + from harness.tests._support.rework import ReworkProviderFixture + publication = ReworkProviderFixture().setup() + self.addCleanup(publication.cleanup) + publication.adapter().snapshot(publication.expected) + publication.pr['comments'] += 1 + with self.assertRaisesRegex(ValueError, 'provider aggregate count mismatch'): + publication.adapter().snapshot(publication.expected) + + def test_complete_lists_with_zero_pr_issue_counter_preserve_no_op(self): + self.rb.pr['comments'] = self.rb.underlying['comments'] = 0 + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'no_op') + for scope in ('pr', 'pr_issue'): + diagnostic = receipt['counter_diagnostics'][scope] + self.assertEqual(diagnostic, {'field': 'comments', 'components': ['top_level', 'reviews', 'inline'], 'observed': 0, 'expected': 2, 'status': 'mismatch'}) + self.assertEqual(receipt['raw_input_snapshot'][scope]['comments'], 0) + + def test_missing_and_invalid_counters_do_not_replace_record_validation(self): + import copy + saved_pr, saved_issue = (copy.deepcopy(self.rb.pr), copy.deepcopy(self.rb.underlying)) + for scope, field in (('inline', 'review_comments'), ('pr', 'comments'), ('pr_issue', 'comments')): + for value in (None, True, -1, '0', 0.5, 'missing'): + with self.subTest(scope=scope, value=value): + self.rb.pr, self.rb.underlying = (copy.deepcopy(saved_pr), copy.deepcopy(saved_issue)) + record = self.rb.underlying if scope == 'pr_issue' else self.rb.pr + if value == 'missing': + record.pop(field) + else: + record[field] = value + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'no_op') + self.assertEqual(receipt['counter_diagnostics'][scope]['status'], 'unverified') + raw_record = receipt['raw_input_snapshot']['pr_issue' if scope == 'pr_issue' else 'pr'] + if value == 'missing': + self.assertNotIn(field, raw_record) + else: + self.assertEqual(raw_record[field], value) + + def test_counter_mismatch_does_not_unblock_lone_approval(self): + self.set_rows(self.rows[:1]) + self.rb.pr['comments'] = self.rb.underlying['comments'] = 0 + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'blocked') + self.assertEqual(receipt['counter_diagnostics']['pr']['status'], 'mismatch') + self.assertEqual(receipt['decision']['rework_review_ids'], []) + + def test_lone_request_changes_keeps_rework_despite_counter_mismatch(self): + import copy + request = copy.deepcopy(self.rows[1]) + request.update(state='REQUEST_CHANGES', body=self.code_body('request_changes')) + self.set_rows([request]) + self.rb.pr['comments'] += 2 + receipt = self.snapshot() + self.assertEqual(receipt['status'], 'rework', receipt['reason_codes']) + self.assertEqual(receipt['decision']['rework_review_ids'], [302]) + self.assertEqual(receipt['counter_diagnostics']['pr']['status'], 'mismatch') + + def test_counter_mismatch_does_not_hide_per_review_inline_count_mismatch(self): + self.rb.reviews[0]['comments_count'] = 1 + self.rb.pr['comments'] = self.rb.underlying['comments'] = 0 + with self.assertRaisesRegex(ValueError, 'per-review inline count mismatch'): + self.snapshot() + + def test_counter_mismatch_does_not_hide_duplicate_review_records(self): + import copy + self.rb.reviews.append(copy.deepcopy(self.rb.reviews[0])) + self.rb.pr['comments'] = self.rb.underlying['comments'] = 0 + with self.assertRaisesRegex(ValueError, 'duplicate'): + self.snapshot() + + def test_counter_mismatch_does_not_hide_incomplete_review_pagination(self): + original = self.rb.request + + def truncated(path): + if '/pulls/9/reviews?' in path: + payload, headers, status = original(path) + return (payload, dict(headers, **{'X-Total-Count': '3'}), status) + return original(path) + self.rb.request = truncated + self.rb.pr['comments'] = self.rb.underlying['comments'] = 0 + with self.assertRaisesRegex(ValueError, 'truncated collection'): + self.snapshot() diff --git a/harness/tests/impl/rework/test_rework_input_gate_schema.py b/harness/tests/impl/rework/test_rework_input_gate_schema.py new file mode 100644 index 0000000..2792305 --- /dev/null +++ b/harness/tests/impl/rework/test_rework_input_gate_schema.py @@ -0,0 +1,36 @@ +import importlib.util +import os +import tempfile +import unittest +from pathlib import Path +from harness.tests._support.paths import ROOT + +SCRIPT = ROOT / 'harness/agents/impl/skills/gitea-issue-rework/scripts/rework_input_gate.py' + +class InputGateTests(unittest.TestCase): + + def load(self): + spec = importlib.util.spec_from_file_location('rework_input_gate', SCRIPT) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + def test_valid_input_schema_is_accepted(self): + gate = self.load() + value = {'schema': 'gitea-agent.rework-input/v1', 'target': {'owner': 'acme', 'repository': 'demo', 'issue': 7, 'pr': 9}, 'head': 'a' * 40, 'contract_sha256': 'b' * 64} + self.assertEqual(gate.validate_input(value), value) + + def test_initial_output_is_immutable_and_rejected_before_credential_or_get(self): + cli_path = SCRIPT.with_name('rework_readback_cli.py') + spec = importlib.util.spec_from_file_location('rework_readback_cli', cli_path) + cli = importlib.util.module_from_spec(spec) + spec.loader.exec_module(cli) + with tempfile.TemporaryDirectory() as root: + output = Path(root) / 'decision.json' + output.write_bytes(b'old\n') + output.chmod(384) + mode = output.stat().st_mode + with self.assertRaisesRegex(ValueError, 'output already exists'): + cli.reject_existing_output(str(output)) + self.assertEqual(output.read_bytes(), b'old\n') + self.assertEqual(output.stat().st_mode, mode) diff --git a/harness/tests/impl/rework/test_rework_publication.py b/harness/tests/impl/rework/test_rework_publication.py new file mode 100644 index 0000000..2258986 --- /dev/null +++ b/harness/tests/impl/rework/test_rework_publication.py @@ -0,0 +1,161 @@ +"""Regression tests for main-owned rework publication integration.""" +import copy +import hashlib +import sys +import unittest +from pathlib import Path +from unittest.mock import Mock + +from harness.tests._support.paths import HARNESS as ROOT +SCRIPT = ROOT / "agents/impl/skills/gitea-issue-rework/scripts" +sys.path.insert(0, str(SCRIPT)) +sys.path.insert(0, str(ROOT)) + +import rework_publication as pub +from harness.lib import impl_operation as ops + +HEAD = "a" * 40 + +def expected(): + # Generate through the canonical renderer to avoid hand-authored bodies. + import rework_comment as rc + d = dict(owner="o", repository="r", issue=4, pr=5, review=6, + reviewed_head="b"*40, final_head=HEAD, contract_sha256="c"*64, + result="partial", findings=[dict(id="F1", status="resolved", classification="valid_fix", + changes="changed safely", verification="checks pass", reason="verified")], + local_checks="pass", runtime_browser="not run", commit_status="없음", actions_runs="없음") + dd, vals, fs, extras, cs, ar = rc.canon(d) + body = rc.render_body(dd, vals, fs, extras, cs, ar) + rh = hashlib.sha256(body.encode()).hexdigest() + corr = hashlib.sha256("\0".join(["o", "r", "4", "5", "6", "b"*40, HEAD, "c"*64]).encode()).hexdigest() + marker = f"\n" + return dict(body=body+marker, marker=marker, marker_free_body=body, result_sha256=rh, correlation=corr, + target=dict(owner="o", repository="r", issue=4, pr=5), review=6, + reviewed_head="b"*40, head=HEAD, contract_sha256="c"*64) + +def setup_case(op="comment.create", payload=None, approval_flags=None): + if payload is None: + e=expected(); payload={"expected":e,"target":e["target"]} + intent=ops.Intent("v1", op, "o/r", 5, HEAD, payload) + approval=ops.Approval("v1", intent.digest, "approved", **(approval_flags or {})) + declaration={"mode":"none"} + calls=[] + adapter=ops.Adapter(preflight=lambda i: {"fresh":True,"scope":"exact"}, capability=True, + mutate=lambda i: calls.append(copy.deepcopy(i)) or {"acknowledgement":"ok"}, + readback=lambda i: {"scope":"exact","head":HEAD}, verify=lambda i,p,o: True) + return intent,approval,declaration,calls,adapter + +class PublicationTests(unittest.TestCase): + + def test_invalid_body_target_head_approval_do_not_write(self): + cases = [] + cases.append(setup_case('pr.patch', {'body': 'body', 'closing': '#4'})) + cases.append(setup_case('comment.create', {'expected': expected(), 'target': {'owner': 'evil', 'repository': 'r', 'issue': 4, 'pr': 5}})) + p = {'head': 'f' * 40} + cases.append(setup_case('push', p)) + for intent, approval, decl, calls, adapter in cases: + with self.subTest(intent=intent.operation), self.assertRaises(Exception): + pub.execute_rework_operation(adapter, intent=intent, approval=approval, declaration=decl, coordinator=ops.Coordinator(), on_attempt=lambda x: None) + self.assertEqual(calls, []) + + def test_single_mutation_and_fresh_adoption_at_boundary(self): + intent, approval, decl, calls, adapter = setup_case() + old = pub.require_adoption + try: + adopted = Mock(return_value=None) + pub.require_adoption = adopted + attempts = [] + result = pub.execute_rework_operation(adapter, intent=intent, approval=approval, declaration=decl, coordinator=ops.Coordinator(), on_attempt=attempts.append) + self.assertEqual(result.status, 'verified') + self.assertEqual(len(calls), 1) + self.assertEqual(len(attempts), 1) + adopted.assert_called_once() + finally: + pub.require_adoption = old + + def test_adoption_and_preflight_failure_have_zero_mutations(self): + intent, approval, decl, calls, adapter = setup_case() + old = pub.require_adoption + try: + pub.require_adoption = Mock(side_effect=ValueError('stale')) + r = pub.execute_rework_operation(adapter, intent=intent, approval=approval, declaration=decl, coordinator=ops.Coordinator(), on_attempt=lambda x: None) + self.assertEqual(r.status, 'partial') + self.assertEqual(calls, []) + adapter.preflight = lambda i: {'fresh': False} + r = pub.execute_rework_operation(adapter, intent=intent, approval=approval, declaration=decl, coordinator=ops.Coordinator(), on_attempt=lambda x: None) + self.assertEqual(r.status, 'partial') + self.assertEqual(calls, []) + finally: + pub.require_adoption = old + + def test_uncertain_ack_readback_once_literal_true_required(self): + intent, approval, decl, calls, adapter = setup_case() + old = pub.require_adoption + try: + pub.require_adoption = Mock(return_value=None) + mutations = [] + reads = [] + adapter.mutate = lambda i: mutations.append(i) or (_ for _ in ()).throw(ops.UncertainAcknowledgement()) + adapter.readback = lambda i: reads.append(i) or {'head': HEAD} + adapter.verify = lambda i, p, o: 1 + result = pub.execute_rework_operation(adapter, intent=intent, approval=approval, declaration=decl, coordinator=ops.Coordinator(), on_attempt=lambda x: None) + self.assertEqual(result.status, 'partial') + self.assertEqual((len(mutations), len(reads)), (1, 1)) + finally: + pub.require_adoption = old + + def test_reuse_collects_fresh_state_without_any_mutation(self): + intent, approval, declaration, calls, adapter = setup_case() + reads = [] + adapter.preflight = lambda value: reads.append('preflight') or {'fresh': True} + adapter.readback = lambda value: reads.append('readback') or {'exact': True} + adapter.verify_reuse = lambda value, before, after: after.get('exact') is True + result = pub.verify_rework_reuse(adapter, intent=intent, approval=approval, declaration=declaration) + self.assertEqual(result['action'], 'reuse') + self.assertEqual(reads, ['preflight', 'readback']) + self.assertEqual(calls, []) + + def test_reuse_stale_preflight_blocks_even_when_callback_would_accept(self): + intent, approval, declaration, calls, adapter = setup_case() + reads = [] + adapter.preflight = lambda value: {'fresh': False} + adapter.readback = lambda value: reads.append(True) + adapter.verify_reuse = lambda *args: True + with self.assertRaises(ValueError): + pub.verify_rework_reuse(adapter, intent=intent, approval=approval, declaration=declaration) + self.assertEqual(reads, []) + self.assertEqual(calls, []) + + def test_canonical_comment_cannot_publish_to_different_intent_scope(self): + intent, approval, declaration, calls, adapter = setup_case() + for target, target_id in [('evil/r', 5), ('o/r', 999)]: + changed = ops.Intent('v1', intent.operation, target, target_id, HEAD, intent.payload) + approved = ops.Approval('v1', changed.digest, 'approved') + with self.subTest(target=target, target_id=target_id), self.assertRaises(ValueError): + pub.execute_rework_operation(adapter, intent=changed, approval=approved, declaration=declaration, coordinator=ops.Coordinator(), on_attempt=lambda record: None) + self.assertEqual(calls, []) + + def test_unimplemented_client_cas_correction_blocks_before_write(self): + import rework_comment as rc + new = expected() + old = copy.deepcopy(new) + body = new['marker_free_body'] + for label in ('Source Issue', 'Pull Request', 'Source Review'): + body = body.replace('- ' + label + ': ', '- ' + label + ': #') + digest = hashlib.sha256(body.encode()).hexdigest() + marker = new['marker'].replace(new['result_sha256'], digest) + old.update(body=body + marker, marker=marker, marker_free_body=body, result_sha256=digest) + payload = {'old_expected': old, 'new_expected': new, 'comment_id': 7, 'actor_id': 42} + intent = ops.Intent('v1', 'correction', 'o/r', 7, HEAD, payload) + approval = ops.Approval('v1', intent.digest, 'approved', correction=True) + calls = [] + adapter = ops.Adapter(lambda value: {'fresh': True}, True, lambda value: calls.append('write'), lambda value: {'exact': True}, lambda *args: True) + with self.assertRaises(ValueError): + pub.execute_rework_operation(adapter, intent=intent, approval=approval, declaration={'mode': 'none'}, coordinator=ops.Coordinator(), on_attempt=lambda record: None) + self.assertEqual(calls, []) + + def test_generic_comment_patch_rejected_and_correction_requires_explicit_approval(self): + intent, approval, decl, calls, adapter = setup_case('comment.patch', {'body': 'arbitrary'}) + with self.assertRaises(ValueError): + pub.execute_rework_operation(adapter, intent=intent, approval=approval, declaration=decl, coordinator=ops.Coordinator(), on_attempt=lambda x: None) + self.assertEqual(calls, []) diff --git a/harness/tests/impl/rework/test_rework_readback_bridge.py b/harness/tests/impl/rework/test_rework_readback_bridge.py new file mode 100644 index 0000000..96987ef --- /dev/null +++ b/harness/tests/impl/rework/test_rework_readback_bridge.py @@ -0,0 +1,157 @@ +import copy +import hashlib +import json +import unittest +from argparse import Namespace +from pathlib import Path +from urllib.parse import urlsplit +from harness.tests._support.contracts import read_contract +from harness.tests._support.rework import DIRECTORY, ORIGIN, T0, T1, ReworkProviderFixture + +class ReworkReadbackBridgeTests(unittest.TestCase): + + def setUp(self): + self._fixture = ReworkProviderFixture().setup() + self.addCleanup(self._fixture.cleanup) + + @property + def temp(self): + return self._fixture._temp + + def adapter(self): + module = self._fixture.load('rework_readback') + return module.ReworkReadbackAdapter(self._fixture._request_override or self._fixture.request, origin=ORIGIN + '/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=42, provider_contract=self._fixture.contract) + + def adapter_for(self, request): + module = self._fixture.load('rework_readback') + return module.ReworkReadbackAdapter(request, origin=ORIGIN + '/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=42, provider_contract=self._fixture.contract) + + def __getattr__(self, name): + fixture = self.__dict__.get('_fixture') + if fixture is not None and name == 'request' and (fixture._request_override is not None): + return fixture._request_override + if fixture is not None and hasattr(fixture, name): + return getattr(fixture, name) + raise AttributeError(name) + + def __setattr__(self, name, value): + fixture = self.__dict__.get('_fixture') + if fixture is not None and name == 'request': + fixture._request_override = value if value is not fixture.request else None + return + if fixture is not None and name != '_fixture' and hasattr(fixture, name): + setattr(fixture, name, value) + return + super().__setattr__(name, value) + + def test_live_bridge_round_trip_over_local_HTTP(self): + import contextlib + import http.server + import io + import os + import threading + import urllib.request + from unittest.mock import patch + cli = self.load('rework_readback_cli') + root = Path(self.temp.name) + profile = root / 'profile' + profile.mkdir(mode=448) + secret = 'synthetic-fixture-' + 'q' * 40 + (profile / '.env').write_text('GITEA_TOKEN=' + secret + '\n') + (profile / '.env').chmod(384) + (root / 'provider.json').write_text(json.dumps(self.contract)) + contract_hash = hashlib.sha256((root / 'provider.json').read_bytes()).hexdigest() + requests = [] + fixture = self + + class Handler(http.server.BaseHTTPRequestHandler): + + def log_message(self, format, *args): + pass + + def do_GET(self): + requests.append((self.command, self.headers.get('Authorization') == 'token ' + secret)) + data, headers, status = fixture.request(self.path[len('/api/v1'):]) + self.send_response(status) + self.send_header('Content-Type', 'application/json') + for key, value in headers.items(): + self.send_header(key, value) + self.end_headers() + self.wfile.write(json.dumps(data).encode()) + server = http.server.ThreadingHTTPServer(('127.0.0.1', 0), Handler) + thread = threading.Thread(target=server.serve_forever, daemon=True) + thread.start() + inner = urllib.request.build_opener(cli.NoRedirect()) + + class Proxy: + + def open(self, request, timeout): + forwarded = urllib.request.Request('http://127.0.0.1:' + str(server.server_port) + request.selector, headers=dict(request.header_items()), method=request.get_method()) + response = inner.open(forwarded, timeout=timeout) + + class Response: + status = response.status + headers = response.headers + + def geturl(self): + return request.full_url + + def read1(self, n): + return response.read1(n) + + def __enter__(self): + return self + + def __exit__(self, *args): + response.close() + return Response() + transport = cli.RestTransport + common = dict(expected=str(root / 'expected.json'), provider_contract=str(root / 'provider.json'), provider_contract_sha256=contract_hash, actor_id=42) + baseline_path, receipt_path = (root / 'baseline.json', root / 'receipt.json') + try: + with patch.dict(os.environ, {'HERMES_HOME': str(profile)}), patch.object(cli, 'RestTransport', side_effect=lambda credential: transport(credential, opener=Proxy())), contextlib.redirect_stdout(io.StringIO()) as stdout: + cli.run_live(Namespace(cmd='snapshot', output=str(baseline_path), **common), self.legacy.verify_readbacks) + baseline_hash = hashlib.sha256(baseline_path.read_bytes()).hexdigest() + self.add_result() + args = Namespace(cmd='verify-live', baseline=str(baseline_path), baseline_sha256=baseline_hash, action='create', output=str(receipt_path), **common) + cli.run_live(args, self.legacy.verify_readbacks) + receipt = json.loads(receipt_path.read_text()) + self.assertEqual(receipt['status'], 'verified') + self.assertEqual(receipt['comment_id'], 501) + self.assertEqual(receipt['readback_provenance']['kind'], 'fresh_authenticated_GET') + original = receipt_path.read_bytes() + self.source['title'] = 'unrelated provider edit' + with self.assertRaises(ValueError): + cli.run_live(args, self.legacy.verify_readbacks) + self.assertEqual(receipt_path.read_bytes(), original) + self.assertNotIn(secret, stdout.getvalue()) + self.assertNotIn(secret, receipt_path.read_text()) + self.assertTrue(requests) + self.assertEqual(set((method for method, _ in requests)), {'GET'}) + self.assertTrue(all((authenticated for _, authenticated in requests))) + finally: + server.shutdown() + server.server_close() + thread.join(timeout=3) + + def test_renderer_to_existing_verifier_create_and_reuse(self): + adapter = self.adapter() + try: + baseline = adapter.snapshot(self.expected) + except Exception as error: + self.fail('actual renderer expected rejected: ' + str(error)) + self.assertEqual(self.calls[0], '/user') + self.add_result() + after = adapter.snapshot(self.expected) + inputs = adapter.normalized_inputs(self.expected, baseline, after, 'create') + receipt = self.legacy.verify_readbacks(self.expected, inputs['pr'], inputs['comments'], inputs['identity'], action='create', baseline_count=inputs['baseline_count']) + self.assertEqual(receipt['status'], 'verified') + self.assertEqual(receipt['comment_id'], 501) + reused = adapter.normalized_inputs(self.expected, after, adapter.snapshot(self.expected), 'reuse') + self.assertEqual(self.legacy.verify_readbacks(self.expected, reused['pr'], reused['comments'], reused['identity'], action='reuse', baseline_count=reused['baseline_count'])['status'], 'verified') + + def test_shared_commit_message_safety_rule_is_linked_and_amend_is_narrow(self): + common = (DIRECTORY.parents[4] / 'operations/repository-cache.md').read_text() + skill = read_contract(DIRECTORY.parent / 'SKILL.md') + self.assertIn('## Commit-message reference safety', common) + self.assertIn('repository-cache.md#commit-message-reference-safety', skill) diff --git a/harness/tests/impl/rework/test_rework_readback_cache.py b/harness/tests/impl/rework/test_rework_readback_cache.py new file mode 100644 index 0000000..9b35fff --- /dev/null +++ b/harness/tests/impl/rework/test_rework_readback_cache.py @@ -0,0 +1,187 @@ +import copy +import hashlib +import json +import unittest +from argparse import Namespace +from pathlib import Path +from urllib.parse import urlsplit +from harness.tests._support.contracts import read_contract +from harness.tests._support.rework import DIRECTORY, ORIGIN, T0, T1, ReworkProviderFixture + +class ReworkReadbackCacheTests(unittest.TestCase): + + def setUp(self): + self._fixture = ReworkProviderFixture().setup() + self.addCleanup(self._fixture.cleanup) + + @property + def temp(self): + return self._fixture._temp + + def adapter(self): + module = self._fixture.load('rework_readback') + return module.ReworkReadbackAdapter(self._fixture._request_override or self._fixture.request, origin=ORIGIN + '/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=42, provider_contract=self._fixture.contract) + + def adapter_for(self, request): + module = self._fixture.load('rework_readback') + return module.ReworkReadbackAdapter(request, origin=ORIGIN + '/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=42, provider_contract=self._fixture.contract) + + def __getattr__(self, name): + fixture = self.__dict__.get('_fixture') + if fixture is not None and name == 'request' and (fixture._request_override is not None): + return fixture._request_override + if fixture is not None and hasattr(fixture, name): + return getattr(fixture, name) + raise AttributeError(name) + + def __setattr__(self, name, value): + fixture = self.__dict__.get('_fixture') + if fixture is not None and name == 'request': + fixture._request_override = value if value is not fixture.request else None + return + if fixture is not None and name != '_fixture' and hasattr(fixture, name): + setattr(fixture, name, value) + return + super().__setattr__(name, value) + + def _cache_adapter(self): + self.contract['version'] = '1.27.3' + self.overrides['/version'] = ({'version': '1.27.3'}, {}, 200) + self.contract['aggregate']['pr'] = {'field': 'comments', 'components': ['top_level', 'reviews'], 'cache_mode': 'gitea-conversation-v1'} + self.contract['aggregate']['pr_issue'] = {'field': 'comments', 'components': ['top_level', 'reviews'], 'cache_mode': 'gitea-conversation-v1'} + return self.adapter() + + def _stale_baseline(self): + self.pr['comments'] = self.underlying['comments'] = 0 + adapter = self._cache_adapter() + return (adapter, adapter.snapshot(self.expected)) + + def test_cache_opt_in_captures_and_validates_bounded_stale_baseline(self): + adapter, baseline = self._stale_baseline() + self.assertEqual(baseline['normalized_counts']['top_level'], 0) + self.assertEqual(baseline['normalized_counts']['reviews'], 1) + self.assertEqual(baseline['raw']['pr']['comments'], 0) + self.assertEqual(baseline['raw']['pr_issue']['comments'], 0) + adapter.validate_snapshot(self.expected, baseline) + self.overrides['/version'] = ({'version': '1.24.6'}, {}, 200) + with self.assertRaisesRegex(ValueError, 'provider version mismatch'): + adapter.snapshot(self.expected) + + def test_cache_create_recomputes_post_conversation_count_and_guards_deltas(self): + adapter, baseline = self._stale_baseline() + self.add_result() + self.pr['comments'] = self.underlying['comments'] = 2 + after = adapter.snapshot(self.expected) + inputs = adapter.normalized_inputs(self.expected, baseline, after, 'create') + receipt = self.legacy.verify_readbacks(self.expected, inputs['pr'], inputs['comments'], inputs['identity'], action='create', baseline_count=inputs['baseline_count']) + self.assertEqual(receipt['status'], 'verified') + self.comments.clear() + self.pr['comments'] = self.underlying['comments'] = 1 + self.pr['updated_at'] = self.underlying['updated_at'] = T0 + synchronized = adapter.snapshot(self.expected) + self.add_result() + synchronized_after = adapter.snapshot(self.expected) + adapter.normalized_inputs(self.expected, synchronized, synchronized_after, 'create') + for bad in (1, None, -1, 3): + changed = copy.deepcopy(after) + changed['raw']['pr']['comments'] = bad + changed['raw']['pr_issue']['comments'] = bad + with self.subTest(after_counter=bad), self.assertRaises(ValueError): + adapter.normalized_inputs(self.expected, baseline, changed, 'create') + for mutation in ('old_comment', 'duplicate', 'native_review', 'source_issue'): + changed = copy.deepcopy(after) + if mutation == 'old_comment': + changed['raw']['comments']['rows'] = [{'id': 400, 'body': 'mutated', 'created_at': T0, 'updated_at': T0, 'user': {'id': 8, 'login': 'other'}}] + changed['raw']['comments']['rows'] + elif mutation == 'duplicate': + changed['raw']['comments']['rows'].append(copy.deepcopy(changed['raw']['comments']['rows'][0])) + elif mutation == 'native_review': + changed['raw']['reviews']['rows'][0]['state'] = 'APPROVED' + else: + changed['raw']['source_issue']['title'] = 'drift' + with self.subTest(mutation=mutation), self.assertRaises(ValueError): + adapter.normalized_inputs(self.expected, baseline, changed, 'create') + + def test_cache_reuse_allows_same_lag_but_rejects_refresh(self): + self.add_result() + self.pr['comments'] = self.underlying['comments'] = 0 + adapter = self._cache_adapter() + baseline = adapter.snapshot(self.expected) + same = adapter.snapshot(self.expected) + inputs = adapter.normalized_inputs(self.expected, baseline, same, 'reuse') + receipt = self.legacy.verify_readbacks(self.expected, inputs['pr'], inputs['comments'], inputs['identity'], action='reuse', baseline_count=inputs['baseline_count']) + self.assertEqual(receipt['status'], 'verified') + self.pr['comments'] = self.underlying['comments'] = 2 + refreshed = adapter.snapshot(self.expected) + with self.assertRaises(ValueError): + adapter.normalized_inputs(self.expected, baseline, refreshed, 'reuse') + + def test_cache_opt_in_rejects_invalid_mode_version_and_components(self): + module = self.load('rework_readback') + for mode, version, components in (('unknown', '1.27.3', ['top_level', 'reviews']), (None, '1.27.3', ['top_level', 'reviews']), (True, '1.27.3', ['top_level', 'reviews']), ('gitea-conversation-v1', '1.27.2', ['top_level', 'reviews']), ('gitea-conversation-v1', '1.27.3', ['reviews']), ('gitea-conversation-v1', '1.27.3', ['top_level']), ('gitea-conversation-v1', '1.27.3', ['top_level', 'reviews', 'inline'])): + contract = copy.deepcopy(self.contract) + contract['version'] = version + contract['aggregate']['pr'] = {'field': 'comments', 'components': components, 'cache_mode': mode} + contract['aggregate']['pr_issue'] = {'field': 'comments', 'components': components, 'cache_mode': mode} + with self.subTest(mode=mode, version=version, components=components), self.assertRaises(ValueError): + module.ReworkReadbackAdapter(self.request, origin=ORIGIN + '/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=42, provider_contract=contract) + + def test_strict_legacy_counter_semantics_are_unchanged_without_cache_mode(self): + self.pr['comments'] = self.underlying['comments'] = 0 + with self.assertRaises(ValueError): + self.adapter().snapshot(self.expected) + + def test_cache_baseline_does_not_count_inline_records_as_conversation(self): + adapter = self._cache_adapter() + self.contract['inline']['components'] = ['reviews'] + adapter = self.adapter() + self.reviews[0]['comments_count'] = 1 + self.inline = [{'id': 801, 'body': 'inline', 'created_at': T0, 'updated_at': T0, 'user': {'id': 33, 'login': 'gitea-review-agent'}}] + self.pr['comments'] = self.underlying['comments'] = 2 + with self.assertRaisesRegex(ValueError, 'cached provider aggregate count exceeds collection'): + adapter.snapshot(self.expected) + + def test_cache_baseline_rejects_malformed_missing_and_ahead_counters(self): + adapter = self._cache_adapter() + for scope in (self.pr, self.underlying): + for bad in (None, -1, True, '0', 1.0, 2): + scope['comments'] = bad + with self.subTest(counter=bad), self.assertRaises(ValueError): + adapter.snapshot(self.expected) + scope.pop('comments') + with self.assertRaises(ValueError): + adapter.snapshot(self.expected) + scope['comments'] = 1 + + def test_cache_transition_rejects_mutation_of_a_genuine_existing_comment(self): + old = {'id': 400, 'html_url': self.pr['html_url'] + '#issuecomment-400', 'body': 'preserved previous comment', 'created_at': T0, 'updated_at': T0, 'user': {'id': 33, 'login': 'gitea-review-agent'}} + self.comments.append(old) + adapter, baseline = self._stale_baseline() + self.add_result() + self.pr['comments'] = self.underlying['comments'] = 3 + after = adapter.snapshot(self.expected) + adapter.normalized_inputs(self.expected, baseline, after, 'create') + changed = copy.deepcopy(after) + changed['raw']['comments']['rows'][0]['body'] = 'unauthorized mutation' + with self.assertRaisesRegex(ValueError, 'existing comment changed or missing'): + adapter.normalized_inputs(self.expected, baseline, changed, 'create') + + def test_cache_opt_in_does_not_relax_an_undeclared_aggregate_scope(self): + self._cache_adapter() + self.contract['aggregate']['pr_issue'].pop('cache_mode') + adapter = self.adapter() + self.pr['comments'] = self.underlying['comments'] = 0 + with self.assertRaisesRegex(ValueError, 'provider aggregate count mismatch'): + adapter.snapshot(self.expected) + self.underlying['comments'] = 1 + adapter.snapshot(self.expected) + + def test_cache_provider_documentation_example_matches_executable_contract(self): + import re + doc = (DIRECTORY.parent / 'references/readback.md').read_text() + match = re.search('```json\\s*(.*?)\\s*```', doc, re.S) + if match is None: + self.fail('provider documentation requires a JSON example') + example = json.loads(match.group(1)) + self._cache_adapter() + self.contract = example + self.adapter().snapshot(self.expected) diff --git a/harness/tests/impl/rework/test_rework_readback_provider.py b/harness/tests/impl/rework/test_rework_readback_provider.py new file mode 100644 index 0000000..cdb245d --- /dev/null +++ b/harness/tests/impl/rework/test_rework_readback_provider.py @@ -0,0 +1,360 @@ +import copy +import hashlib +import json +import unittest +from argparse import Namespace +from pathlib import Path +from urllib.parse import urlsplit +from harness.tests._support.contracts import read_contract +from harness.tests._support.rework import DIRECTORY, ORIGIN, T0, T1, ReworkProviderFixture + +class ReworkReadbackProviderTests(unittest.TestCase): + + def setUp(self): + self._fixture = ReworkProviderFixture().setup() + self.addCleanup(self._fixture.cleanup) + + @property + def temp(self): + return self._fixture._temp + + def adapter(self): + module = self._fixture.load('rework_readback') + return module.ReworkReadbackAdapter(self._fixture._request_override or self._fixture.request, origin=ORIGIN + '/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=42, provider_contract=self._fixture.contract) + + def adapter_for(self, request): + module = self._fixture.load('rework_readback') + return module.ReworkReadbackAdapter(request, origin=ORIGIN + '/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=42, provider_contract=self._fixture.contract) + + def __getattr__(self, name): + fixture = self.__dict__.get('_fixture') + if fixture is not None and name == 'request' and (fixture._request_override is not None): + return fixture._request_override + if fixture is not None and hasattr(fixture, name): + return getattr(fixture, name) + raise AttributeError(name) + + def __setattr__(self, name, value): + fixture = self.__dict__.get('_fixture') + if fixture is not None and name == 'request': + fixture._request_override = value if value is not fixture.request else None + return + if fixture is not None and name != '_fixture' and hasattr(fixture, name): + setattr(fixture, name, value) + return + super().__setattr__(name, value) + + def test_review_only_projection_preserves_inline_records_without_counting_them(self): + module = self.load('rework_readback') + contract = copy.deepcopy(self.contract) + contract['inline']['components'] = ['reviews'] + adapter = module.ReworkReadbackAdapter(self.request, origin=ORIGIN + '/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=42, provider_contract=contract) + self.pr['review_comments'] = 1 + self.reviews[0]['comments_count'] = 0 + snapshot = adapter.snapshot(self.expected) + self.assertEqual(snapshot['normalized_counts']['inline'], 0) + self.reviews[0]['comments_count'] = 1 + self.inline = [{'id': 801, 'body': 'inline', 'created_at': T0, 'updated_at': T0, 'user': {'id': 33, 'login': 'gitea-review-agent'}}] + self.pr['review_comments'] = 1 + self.pr['comments'] = self.underlying['comments'] = 2 + snapshot = adapter.snapshot(self.expected) + self.assertEqual(snapshot['normalized_counts']['inline'], 1) + + def test_review_only_projection_rejects_counter_mismatch(self): + module = self.load('rework_readback') + contract = copy.deepcopy(self.contract) + contract['inline']['components'] = ['reviews'] + adapter = module.ReworkReadbackAdapter(self.request, origin=ORIGIN + '/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=42, provider_contract=contract) + self.reviews[0]['comments_count'] = 1 + self.inline = [{'id': 801, 'body': 'inline', 'created_at': T0, 'updated_at': T0, 'user': {'id': 33, 'login': 'gitea-review-agent'}}] + self.pr['review_comments'] = 0 + with self.assertRaisesRegex(ValueError, 'inline provider counter crosscheck failed'): + adapter.snapshot(self.expected) + + def test_inline_component_contract_validation_remains_strict(self): + module = self.load('rework_readback') + for parts in ([], None, 'reviews', ['inline', 'inline'], ['reviews', 'reviews'], ['unknown'], ['reviews', 'unknown'], ['reviews', 'inline', 'reviews']): + contract = copy.deepcopy(self.contract) + contract['inline']['components'] = parts + with self.subTest(components=parts), self.assertRaises(ValueError): + module.ReworkReadbackAdapter(self.request, origin=ORIGIN + '/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=42, provider_contract=contract) + + def test_review_only_projection_retains_inline_evidence_rejections(self): + module = self.load('rework_readback') + contract = copy.deepcopy(self.contract) + contract['inline']['components'] = ['reviews'] + adapter = module.ReworkReadbackAdapter(self.request, origin=ORIGIN + '/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=42, provider_contract=contract) + self.reviews[0]['comments_count'] = 1 + self.inline = [{'id': 801, 'body': 'inline', 'created_at': T0, 'updated_at': T0, 'user': {'id': 33, 'login': 'gitea-review-agent'}}] + self.pr['review_comments'] = 2 + with self.assertRaises(ValueError): + adapter.snapshot(self.expected) + self.pr['review_comments'] = 1 + self.inline = [] + with self.assertRaisesRegex(ValueError, 'per-review inline count mismatch'): + adapter.snapshot(self.expected) + self.inline = [{'id': 801, 'body': 'inline', 'created_at': T0, 'updated_at': T0, 'user': {'id': 33, 'login': 'gitea-review-agent'}}, {'id': 801, 'body': 'duplicate', 'created_at': T0, 'updated_at': T0, 'user': {'id': 33, 'login': 'gitea-review-agent'}}] + with self.assertRaisesRegex(ValueError, 'duplicate'): + adapter.snapshot(self.expected) + self.inline = [{'id': 801, 'body': 'inline', 'created_at': T0, 'updated_at': T0, 'user': {'id': 33, 'login': 'gitea-review-agent'}}] + self.overrides = {'/repos/acme/demo/pulls/9/reviews/301/comments?page=1&limit=50': (self.inline, {'X-Total-Count': '2'}, 200)} + with self.assertRaises(ValueError): + adapter.snapshot(self.expected) + + def test_snapshot_rejects_wrong_provider_bindings(self): + cases = [('repo', ['id'], True), ('repo', ['full_name'], 'elsewhere/demo'), ('repo', ['html_url'], 'https://foreign.invalid/acme/demo'), ('source', ['number'], 8), ('source', ['id'], True), ('source', ['html_url'], ORIGIN + '/elsewhere/demo/issues/7'), ('source', ['pull_request'], {}), ('pr', ['number'], 10), ('pr', ['id'], True), ('pr', ['url'], 'https://foreign.invalid/api/v1/repos/acme/demo/pulls/9'), ('pr', ['html_url'], ORIGIN + '/elsewhere/demo/pulls/9'), ('pr', ['head', 'sha'], 'c' * 40), ('pr', ['base', 'repo', 'id'], 4), ('pr', ['state'], 'closed'), ('underlying', ['number'], 10), ('underlying', ['pull_request'], True), ('reviews', [0, 'commit_id'], 'c' * 40)] + error = self.load('rework_readback').ReadbackError + self.assertTrue(issubclass(error, ValueError)) + for attribute, keys, value in cases: + with self.subTest(attribute=attribute, keys=keys): + original = copy.deepcopy(getattr(self, attribute)) + item = getattr(self, attribute) + for key in keys[:-1]: + item = item[key] + item[keys[-1]] = value + try: + with self.assertRaises(ValueError): + self.adapter().snapshot(self.expected) + finally: + setattr(self, attribute, original) + + def test_pagination_complete_51_and_rejects_bad_pages(self): + self.comments = [{'id': 600 + i, 'body': 'existing ' + str(i), 'created_at': T0, 'updated_at': T0, 'user': {'id': 8, 'login': 'other'}} for i in range(51)] + self.pr['comments'] = self.underlying['comments'] = 52 + snapshot = self.adapter().snapshot(self.expected) + self.assertEqual(snapshot['normalized_counts']['top_level'], 51) + self.assertEqual(len(snapshot['raw']['comments']['pages']), 2) + first = '/repos/acme/demo/issues/9/comments?page=1&limit=50' + second = '/repos/acme/demo/issues/9/comments?page=2&limit=50' + invalid = [(second, [self.comments[0]], {'X-Total-Count': '51'}, 200), (second, [self.comments[-1]], {'X-Total-Count': '52'}, 200), (first, None, {}, 200), (first, [], {'X-Total-Count': '0', 'x-total-count': '0'}, 200), (first, [], {'X-Total-Count': '0', 'Link': '; rel="next"'}, 200), (first, self.comments[:50], {}, 200), (first, [{'id': True}], {'X-Total-Count': '1'}, 200), (first, [], {'X-Total-Count': '0', 'Link': 'not a Link'}, 200), (first, [], {}, True), (first, [], {}, 403), (first, [], {}, 302)] + for path, rows, headers, status in invalid: + with self.subTest(path=path, headers=headers, status=status): + self.overrides = {path: (rows, headers, status)} + with self.assertRaises(ValueError): + self.adapter().snapshot(self.expected) + self.overrides = {} + + def test_counts_only_inline_fallback_is_explicit_and_crosschecked(self): + self.contract['inline']['mode'] = 'per_review_or_pr_count' + self.reviews[0]['comments_count'] = 3 + self.pr['comments'] = self.underlying['comments'] = self.pr['review_comments'] = 4 + endpoint = '/repos/acme/demo/pulls/9/reviews/301/comments?page=1&limit=50' + self.overrides = {endpoint: (None, {}, 404)} + try: + snapshot = self.adapter().snapshot(self.expected) + except ValueError as error: + self.fail('configured counts-only fallback missing: ' + str(error)) + self.assertEqual(snapshot['normalized_counts']['inline'], 3) + self.assertEqual(snapshot['fallback'], 'counts_only') + self.assertEqual(snapshot['raw']['inline']['301']['rows'], []) + self.pr['review_comments'] = 5 + with self.assertRaises(ValueError): + self.adapter().snapshot(self.expected) + self.pr['review_comments'] = 4 + for status, payload in ((403, None), (302, None), (200, None)): + self.overrides = {endpoint: (payload, {}, status)} + with self.subTest(status=status): + with self.assertRaises(ValueError): + self.adapter().snapshot(self.expected) + self.contract['inline']['mode'] = 'per_review' + self.overrides = {endpoint: (None, {}, 404)} + with self.assertRaises(ValueError): + self.adapter().snapshot(self.expected) + + def test_preflight_rejects_unbound_contract_and_target_before_GET(self): + module = self.load('rework_readback') + changes = [(['schema'], 'unsupported'), (['version'], True), (['aggregate', 'pr', 'components'], ['inline']), (['aggregate', 'pr', 'components'], ['top_level', 'top_level']), (['aggregate', 'pr_issue', 'field'], 'other'), (['inline', 'mode'], 'automatic'), (['inline', 'components'], ['other'])] + for keys, value in changes: + contract = copy.deepcopy(self.contract) + item = contract + for key in keys[:-1]: + item = item[key] + item[keys[-1]] = value + with self.subTest(contract=keys): + with self.assertRaises(ValueError): + module.ReworkReadbackAdapter(self.request, origin=ORIGIN + '/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=42, provider_contract=contract) + self.assertEqual(self.calls, []) + for keys, value in ((['target', 'owner'], 'acme/../elsewhere'), (['target', 'issue'], 9), (['review'], True), (['head'], 'B' * 40), (['body'], 'forged body')): + expected = copy.deepcopy(self.expected) + item = expected + for key in keys[:-1]: + item = item[key] + item[keys[-1]] = value + self.calls = [] + with self.subTest(expected=keys): + with self.assertRaises(ValueError): + self.adapter().snapshot(expected) + self.assertEqual(self.calls, [], 'invalid expectation reached provider') + + def test_snapshot_detects_drift_during_collection(self): + original = self._fixture.request + endpoints = ['/user', '/version', '/repos/acme/demo', '/repos/acme/demo/issues/7', '/repos/acme/demo/pulls/9', '/repos/acme/demo/issues/9', '/repos/acme/demo/pulls/9/reviews'] + for endpoint in endpoints: + counts = {} + + def changing(path, base_request=original, drift_endpoint=endpoint, call_counts=counts): + value, headers, status = base_request(path) + key = urlsplit(path).path + call_counts[key] = call_counts.get(key, 0) + 1 + if key == drift_endpoint and call_counts[key] > 1: + if type(value) is list: + value[0]['state'] = 'APPROVED' + else: + value['unexpected_drift'] = True + return (value, headers, status) + self.request = changing + try: + with self.subTest(endpoint=endpoint): + with self.assertRaisesRegex(ValueError, 'drift'): + self.adapter_for(changing).snapshot(self.expected) + finally: + self._fixture._request_override = None + + def test_normalization_rejects_unrelated_delta_and_forged_pages(self): + adapter = self.adapter() + baseline = adapter.snapshot(self.expected) + self.add_result() + after = adapter.snapshot(self.expected) + adapter.normalized_inputs(self.expected, baseline, after, 'create') + cases = [(['schema'], 'forged'), (['actor_id'], True), (['origin'], 'https://foreign.invalid/api/v1'), (['target', 'owner'], 'other'), (['provider', 'version'], 'different'), (['expected_sha256'], '0' * 64), (['raw', 'identity', 'login'], 'other'), (['raw', 'source_issue', 'body'], 'unrelated edit'), (['raw', 'reviews', 'rows', 0, 'state'], 'APPROVED'), (['raw', 'pr', 'head', 'sha'], 'c' * 40), (['raw', 'pr', 'title'], 'unrelated edit'), (['raw', 'pr', 'updated_at'], '2026-01-03T00:00:00Z'), (['raw', 'comments', 'pages', 0, 'path'], '/repos/elsewhere/demo/issues/9/comments?page=1&limit=50'), (['raw', 'comments', 'pages', 0, 'count'], 0), (['raw', 'comments', 'rows', 0, 'body'], 'forged body'), (['raw', 'comments', 'rows', 0, 'user', 'id'], 8), (['raw', 'comments', 'rows', 0, 'created_at'], '2025-01-01T00:00:00Z'), (['raw', 'comments', 'rows', 0, 'updated_at'], '2026-01-03T00:00:00Z')] + for keys, value in cases: + changed = copy.deepcopy(after) + item = changed + for key in keys[:-1]: + item = item[key] + item[keys[-1]] = value + with self.subTest(keys=keys): + with self.assertRaises(ValueError): + adapter.normalized_inputs(self.expected, baseline, changed, 'create') + with self.assertRaises(ValueError): + adapter.normalized_inputs(self.expected, baseline, after, 'reuse') + + def test_existing_comment_rows_are_immutable(self): + self.comments = [{'id': 400, 'body': 'prior comment', 'created_at': T0, 'updated_at': T0, 'user': {'id': 8, 'login': 'other'}}] + self.pr['comments'] = self.underlying['comments'] = 2 + adapter = self.adapter() + baseline = adapter.snapshot(self.expected) + self.add_result() + after = adapter.snapshot(self.expected) + adapter.normalized_inputs(self.expected, baseline, after, 'create') + after['raw']['comments']['rows'][0]['body'] = 'changed prior comment' + with self.assertRaisesRegex(ValueError, 'existing comment'): + adapter.normalized_inputs(self.expected, baseline, after, 'create') + + def test_snapshot_validates_captured_rows_before_persisting(self): + self.add_result() + self.adapter().snapshot(self.expected) + for keys, value in ((['body'], True), (['user', 'id'], True), (['created_at'], '2026-01-02T00:00:00'), (['updated_at'], T0)): + saved = copy.deepcopy(self.comments) + item = self.comments[0] + for key in keys[:-1]: + item = item[key] + item[keys[-1]] = value + with self.subTest(keys=keys): + with self.assertRaises(ValueError): + self.adapter().snapshot(self.expected) + self.comments = saved + self.source['updated_at'] = 'not a timestamp' + with self.assertRaises(ValueError): + self.adapter().snapshot(self.expected) + + def test_underlying_relation_requires_native_html_url(self): + self.adapter().snapshot(self.expected) + self.underlying['pull_request'].pop('html_url') + with self.assertRaises(ValueError): + self.adapter().snapshot(self.expected) + + def test_non_next_Link_relations_are_coherent_and_preserved(self): + endpoint = '/repos/acme/demo/issues/7/comments' + path = endpoint + '?page=1&limit=50' + for relation, page in (('last', 21), ('prev', 1), ('first', 2)): + self.overrides = {path: ([], {'X-Total-Count': '0', 'Link': '; rel="' + relation + '"'}, 200)} + with self.subTest(relation=relation): + with self.assertRaises(ValueError): + self.adapter().snapshot(self.expected) + header = '; rel="last"' + self.overrides = {path: ([], {'X-Total-Count': '0', 'Link': header}, 200)} + snapshot = self.adapter().snapshot(self.expected) + self.assertEqual(snapshot['raw']['source_comments']['pages'][0].get('link'), header) + + def test_pagination_link_rejects_unbalanced_relation_quotes(self): + endpoint = '/repos/acme/demo/issues/7/comments' + path = endpoint + '?page=1&limit=50' + for relation in ('"last', 'last"'): + self.overrides = {path: ([], {'X-Total-Count': '0', 'Link': '; rel=' + relation}, 200)} + with self.subTest(relation=relation): + with self.assertRaises(ValueError): + self.adapter().snapshot(self.expected) + + def test_native_Gitea_PR_URL_and_omitted_proven_zero_inline_count(self): + self.pr['url'] = self.pr['html_url'] + self.contract['inline']['components'] = ['inline'] + self.pr.pop('review_comments') + try: + snapshot = self.adapter().snapshot(self.expected) + except ValueError as error: + self.fail('native Gitea wire shape rejected: ' + str(error)) + self.assertEqual(snapshot['normalized_counts']['inline'], 0) + self.assertEqual(snapshot['fallback'], 'none') + self.contract['inline']['mode'] = 'per_review_or_pr_count' + self.overrides = {'/repos/acme/demo/pulls/9/reviews/301/comments?page=1&limit=50': (None, {}, 404)} + with self.assertRaises(ValueError): + self.adapter().snapshot(self.expected) + + def test_boolean_actor_is_rejected_before_other_GETs(self): + module = self.load('rework_readback') + self.overrides = {'/user': ({'id': True, 'login': 'gitea-impl-agent'}, {}, 200)} + adapter = module.ReworkReadbackAdapter(self.request, origin=ORIGIN + '/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=1, provider_contract=self.contract) + with self.assertRaises(ValueError): + adapter.snapshot(self.expected) + self.assertEqual(self.calls, ['/user']) + + def test_required_native_metadata_and_PR_author_are_verified(self): + self.adapter().snapshot(self.expected) + cases = [('pr', ['user'], None), ('pr', ['user', 'id'], 43), ('pr', ['user', 'login'], 'different'), ('source', ['labels'], True), ('source', ['assignees'], True), ('source', ['milestone'], []), ('pr', ['base', 'ref'], None), ('pr', ['head', 'repo_id'], 99), ('reviews', [0, 'state'], None), ('reviews', [0, 'body'], None), ('reviews', [0, 'user', 'login'], 'unapproved'), ('reviews', [0, 'submitted_at'], None), ('reviews', [0, 'dismissed'], True), ('reviews', [0, 'stale'], 'true')] + for name, keys, value in cases: + original = copy.deepcopy(getattr(self, name)) + item = getattr(self, name) + for key in keys[:-1]: + item = item[key] + item[keys[-1]] = value + with self.subTest(name=name, keys=keys): + with self.assertRaises(ValueError): + self.adapter().snapshot(self.expected) + setattr(self, name, original) + for name, field in (('source', 'labels'), ('source', 'assignees'), ('source', 'milestone'), ('underlying', 'user'), ('reviews', 'submitted_at')): + original = copy.deepcopy(getattr(self, name)) + record = getattr(self, name)[0] if name == 'reviews' else getattr(self, name) + record.pop(field) + with self.subTest(missing=name + '.' + field): + with self.assertRaises(ValueError): + self.adapter().snapshot(self.expected) + setattr(self, name, original) + + def test_source_timestamp_change_is_not_admitted_as_causal_proof(self): + adapter = self.adapter() + baseline = adapter.snapshot(self.expected) + self.add_result() + after = adapter.snapshot(self.expected) + adapter.normalized_inputs(self.expected, baseline, after, 'create') + after['raw']['source_issue']['updated_at'] = T1 + with self.assertRaises(ValueError): + adapter.normalized_inputs(self.expected, baseline, after, 'create') + + def test_native_nullable_assignees_and_milestone_without_number(self): + for record in (self.source, self.pr, self.underlying): + record['assignees'] = None + record['milestone'] = {'id': 55, 'title': 'Native milestone', 'state': 'open'} + try: + self.adapter().snapshot(self.expected) + except ValueError as error: + self.fail('native nullable-assignee/milestone shape rejected: ' + str(error)) + self.source.pop('assignees') + with self.assertRaises(ValueError): + self.adapter().snapshot(self.expected) + + def test_closing_keyword_must_match_canonical_Closes(self): + self.adapter().snapshot(self.expected) + self.pr['body'] = 'Fixes #7' + with self.assertRaises(ValueError): + self.adapter().snapshot(self.expected) diff --git a/harness/tests/impl/test_rework_readback_transport.py b/harness/tests/impl/rework/test_rework_readback_transport.py similarity index 73% rename from harness/tests/impl/test_rework_readback_transport.py rename to harness/tests/impl/rework/test_rework_readback_transport.py index f412c24..7d1dfc9 100644 --- a/harness/tests/impl/test_rework_readback_transport.py +++ b/harness/tests/impl/rework/test_rework_readback_transport.py @@ -6,10 +6,12 @@ import tempfile import unittest from pathlib import Path -SCRIPT = Path(__file__).parents[2] / 'agents/impl/skills/gitea-issue-rework/scripts/rework_comment.py' +from harness.tests._support.paths import ROOT +SCRIPT = ROOT / 'harness/agents/impl/skills/gitea-issue-rework/scripts/rework_comment.py' class ReworkReadbackTransportTests(unittest.TestCase): + def module(self): path = SCRIPT.with_name('rework_readback_cli.py') self.assertTrue(path.is_file(), 'canonical live readback bridge is missing') @@ -23,19 +25,22 @@ class ReworkReadbackTransportTests(unittest.TestCase): module = self.module() with tempfile.TemporaryDirectory() as directory: home = Path(directory) - with self.assertRaises(ValueError): module.load_role_token(home) + with self.assertRaises(ValueError): + module.load_role_token(home) source = home / '.env' source.write_text('GITEA_TOKEN=' + 's' * 48 + '\n') - source.chmod(0o644) - with self.assertRaises(ValueError): module.load_role_token(home) - source.chmod(0o600) + source.chmod(420) + with self.assertRaises(ValueError): + module.load_role_token(home) + source.chmod(384) self.assertEqual(module.load_role_token(home), 's' * 48) source.unlink() external = home / 'external' external.write_text('GITEA_TOKEN=' + 's' * 48) - external.chmod(0o600) + external.chmod(384) source.symlink_to(external) - with self.assertRaises(ValueError): module.load_role_token(home) + with self.assertRaises(ValueError): + module.load_role_token(home) def test_http_transport_get_only_redirects_and_secret_echo_are_rejected(self): module = self.module() @@ -47,7 +52,10 @@ class ReworkReadbackTransportTests(unittest.TestCase): secret = 'z' * 48 class Handler(http.server.BaseHTTPRequestHandler): - def log_message(self, format, *args): pass + + def log_message(self, format, *args): + pass + def do_GET(self): requests.append((self.path, self.headers.get('Authorization') == 'token ' + secret)) if self.path == '/api/v1/version': @@ -55,8 +63,7 @@ class ReworkReadbackTransportTests(unittest.TestCase): self.send_header('Location', 'https://outside.invalid/user') self.end_headers() return - body = (b'{"credential":"' + secret.encode() + b'"}' - if self.path.endswith('/secret') else b'{"id":42,"login":"gitea-impl-agent"}') + body = b'{"credential":"' + secret.encode() + b'"}' if self.path.endswith('/secret') else b'{"id":42,"login":"gitea-impl-agent"}' self.send_response(200) self.send_header('Content-Type', 'application/json') if self.path.endswith('/duplicate'): @@ -66,29 +73,37 @@ class ReworkReadbackTransportTests(unittest.TestCase): self.wfile.write(body) class NoRedirect(urllib.request.HTTPRedirectHandler): - def redirect_request(self, *args, **kwargs): return None + def redirect_request(self, *args, **kwargs): + return None server = http.server.ThreadingHTTPServer(('127.0.0.1', 0), Handler) thread = threading.Thread(target=server.serve_forever, daemon=True) thread.start() inner = urllib.request.build_opener(NoRedirect()) class Proxy: + def open(self, request, timeout): - self_request = urllib.request.Request( - 'http://127.0.0.1:' + str(server.server_port) + request.selector, - headers=dict(request.header_items()), method=request.get_method()) + self_request = urllib.request.Request('http://127.0.0.1:' + str(server.server_port) + request.selector, headers=dict(request.header_items()), method=request.get_method()) response = inner.open(self_request, timeout=timeout) original = request.full_url + class Response: status = response.status headers = response.headers - def geturl(self): return original - def read1(self, n): return response.read1(n) - def __enter__(self): return self - def __exit__(self, *args): response.close() - return Response() + def geturl(self): + return original + + def read1(self, n): + return response.read1(n) + + def __enter__(self): + return self + + def __exit__(self, *args): + response.close() + return Response() try: client = module.RestTransport(secret, opener=Proxy()) data, _, status = client('/user') @@ -98,8 +113,9 @@ class ReworkReadbackTransportTests(unittest.TestCase): with self.assertRaises(ValueError) as raised: client('/repos/acme/widget/secret') self.assertNotIn(secret, str(raised.exception)) - with self.assertRaises(ValueError): client('/repos/acme/widget/duplicate') - self.assertTrue(all(authenticated for _, authenticated in requests)) + with self.assertRaises(ValueError): + client('/repos/acme/widget/duplicate') + self.assertTrue(all((authenticated for _, authenticated in requests))) self.assertEqual(len(requests), 4) finally: server.shutdown() @@ -112,18 +128,20 @@ class ReworkReadbackTransportTests(unittest.TestCase): with tempfile.TemporaryDirectory() as directory: receipt = Path(directory) / 'receipt.json' receipt.write_text('original') - receipt.chmod(0o600) - with module.OutputFile(receipt): pass + receipt.chmod(384) + with module.OutputFile(receipt): + pass self.assertEqual(receipt.read_text(), 'original') with module.OutputFile(receipt) as output: output.write({'status': 'verified'}) self.assertIn('verified', receipt.read_text()) - self.assertEqual(receipt.stat().st_mode & 0o777, 0o600) + self.assertEqual(receipt.stat().st_mode & 511, 384) foreign = Path(directory) / 'foreign' foreign.write_text('preserve') receipt.unlink() receipt.symlink_to(foreign) - with self.assertRaises(ValueError): module.OutputFile(receipt) + with self.assertRaises(ValueError): + module.OutputFile(receipt) self.assertEqual(foreign.read_text(), 'preserve') def test_http_operation_budget_stops_before_next_get(self): @@ -131,26 +149,38 @@ class ReworkReadbackTransportTests(unittest.TestCase): from unittest.mock import patch clock = [0] calls = [] + class Opener: + def open(self, request, timeout): calls.append(request.full_url) + class Response: status = 200 headers = {'Content-Type': 'application/json'} emitted = False - def geturl(self): return request.full_url + + def geturl(self): + return request.full_url + def read1(self, n): - if self.emitted: return b'' + if self.emitted: + return b'' self.emitted = True return b'{"id":42}' - def __enter__(self): return self - def __exit__(self, *args): pass + + def __enter__(self): + return self + + def __exit__(self, *args): + pass return Response() with patch.object(module.time, 'monotonic', side_effect=lambda: clock[0]): client = module.RestTransport('b' * 48, opener=Opener()) self.assertEqual(client('/user')[2], 200) clock[0] = 121 - with self.assertRaises(ValueError): client('/version') + with self.assertRaises(ValueError): + client('/version') self.assertEqual(len(calls), 1) def test_live_bridge_rejects_unbound_provider_before_transport(self): @@ -169,26 +199,23 @@ class ReworkReadbackTransportTests(unittest.TestCase): self.assertEqual(module.read_bound_json(contract, trusted)['schema'], 'test-contract') receipt = root / 'receipt.json' receipt.write_text('preserve') - receipt.chmod(0o600) - args = SimpleNamespace(cmd='snapshot', expected=str(expected), - provider_contract=str(contract), provider_contract_sha256='0' * 64, - actor_id=42, output=str(receipt)) + receipt.chmod(384) + args = SimpleNamespace(cmd='snapshot', expected=str(expected), provider_contract=str(contract), provider_contract_sha256='0' * 64, actor_id=42, output=str(receipt)) with patch.object(module, 'RestTransport') as transport: with patch.object(module, 'read_bound_json', wraps=module.read_bound_json) as reader: - with self.assertRaises(ValueError): module.run_live(args, lambda *a, **k: None) + with self.assertRaises(ValueError): + module.run_live(args, lambda *a, **k: None) self.assertGreaterEqual(reader.call_count, 2) transport.assert_not_called() self.assertEqual(receipt.read_text(), 'preserve') def test_existing_command_exposes_readonly_snapshot(self): - result = subprocess.run([sys.executable, '-B', str(SCRIPT), 'snapshot', '--help'], - capture_output=True, text=True, timeout=10) + result = subprocess.run([sys.executable, '-B', str(SCRIPT), 'snapshot', '--help'], capture_output=True, text=True, timeout=10) self.assertEqual(result.returncode, 0, result.stderr) self.assertIn('--provider-contract', result.stdout) self.assertIn('--actor-id', result.stdout) def test_existing_command_exposes_live_verification(self): - result = subprocess.run([sys.executable, '-B', str(SCRIPT), 'verify-live', '--help'], - capture_output=True, text=True, timeout=10) + result = subprocess.run([sys.executable, '-B', str(SCRIPT), 'verify-live', '--help'], capture_output=True, text=True, timeout=10) self.assertEqual(result.returncode, 0, result.stderr) self.assertIn('--baseline-sha256', result.stdout) diff --git a/harness/tests/impl/test_evidence_manifest.py b/harness/tests/impl/test_evidence_manifest.py deleted file mode 100644 index 72adc6e..0000000 --- a/harness/tests/impl/test_evidence_manifest.py +++ /dev/null @@ -1,484 +0,0 @@ -import hashlib -import json -import os -import subprocess -import sys -import tempfile -import unittest -from pathlib import Path - -SCRIPT = Path(__file__).resolve().parents[2] / "agents/impl/skills/gitea-issue-rework/scripts/evidence_manifest.py" - - -def run_git(cwd, *args): - return subprocess.run(["git", *args], cwd=cwd, text=True, capture_output=True, check=True) - - -class EvidenceManifestTests(unittest.TestCase): - def setUp(self): - self.tmp = tempfile.TemporaryDirectory(dir=os.environ.get("TMPDIR", "/opt/data/cache/scratch")) - self.root = Path(self.tmp.name) - self.contract = self.root / "harness/operations/repository-cache.md" - self.contract.parent.mkdir(parents=True) - self.contract.write_text("# Repository cache 운영 지침\n") - self.repo = self.root / "repositories/owner/project/worktrees/issue-1/checkout" - self.artifacts = self.repo.parent / "verification/run-a" - self.repo.mkdir(parents=True) - self.artifacts.mkdir(parents=True) - for path in (self.root / "repositories", *self.repo.parents): - if path == self.root: - break - path.chmod(0o700) - self.repo.chmod(0o700) - self.artifacts.parent.chmod(0o700) - self.artifacts.chmod(0o700) - run_git(self.repo, "init", "-q") - run_git(self.repo, "config", "user.email", "test@example.invalid") - run_git(self.repo, "config", "user.name", "Test") - (self.repo / "input.txt").write_text("hello\n") - run_git(self.repo, "add", "input.txt") - run_git(self.repo, "commit", "-qm", "initial") - - def tearDown(self): self.tmp.cleanup() - - def invoke(self, *args, env=None): - op, *rest = args - common = [] if op == "validate-legacy" else ["--contract-path", str(self.contract), "--artifact-root", str(self.artifacts)] - return subprocess.run([sys.executable, str(SCRIPT), str(op), *common, *map(str, rest)], cwd=self.repo, text=True, capture_output=True, env=env) - - def plan(self, commands=None, **extra): - if commands is None: - commands = [{"id": "check", "purpose": "show input", "argv": ["git", "show", "HEAD:input.txt"], "expected_exit_code": 0, "claims": ["output"]}] - data = {"hash_paths": ["input.txt"], "commands": [dict({"timeout_seconds": 5}, **command) for command in commands]} - data.update(extra); p = self.artifacts / "plan.json"; p.write_text(json.dumps(data)); p.chmod(0o600); return p - - def run_manifest(self, plan=None): - if plan is None: plan = self.plan() - out = self.artifacts / "manifest.json"; result = self.invoke("run", "--repo-root", self.repo, "--plan", plan.name, "--output", "manifest.json"); return result, out - - def test_run_captures_success_claim_and_hash(self): - result, out = self.run_manifest(); self.assertEqual(result.returncode, 0, result.stderr) - manifest = json.loads(out.read_text()) - self.assertEqual(manifest["commands"][0]["argv"], ["git", "show", "HEAD:input.txt"]) - self.assertEqual(manifest["commands"][0]["exit_code"], 0) - self.assertEqual(manifest["commands"][0]["claim_ids"], ["output"]) - self.assertEqual( - manifest["claims"]["output"], - [ - "/commands/0/argv", - "/commands/0/exit_code", - "/commands/0/expected_exit_code", - "/commands/0/output", - "/commands/0/started_at_utc", - "/commands/0/finished_at_utc", - "/commands/0/timeout_seconds", - "/commands/0/execution_status", - "/commands/0/termination", - ], - ) - self.assertEqual(manifest["files"][0]["sha256"], hashlib.sha256(b"hello\n").hexdigest()) - self.assertEqual(self.invoke("validate", "--repo-root", self.repo, "--manifest", "manifest.json").returncode, 0) - - def test_unexpected_exit_is_recorded_and_nonzero(self): - result, out = self.run_manifest(self.plan([{"argv": [sys.executable, "-c", "print('no'); raise SystemExit(3)"], "expected_exit_code": 0, "claims": ["result"]}])) - self.assertNotEqual(result.returncode, 0); self.assertEqual(json.loads(out.read_text())["commands"][0]["exit_code"], 3) - - def test_failed_run_preserves_prior_manifest(self): - result, out = self.run_manifest() - self.assertEqual(result.returncode, 0, result.stderr) - prior = out.read_bytes() - bad = self.plan([{"argv": [sys.executable, "-c", "print('Authorization: Bearer ***')"]}]) - result = self.invoke("run", "--repo-root", self.repo, "--plan", bad, "--output", out) - self.assertNotEqual(result.returncode, 0) - self.assertEqual(out.read_bytes(), prior) - - def test_command_environment_is_allowlisted_and_output_is_bounded(self): - env = os.environ.copy() - env["UNLISTED_SECRET"] = "must-not-reach-child" - command = [{ - "argv": [sys.executable, "-c", "import os; print(os.getenv('UNLISTED_SECRET'))"], - "claims": ["environment"], - }] - plan = self.plan(command) - out = self.artifacts / "manifest.json" - result = self.invoke("run", "--repo-root", self.repo, "--plan", plan.name, "--output", out.name, env=env) - self.assertEqual(result.returncode, 0, result.stderr) - self.assertEqual(json.loads(out.read_text())["commands"][0]["output"], "None\n") - - env["LEAK"] = "SECRET_VALUE" - plan = self.plan(command, command_env_allowlist=["LEAK"]) - result = self.invoke("run", "--repo-root", self.repo, "--plan", plan.name, "--output", out.name, env=env) - self.assertNotEqual(result.returncode, 0) - self.assertNotIn("SECRET_VALUE", out.read_text()) - - prior = out.read_bytes() - plan = self.plan([{ - "argv": [sys.executable, "-c", "print('x' * 100000)"], - "max_output_bytes": 16, - "claims": ["bounded"], - }]) - result = self.invoke("run", "--repo-root", self.repo, "--plan", plan.name, "--output", "overflow.json") - self.assertEqual(result.returncode, 1) - overflow = json.loads((self.artifacts / "overflow.json").read_text()) - self.assertEqual(overflow["result"], "failure") - self.assertEqual(overflow["commands"][0]["execution_status"], "output_limit") - self.assertLessEqual(len(overflow["commands"][0]["output"].encode()), 16) - self.assertEqual(out.read_bytes(), prior) - - def test_output_collision_and_password_argument_are_rejected(self): - plan = self.plan() - result = self.invoke("run", "--repo-root", self.repo, "--plan", plan.name, "--output", "../checkout/input.txt") - self.assertNotEqual(result.returncode, 0) - self.assertEqual((self.repo / "input.txt").read_text(), "hello\n") - - result, out = self.run_manifest(self.plan([{ - "argv": [sys.executable, "-c", "from pathlib import Path; Path('ran.txt').write_text('ran')", "--password", "visible-value"], - "claims": ["password-leak"], - }])) - self.assertNotEqual(result.returncode, 0) - self.assertFalse(out.exists()) - self.assertFalse((self.repo / "ran.txt").exists()) - - def test_argv_only_and_traversal_rejected(self): - result, out = self.run_manifest(self.plan([{"argv": ["echo", "shell=true"]}])) - self.assertNotEqual(result.returncode, 0); self.assertFalse(out.exists()) - result, out = self.run_manifest(self.plan(hash_paths=["../outside"])) - self.assertNotEqual(result.returncode, 0); self.assertFalse(out.exists()) - result, out = self.run_manifest(self.plan(hash_paths=["nested/../input.txt"])) - self.assertNotEqual(result.returncode, 0); self.assertFalse(out.exists()) - - def test_empty_plan_and_malformed_command_record_are_rejected(self): - result, out = self.run_manifest(self.plan([])) - self.assertNotEqual(result.returncode, 0) - self.assertFalse(out.exists()) - - result, out = self.run_manifest() - self.assertEqual(result.returncode, 0, result.stderr) - data = json.loads(out.read_text()) - data["commands"][0]["duration_seconds"] = -1 - out.write_text(json.dumps(data, sort_keys=True, separators=(",", ":")) + "\n") - self.assertNotEqual(self.invoke("validate", "--repo-root", self.repo, "--manifest", out.name).returncode, 0) - - def test_secret_or_header_leak_fails_closed(self): - env = os.environ.copy(); env["TEST_SECRET"] = "never-write-this" - p = self.plan([{"argv": [sys.executable, "-c", "import os; print(os.environ['TEST_SECRET'])"]}], secret_env_names=["TEST_SECRET"]) - result = self.invoke("run", "--repo-root", self.repo, "--plan", "plan.json", "--output", "manifest.json", env=env) - self.assertNotEqual(result.returncode, 0); self.assertFalse((self.artifacts / "manifest.json").exists()) - - def test_source_drift_hash_mismatch_rejected(self): - result, out = self.run_manifest(); self.assertEqual(result.returncode, 0) - (self.repo / "input.txt").write_text("changed\n") - self.assertNotEqual(self.invoke("validate", "--repo-root", self.repo, "--manifest", out.name).returncode, 0) - - def test_command_source_mutation_is_rejected_before_manifest_write(self): - command = [{ - "argv": [sys.executable, "-c", "from pathlib import Path; Path('input.txt').write_text('changed\\n')"], - "claims": ["mutation"], - }] - result, out = self.run_manifest(self.plan(command)) - self.assertNotEqual(result.returncode, 0) - self.assertFalse(out.exists()) - - def test_command_commit_drift_is_rejected_before_manifest_write(self): - script = ( - "from pathlib import Path; import subprocess; " - "Path('other.txt').write_text('committed\\n'); " - "subprocess.run(['git','add','other.txt'],check=True); " - "subprocess.run(['git','commit','-qm','drift'],check=True)" - ) - result, out = self.run_manifest(self.plan([{ - "argv": [sys.executable, "-c", script], - "claims": ["commit-drift"], - }])) - self.assertNotEqual(result.returncode, 0) - self.assertFalse(out.exists()) - - def test_binary_source_hash_is_byte_accurate(self): - payload = b"\xff\x00\x80binary\n" - (self.repo / "binary.bin").write_bytes(payload) - run_git(self.repo, "add", "binary.bin") - run_git(self.repo, "commit", "-qm", "binary") - result, out = self.run_manifest(self.plan(hash_paths=["binary.bin"])) - self.assertEqual(result.returncode, 0, result.stderr) - self.assertEqual(json.loads(out.read_text())["files"][0]["sha256"], hashlib.sha256(payload).hexdigest()) - - def test_invalid_claim_pointer_rejected(self): - result, out = self.run_manifest(self.plan([{"argv": ["true"], "claims": ["bad"]}])) - self.assertEqual(result.returncode, 0) - data=json.loads(out.read_text()); data["claims"]["bad"]="/missing"; out.write_text(json.dumps(data, sort_keys=True, separators=(",", ":"))+"\n") - self.assertNotEqual(self.invoke("validate", "--repo-root", self.repo, "--manifest", out.name).returncode, 0) - - def test_v2_validate_rejects_forged_plan_exclusion(self): - result, out = self.run_manifest() - self.assertEqual(result.returncode, 0, result.stderr) - (self.artifacts / "plan.json").unlink() - (self.repo / "dirty.txt").write_text("dirty\n") - data = json.loads(out.read_text()) - data["excluded_paths"] = ["dirty.txt", "manifest.json"] - out.write_text(json.dumps(data, sort_keys=True, separators=(",", ":")) + "\n") - self.assertNotEqual(self.invoke("validate", "--repo-root", self.repo, "--manifest", out.name).returncode, 0) - - def test_validate_rejects_empty_or_shell_argv(self): - result, out = self.run_manifest() - self.assertEqual(result.returncode, 0, result.stderr) - data = json.loads(out.read_text()) - data["commands"][0]["argv"] = [] - out.write_text(json.dumps(data, sort_keys=True, separators=(",", ":")) + "\n") - self.assertNotEqual(self.invoke("validate", "--repo-root", self.repo, "--manifest", out.name).returncode, 0) - - data["commands"][0]["argv"] = ["bash", "-c", "true"] - out.write_text(json.dumps(data, sort_keys=True, separators=(",", ":")) + "\n") - self.assertNotEqual(self.invoke("validate", "--repo-root", self.repo, "--manifest", out.name).returncode, 0) - - def test_archived_v1_descendant_validates_read_only(self): - result, out = self.run_manifest(); self.assertEqual(result.returncode, 0) - obj=json.loads(out.read_text()) - obj["schema"]="evidence-manifest/v1" - del obj["artifact"] - obj["excluded_paths"]=["plan.json", "manifest.json"] - archived_plan=json.loads((self.artifacts / "plan.json").read_text()) - archived_plan["commands"][0].pop("timeout_seconds", None) - (self.repo / "plan.json").write_text(json.dumps(archived_plan)) - obj["plan"]["sha256"] = hashlib.sha256((self.repo / "plan.json").read_bytes()).hexdigest() - for key in ("timeout_seconds", "execution_status", "termination"): - obj["commands"][0].pop(key, None) - obj["claims"] = {key: pointers[:6] for key, pointers in obj["claims"].items()} - archived=self.repo / "manifest.json" - archived.write_text(json.dumps(obj, sort_keys=True, separators=(",", ":"), ensure_ascii=False)+"\n") - run_git(self.repo, "add", "plan.json", "manifest.json"); run_git(self.repo, "commit", "-qm", "historical evidence fixture") - before=archived.read_bytes() - archived_result = self.invoke("validate-legacy", "--repo-root", self.repo, "--manifest", "manifest.json") - self.assertEqual(archived_result.returncode, 0, archived_result.stderr) - self.assertEqual(archived.read_bytes(), before) - self.assertEqual(run_git(self.repo, "status", "--porcelain").stdout, "") - - -class ExternalEvidenceTests(unittest.TestCase): - def setUp(self): - self.tmp = tempfile.TemporaryDirectory(dir=os.environ.get("TMPDIR", "/opt/data/cache/scratch")) - self.root = Path(self.tmp.name) - self.contract = self.root / "harness/operations/repository-cache.md" - self.contract.parent.mkdir(parents=True) - self.contract.write_text("# Repository cache 운영 지침\n", encoding="utf-8") - self.repo = self.root / "repositories/owner/project/worktrees/issue-1/checkout" - self.artifacts = self.repo.parent / "verification/run-a" - self.repo.mkdir(parents=True) - self.artifacts.mkdir(parents=True) - for path in (self.root / "repositories", *self.repo.parents): - if path == self.root: - break - path.chmod(0o700) - self.repo.chmod(0o700) - self.artifacts.parent.chmod(0o700) - self.artifacts.chmod(0o700) - run_git(self.repo, "init", "-q") - run_git(self.repo, "config", "user.email", "test@example.invalid") - run_git(self.repo, "config", "user.name", "Test") - (self.repo / "input.txt").write_text("hello\n") - run_git(self.repo, "add", "input.txt") - run_git(self.repo, "commit", "-qm", "initial") - self.plan = self.artifacts / "evidence-plan.json" - self.plan.write_text(json.dumps({"hash_paths": ["input.txt"], "commands": [{"id": "check", "argv": ["git", "show", "HEAD:input.txt"], "timeout_seconds": 5, "claims": ["output"]}]})) - self.plan.chmod(0o600) - self.manifest = self.artifacts / "evidence-manifest.json" - - def tearDown(self): - self.tmp.cleanup() - - def invoke(self, op, *args): - return subprocess.run([sys.executable, str(SCRIPT), op, "--contract-path", str(self.contract), "--repo-root", str(self.repo), "--artifact-root", str(self.artifacts), *map(str, args)], text=True, capture_output=True) - - def test_ignored_source_input_is_rejected_for_run_and_validate(self): - (self.repo / ".gitignore").write_text("ignored.txt\n") - run_git(self.repo, "add", ".gitignore") - run_git(self.repo, "commit", "-qm", "ignore fixture") - ignored=self.repo / "ignored.txt" - ignored.write_text("unbound input\n") - result=self.invoke("run", "--plan", self.plan.name, "--output", self.manifest.name) - self.assertEqual(result.returncode, 2, result.stderr) - self.assertIn("not clean", result.stderr) - self.assertFalse(self.manifest.exists()) - ignored.unlink() - self.assertEqual(self.invoke("run", "--plan", self.plan.name, "--output", self.manifest.name).returncode, 0) - prior=self.manifest.read_bytes() - ignored.write_text("unbound input again\n") - result=self.invoke("validate", "--manifest", self.manifest.name) - self.assertEqual(result.returncode, 2, result.stderr) - self.assertIn("not clean", result.stderr) - self.assertEqual(self.manifest.read_bytes(), prior) - - def test_canonical_control_directory_modes_match_installed_harness(self): - # The cache contract permits a trusted NFS root with group write and - # imposes no runtime-directory mode on read-only harness code parents. - self.root.chmod(0o770) - self.contract.parent.parent.chmod(0o777) - result=self.invoke("run", "--plan", self.plan.name, "--output", self.manifest.name) - self.assertEqual(result.returncode, 0, result.stderr) - self.assertEqual(self.invoke("validate", "--manifest", self.manifest.name).returncode, 0) - self.assertEqual(self.root.stat().st_mode & 0o777, 0o770) - self.assertEqual(self.contract.parent.parent.stat().st_mode & 0o777, 0o777) - - def test_same_filename_in_source_and_artifacts_is_not_a_collision(self): - result=self.invoke("run", "--plan", self.plan.name, "--output", "input.txt") - self.assertEqual(result.returncode, 0, result.stderr) - self.assertEqual((self.repo / "input.txt").read_text(), "hello\n") - self.assertEqual(self.invoke("validate", "--manifest", "input.txt").returncode, 0) - - def test_real_linked_detached_worktree_preserves_git_registration(self): - seed=self.root / "seed" - self.repo.rename(seed) - bare=self.repo.parents[2] / "repository.git" - subprocess.run(["git", "clone", "--bare", str(seed), str(bare)], check=True, capture_output=True) - subprocess.run(["git", "--git-dir", str(bare), "worktree", "add", "--detach", str(self.repo), "HEAD"], check=True, capture_output=True) - subprocess.run(["git", "--git-dir", str(bare), "worktree", "lock", str(self.repo)], check=True, capture_output=True) - before=run_git(self.repo, "worktree", "list", "--porcelain").stdout - result=self.invoke("run", "--plan", self.plan.name, "--output", self.manifest.name) - self.assertEqual(result.returncode, 0, result.stderr) - self.assertEqual(self.invoke("validate", "--manifest", self.manifest.name).returncode, 0) - self.assertEqual(run_git(self.repo, "worktree", "list", "--porcelain").stdout, before) - self.assertNotEqual(run_git(self.repo, "rev-parse", "--git-common-dir").stdout.strip(), ".git") - self.assertEqual(run_git(self.repo, "status", "--porcelain").stdout, "") - - def test_canonical_root_identity_and_traversal_denied_before_execution(self): - targets=(self.repo / "docs/verification/run-a", self.root / "scratch/run-a", self.repo.parent.parent / "issue-2/verification/run-a", self.artifacts.parent / "../run-a") - for target in targets: - with self.subTest(target=str(target)): - result=self.invoke("run", "--artifact-root", target, "--plan", self.plan.name, "--output", self.manifest.name) - self.assertEqual(result.returncode, 2) - self.assertFalse(self.manifest.exists()) - for filename in ("../escape.json", "/absolute.json", "a/../escape.json", "./plan.json", "plan.json/"): - with self.subTest(filename=filename): - result=self.invoke("run", "--plan", filename, "--output", self.manifest.name) - self.assertEqual(result.returncode, 2) - result=self.invoke("run", "--contract-path", self.root / "wrong.md", "--plan", self.plan.name, "--output", self.manifest.name) - self.assertEqual(result.returncode, 2) - - def test_artifact_file_and_directory_symlinks_denied(self): - alias=self.artifacts / "alias.json" - alias.symlink_to(self.plan.name) - result=self.invoke("run", "--plan", alias.name, "--output", self.manifest.name) - self.assertEqual(result.returncode, 2) - self.assertIn("unsafe path", result.stderr) - outside=self.root / "outside" - self.artifacts.rename(outside) - self.artifacts.symlink_to(outside, target_is_directory=True) - result=self.invoke("run", "--plan", self.plan.name, "--output", self.manifest.name) - self.assertEqual(result.returncode, 2) - self.assertIn("symlink", result.stderr) - self.assertFalse((outside / self.manifest.name).exists()) - - def test_private_permissions_fail_closed_without_repair(self): - self.plan.chmod(0o644) - result=self.invoke("run", "--plan", self.plan.name, "--output", self.manifest.name) - self.assertEqual(result.returncode, 2) - self.assertEqual(self.plan.stat().st_mode & 0o777, 0o644) - self.plan.chmod(0o600) - self.artifacts.chmod(0o755) - result=self.invoke("run", "--plan", self.plan.name, "--output", self.manifest.name) - self.assertEqual(result.returncode, 2) - self.assertEqual(self.artifacts.stat().st_mode & 0o777, 0o755) - self.assertFalse(self.manifest.exists()) - - def test_wrong_uid_is_denied(self): - import importlib.util - from unittest.mock import patch - spec=importlib.util.spec_from_file_location("evidence_manifest_uid_test", SCRIPT) - assert spec is not None and spec.loader is not None - module=importlib.util.module_from_spec(spec) - spec.loader.exec_module(module) - with patch.object(module.os, "getuid", return_value=self.artifacts.stat().st_uid+1): - with self.assertRaisesRegex(ValueError, "ownership"): - module.safe_entry(self.artifacts, directory=True, mode=0o700) - - def test_group_writable_cache_parent_requires_exclusive_role_membership(self): - import importlib.util - from types import SimpleNamespace - from unittest.mock import patch - spec=importlib.util.spec_from_file_location("evidence_manifest_group_test", SCRIPT) - assert spec is not None and spec.loader is not None - module=importlib.util.module_from_spec(spec); spec.loader.exec_module(module) - parent=self.root / "repositories" - parent.chmod(0o775) - uid=parent.stat().st_uid; gid=parent.stat().st_gid - role=SimpleNamespace(pw_name="role", pw_uid=uid, pw_gid=gid) - foreign=SimpleNamespace(pw_name="foreign", pw_uid=uid+1, pw_gid=gid) - with patch.object(module.grp, "getgrgid", return_value=SimpleNamespace(gr_mem=[])), patch.object(module.pwd, "getpwall", return_value=[role]), patch.object(module.pwd, "getpwnam", return_value=role): - module.safe_repositories(parent) - with patch.object(module.grp, "getgrgid", return_value=SimpleNamespace(gr_mem=["foreign"])), patch.object(module.pwd, "getpwall", return_value=[role, foreign]), patch.object(module.pwd, "getpwnam", side_effect=lambda name: role if name=="role" else foreign): - with self.assertRaisesRegex(ValueError, "exclusive"): - module.safe_repositories(parent) - with patch.object(module.grp, "getgrgid", side_effect=KeyError("unresolved")): - with self.assertRaisesRegex(ValueError, "unverifiable"): - module.safe_repositories(parent) - self.assertEqual(parent.stat().st_mode & 0o777, 0o775) - - def test_no_clobber_rejects_rerun_before_command_execution(self): - self.assertEqual(self.invoke("run", "--plan", self.plan.name, "--output", self.manifest.name).returncode, 0) - prior=self.manifest.read_bytes() - obj=json.loads(self.plan.read_text()) - marker=self.root / "rerun-executed" - obj["commands"][0]["argv"]=[sys.executable, "-c", "from pathlib import Path; Path("+repr(str(marker))+").write_text('ran')"] - self.plan.write_text(json.dumps(obj)) - result=self.invoke("run", "--plan", self.plan.name, "--output", self.manifest.name) - self.assertEqual(result.returncode, 2) - self.assertIn("already exists", result.stderr) - self.assertEqual(self.manifest.read_bytes(), prior) - self.assertFalse(marker.exists()) - - def test_v2_rejects_forged_issue_binding_and_v1_normal_validation(self): - self.assertEqual(self.invoke("run", "--plan", self.plan.name, "--output", self.manifest.name).returncode, 0) - original=json.loads(self.manifest.read_text()) - for replacement in (2, True, "1"): - obj=json.loads(json.dumps(original)); obj["artifact"]["issue"]=replacement - self.manifest.write_text(json.dumps(obj, sort_keys=True, separators=(",", ":"))+"\n") - self.assertEqual(self.invoke("validate", "--manifest", self.manifest.name).returncode, 2) - original["schema"]="evidence-manifest/v1" - self.manifest.write_text(json.dumps(original, sort_keys=True, separators=(",", ":"))+"\n") - self.assertEqual(self.invoke("validate", "--manifest", self.manifest.name).returncode, 2) - - def test_artifact_boundary_mutation_by_command_rejected_before_write(self): - obj=json.loads(self.plan.read_text()) - obj["commands"][0]["argv"]=[sys.executable, "-c", "from pathlib import Path; Path("+repr(str(self.artifacts))+").chmod(0o755)"] - self.plan.write_text(json.dumps(obj)) - result=self.invoke("run", "--plan", self.plan.name, "--output", self.manifest.name) - self.assertEqual(result.returncode, 2, result.stderr) - self.assertIn("permission", result.stderr) - self.assertFalse(self.manifest.exists()) - - def test_v2_stale_head_rejected_even_when_bound_file_is_unchanged(self): - self.assertEqual(self.invoke("run", "--plan", self.plan.name, "--output", self.manifest.name).returncode, 0) - (self.repo / "other.txt").write_text("new source revision\n") - run_git(self.repo, "add", "other.txt") - run_git(self.repo, "commit", "-qm", "advance head") - result=self.invoke("validate", "--manifest", self.manifest.name) - self.assertEqual(result.returncode, 2, result.stderr) - self.assertIn("source revision is not current HEAD", result.stderr) - - def test_v2_rejects_source_symlink_alias(self): - (self.repo / "alias.txt").symlink_to("input.txt") - run_git(self.repo, "add", "alias.txt") - run_git(self.repo, "commit", "-qm", "source symlink fixture") - obj=json.loads(self.plan.read_text()) - obj["hash_paths"]=["alias.txt"] - self.plan.write_text(json.dumps(obj)) - result=self.invoke("run", "--plan", self.plan.name, "--output", self.manifest.name) - self.assertEqual(result.returncode, 2, result.stderr) - self.assertIn("symlink", result.stderr) - self.assertFalse(self.manifest.exists()) - - def test_external_artifacts_leave_checkout_and_head_unchanged(self): - head = run_git(self.repo, "rev-parse", "HEAD").stdout.strip() - result = self.invoke("run", "--plan", self.plan.name, "--output", self.manifest.name) - self.assertEqual(result.returncode, 0, result.stderr) - data = json.loads(self.manifest.read_text()) - self.assertEqual(data["schema"], "evidence-manifest/v3") - self.assertEqual(data["source_revision"], head) - self.assertEqual(data["artifact"]["run_id"], "run-a") - self.assertEqual(data["artifact"]["issue"], 1) - self.assertEqual(run_git(self.repo, "status", "--porcelain").stdout, "") - self.assertEqual(run_git(self.repo, "rev-parse", "HEAD").stdout.strip(), head) - self.assertEqual(self.manifest.stat().st_mode & 0o777, 0o600) - self.assertEqual(self.invoke("validate", "--manifest", self.manifest.name).returncode, 0) - - -if __name__ == "__main__": unittest.main() diff --git a/harness/tests/impl/test_impl_architecture_docs.py b/harness/tests/impl/test_impl_architecture_docs.py deleted file mode 100644 index 64703b8..0000000 --- a/harness/tests/impl/test_impl_architecture_docs.py +++ /dev/null @@ -1,212 +0,0 @@ -"""Canonical impl contracts stay reachable without duplicated skill bodies.""" -import re -import unittest -from pathlib import Path -from harness.tests.common.impl_contract_docs import read_contract, REQUIRED - -ROOT = Path(__file__).resolve().parents[3] - -class ImplArchitectureDocsTests(unittest.TestCase): - def test_skills_require_single_detailed_workflow_source(self): - for name in ("gitea-issue-implementation", "gitea-issue-rework"): - path = ROOT / "harness/agents/impl/skills" / name / "SKILL.md" - text = path.read_text() - reference = path.parent / "references/workflow.md" - with self.subTest(skill=name): - self.assertIn(REQUIRED, text) - self.assertLess(len(text.splitlines()), 100) - self.assertEqual(text.count("name: " + name), 1) - self.assertNotIn("## Managed-lock broker gate", text) - self.assertFalse(reference.read_text().startswith("---")) - contract = read_contract(path) - self.assertEqual(contract.count("## Managed-lock broker gate"), 1) - for gate in ("## Delegation evidence acceptance gate", "## Test inventory source-binding gate", "## Canonical external verification evidence"): - self.assertEqual(contract.count(gate), 1) - - def test_all_canonical_impl_markdown_links_resolve(self): - for name in ("gitea-issue-implementation", "gitea-issue-rework"): - directory = ROOT / "harness/agents/impl/skills" / name - for path in [directory / "SKILL.md", directory / "references/workflow.md"]: - for link in re.findall(r"\[[^\]]+\]\(([^)]+)\)", path.read_text()): - if link.startswith(("https://", "http://", "#")): - continue - target = link.split("#", 1)[0] - with self.subTest(path=path.name, link=link): - self.assertTrue((path.parent / target).resolve().exists()) - - def test_correction_default_and_exception_remain_distinct(self): - text = (ROOT / "harness/agents/impl/skills/gitea-issue-rework/references/readback.md").read_text() - for token in ("strict", "separately", "atomic_client_precondition=false", "residual", "neutered"): - self.assertIn(token.lower(), text.lower()) - -# The comparison recipe is executable documentation, not a second implementation. -import copy -import json - -COMMIT_FILES_START = "" -COMMIT_FILES_END = "" - - -def _execute_documented_commit_files_comparator(): - """Load the single trusted Python example from the canonical workflow.""" - path = ROOT / "harness/agents/impl/skills/gitea-issue-rework/references/workflow.md" - text = path.read_text() - start = text.count(COMMIT_FILES_START) - end = text.count(COMMIT_FILES_END) - if start != 1 or end != 1: - raise AssertionError("workflow must contain exactly one commit-files example boundary pair") - section = text.split(COMMIT_FILES_START, 1)[1].split(COMMIT_FILES_END, 1)[0] - snippets = re.findall(r"```python\s*\n(.*?)\n```", section, re.DOTALL) - if len(snippets) != 1: - raise AssertionError("commit-files example must contain exactly one Python fenced block") - namespace = {} - exec(compile(snippets[0], str(path) + "#commit-files-example", "exec"), namespace) - comparator = namespace.get("normalize_commit_files") - if not callable(comparator): - raise AssertionError("Python example must define normalize_commit_files(rows)") - heading = "#### commit.files 순서 비교 계약\n" - if text.count(heading) != 1: - raise AssertionError("workflow must contain one scoped commit-files comparison contract") - contract = text.split(heading, 1)[1].split("\n### 7.", 1)[0] - return comparator, contract - - -def _commit(sha, message, parent, files): - return { - "sha": sha, - "commit": {"message": message, "author": {"name": "Test Author", "email": "author@example.invalid"}}, - "url": "https://example.invalid/commit/" + sha, - "parents": [{"sha": parent, "url": "https://example.invalid/commit/" + parent}], - "verification": {"verified": True, "reason": "valid", "arbitrary": [1, {"keep": True}]}, - "files": files, - "provider_extension": {"retained": ["opaque", 7]}, - } - - -def _file(filename, status, additions): - return { - "filename": filename, - "status": status, - "additions": additions, - "deletions": 2, - "changes": additions + 2, - "patch": "@@ -1 +1 @@\n-old\n+new", - "provider_extension": {"nested": [filename, {"opaque": True}]}, - } - - -class DocumentedCommitFilesComparatorTests(unittest.TestCase): - SHA_A = "0123456789abcdef0123456789abcdef01234567" - SHA_B = "89abcdef0123456789abcdef0123456789abcdef" - PARENT_A = "1111111111111111111111111111111111111111" - PARENT_B = "2222222222222222222222222222222222222222" - - def setUp(self): - self.normalize, self.workflow = _execute_documented_commit_files_comparator() - - def fixture(self): - return [ - _commit(self.SHA_A, "first message", self.PARENT_A, [ - _file("src/zeta.py", "modified", 3), _file("docs/alpha.md", "added", 5), - ]), - _commit(self.SHA_B, "second message", self.PARENT_B, [ - _file("tests/test_beta.py", "modified", 7), _file("README.md", "added", 11), - ]), - ] - - def test_inner_file_order_is_ignored_but_outer_commit_order_is_preserved(self): - first = self.fixture() - second = copy.deepcopy(first) - for commit in second: - commit["files"].reverse() - self.assertEqual(self.normalize(first), self.normalize(second)) - self.assertNotEqual(self.normalize(first), self.normalize(list(reversed(second)))) - - def test_identity_and_all_file_metadata_remain_comparison_bound(self): - baseline = self.normalize(self.fixture()) - mutations = ( - lambda rows: rows[0]["files"][0].update(filename="src/renamed.py"), - lambda rows: rows[0]["files"][0].update(status="added"), - lambda rows: rows[0]["files"][0].update(provider_extension={"changed": True}), - lambda rows: rows[0]["files"].append(_file("new.txt", "added", 1)), - lambda rows: rows[0]["files"].pop(), - lambda rows: rows[0].update(commit={"message": "changed message", "author": {"name": "Test Author", "email": "author@example.invalid"}}), - lambda rows: rows[0]["parents"].__setitem__(0, {"sha": self.SHA_B, "url": "https://example.invalid/commit/" + self.SHA_B}), - lambda rows: rows[0].update(sha="3333333333333333333333333333333333333333"), - ) - for mutate in mutations: - with self.subTest(mutation=mutate): - changed = self.fixture() - mutate(changed) - self.assertNotEqual(baseline, self.normalize(changed)) - - def test_normalization_preserves_input_and_returns_independent_deep_copy(self): - rows = self.fixture() - before = json.dumps(rows, sort_keys=True, separators=(",", ":")) - normalized = self.normalize(rows) - self.assertEqual(before, json.dumps(rows, sort_keys=True, separators=(",", ":"))) - self.assertEqual(rows, self.fixture()) - normalized[0]["files"][0]["provider_extension"]["nested"][1]["opaque"] = False - normalized[0]["provider_extension"]["retained"].append("output-only") - self.assertTrue(rows[0]["files"][0]["provider_extension"]["nested"][1]["opaque"]) - self.assertNotIn("output-only", rows[0]["provider_extension"]["retained"]) - - def test_empty_file_list_is_valid_and_order_is_only_normalized_within_commits(self): - rows = self.fixture() - rows[0]["files"] = [] - result = self.normalize(rows) - self.assertEqual([], result[0]["files"]) - self.assertEqual([self.SHA_A, self.SHA_B], [row["sha"] for row in result]) - self.assertEqual(sorted(file["filename"] for file in rows[1]["files"]), [file["filename"] for file in result[1]["files"]]) - - def test_invalid_collection_commit_and_file_shapes_raise_value_error(self): - valid = self.fixture() - invalid_cases = [] - for value in (None, {}, "rows", (valid,)): - invalid_cases.append(value) - invalid_cases.extend(([None], ["commit"], [{"sha": self.SHA_A}], [{"sha": self.SHA_A, "files": None}], [{"sha": self.SHA_A, "files": {}}])) - for bad in invalid_cases: - with self.subTest(rows=bad): - with self.assertRaises(ValueError): - self.normalize(bad) - for bad_sha in ("", "A" * 40, "a" * 39, "g" * 40, None, 7): - with self.subTest(sha=bad_sha): - rows = self.fixture() - rows[0]["sha"] = bad_sha - with self.assertRaises(ValueError): - self.normalize(rows) - duplicate_sha = self.fixture() - duplicate_sha[1]["sha"] = duplicate_sha[0]["sha"] - with self.assertRaises(ValueError): - self.normalize(duplicate_sha) - for bad_file in (None, "file", {}, {"filename": "x"}): - with self.subTest(file=bad_file): - rows = self.fixture() - rows[0]["files"] = [bad_file] - with self.assertRaises(ValueError): - self.normalize(rows) - for key in ("filename", "status"): - for value in (None, 1, ""): - with self.subTest(key=key, value=value): - rows = self.fixture() - rows[0]["files"][0][key] = value - with self.assertRaises(ValueError): - self.normalize(rows) - duplicate_file = self.fixture() - duplicate_file[0]["files"][1]["filename"] = duplicate_file[0]["files"][0]["filename"] - with self.assertRaises(ValueError): - self.normalize(duplicate_file) - - def test_minimal_file_record_and_literal_filename_are_retained(self): - rows = [{"sha": self.SHA_A, "files": [{"filename": " dir/../literal name.txt ", "status": "modified"}]}] - self.assertEqual(rows, self.normalize(rows)) - - def test_document_states_collection_admission_is_a_separate_prerequisite(self): - self.assertIn("complete collection", self.workflow) - self.assertIn("provider scope", self.workflow) - self.assertIn("admission", self.workflow) - self.assertIn("does not prove", self.workflow) - - -if __name__ == "__main__": - unittest.main() diff --git a/harness/tests/impl/test_rework_comment.py b/harness/tests/impl/test_rework_comment.py deleted file mode 100644 index d391263..0000000 --- a/harness/tests/impl/test_rework_comment.py +++ /dev/null @@ -1,183 +0,0 @@ -import hashlib, json, subprocess, sys, tempfile, unittest -from pathlib import Path - -SCRIPT = Path(__file__).parents[2] / "agents/impl/skills/gitea-issue-rework/scripts/rework_comment.py" - -class ReworkCommentTests(unittest.TestCase): - def setUp(self): - self.t = tempfile.TemporaryDirectory(); self.d = Path(self.t.name) - self.data = {"owner":"acme","repository":"widget","issue":7,"pr":12,"review":99, - "reviewed_head":"a"*40,"final_head":"b"*40,"contract_sha256":"c"*64, - "result":"partial","findings":[{"id":"F-1","status":"resolved","classification":"valid_fix","changes":"src/main.py","verification":"pytest -q (0), pass","reason":"bounded fix"}], - "local_checks":"pytest -q (0), pass","runtime_browser":"해당 없음·미검증","commit_status":"success","actions_runs":"없음"} - self.write("input.json", self.data) - self.invoke("render", "--no-delegated-evidence", "--input", self.p("input.json"), "--comment", self.p("comment.md"), "--expected", self.p("expected.json")) - self.expected = json.loads(self.d.joinpath("expected.json").read_text()) - - def tearDown(self): self.t.cleanup() - def p(self,n): return str(self.d/n) - def write(self,n,x): self.d.joinpath(n).write_text(json.dumps(x,ensure_ascii=False) if not isinstance(x,str) else x) - def invoke(self,*args): return subprocess.run([sys.executable,str(SCRIPT),*args],text=True,capture_output=True) - def read(self,n): return json.loads(self.d.joinpath(n).read_text()) - def readback(self, comments=None, head=None): - self.write("pr.json", {"object_kind":"pull_request","owner":"acme","repository":"widget","number":12,"state":"open","head":{"sha":head or self.data["final_head"]},"html_url":"https://gitea/acme/widget/pulls/12","metadata_binding_verified":True}) - self.write("identity.json", {"id":42,"login":"bot","role":"gitea-impl-agent","authenticated":True,"stable_account_verified":True}) - items=comments if comments is not None else [{"id":501,"html_url":"https://gitea/x/501","created_at":"2026-01-01T00:00:00Z","updated_at":"2026-01-01T00:00:00Z","body":self.expected["body"],"user":{"id":42,"login":"bot"}}] - self.write("comments.json", {"object_kind":"pull_request_top_level_comments","owner":"acme","repository":"widget","pr":12,"complete":True,"endpoint_binding_verified":True,"source_issue_unchanged":True,"review_collection_unchanged":True,"pr_metadata_unchanged":True,"page_count":1,"total_count":len(items),"provider_counts":{"top_level":len(items),"reviews":2,"inline":3,"aggregate_comments":len(items),"aggregate_components":["top_level"]},"comments":items}) - def verify(self, action="create", baseline=0): - return self.invoke("verify","--expected",self.p("expected.json"),"--pr-readback",self.p("pr.json"),"--comments-readback",self.p("comments.json"),"--identity-readback",self.p("identity.json"),"--action",action,"--baseline-count",str(baseline),"--output",self.p("receipt.json")) - - def test_render_digest_and_correlation(self): - self.assertEqual(hashlib.sha256(self.expected["marker_free_body"].encode()).hexdigest(), self.expected["result_sha256"]) - parts=["acme","widget","7","12","99","a"*40,"b"*40,"c"*64] - self.assertEqual(hashlib.sha256("\0".join(parts).encode()).hexdigest(), self.expected["correlation"]) - self.assertTrue(self.d.joinpath("comment.md").read_text().endswith("-->\n")) - - def test_no_comment_and_marker_body_mismatch(self): - self.readback([]); self.assertNotEqual(self.verify().returncode,0) - self.readback([{"id":1,"body":self.expected["body"].replace("correlation=","correlation=x"),"user":{"id":42,"login":"bot"}}]); self.assertNotEqual(self.verify().returncode,0) - - def test_wrong_head_and_author(self): - self.readback(head="d"*40); self.assertNotEqual(self.verify().returncode,0) - self.readback(); c=self.read("comments.json"); c["comments"][0]["user"]={"id":43,"login":"bot"}; self.write("comments.json",c); self.assertNotEqual(self.verify().returncode,0) - - def test_digest_and_correlation_tampering_duplicate(self): - self.expected["result_sha256"]="0"*64; self.write("expected.json",self.expected); self.readback(); self.assertNotEqual(self.verify().returncode,0) - self.setUp_render_again(); self.expected["correlation"]="0"*64; self.write("expected.json",self.expected); self.readback(); self.assertNotEqual(self.verify().returncode,0) - self.setUp_render_again(); self.readback(); c=self.read("comments.json"); c["comments"].append(dict(c["comments"][0],id=502)); c["total_count"]=2; self.write("comments.json",c); self.assertNotEqual(self.verify("reuse",1).returncode,0) - - def setUp_render_again(self): - self.invoke("render","--no-delegated-evidence","--input",self.p("input.json"),"--comment",self.p("comment.md"),"--expected",self.p("expected.json")); self.expected=self.read("expected.json") - def test_create_reuse_cardinality(self): - self.readback(); self.assertEqual(self.verify("create",0).returncode,0); self.assertEqual(self.read("receipt.json")["status"],"verified") - original=self.read("comments.json")["comments"][0] - self.readback([original,{"id":2,"html_url":"u","created_at":"t","updated_at":"t","body":"other","user":{"id":42,"login":"bot"}}]); self.assertEqual(self.verify("reuse",2).returncode,0) - def test_exact_success_and_receipt_preserved(self): - self.readback(); self.assertEqual(self.verify().returncode,0) - old=self.d.joinpath("receipt.json").read_bytes(); self.readback(head="e"*40); self.assertNotEqual(self.verify().returncode,0); self.assertEqual(old,self.d.joinpath("receipt.json").read_bytes()) - - def test_unresolved_delivery_and_incomplete_or_wrong_scope_readback_fail(self): - bad=dict(self.data); bad["result"]="rework_delivered"; bad["findings"]=[dict(self.data["findings"][0],status="unresolved")] - self.write("bad-input.json",bad) - self.assertNotEqual(self.invoke("render","--no-delegated-evidence","--input",self.p("bad-input.json"),"--comment",self.p("bad.md"),"--expected",self.p("bad.json")).returncode,0) - self.readback(); comments=self.read("comments.json"); comments["complete"]=False; self.write("comments.json",comments); self.assertNotEqual(self.verify().returncode,0) - self.readback(); pr=self.read("pr.json"); pr["repository"]="other"; self.write("pr.json",pr); self.assertNotEqual(self.verify().returncode,0) - - def test_impl_role_and_provider_aggregate_binding_are_required(self): - self.readback(); identity=self.read("identity.json"); identity["role"]="other"; self.write("identity.json",identity); self.assertNotEqual(self.verify().returncode,0) - self.readback(); comments=self.read("comments.json"); comments["provider_counts"]["aggregate_comments"]+=1; self.write("comments.json",comments); self.assertNotEqual(self.verify().returncode,0) - - def test_render_visible_identifier_lines_do_not_emit_issue_references(self): - body=self.expected["marker_free_body"] - self.assertIn("- Source Issue: 7",body) - self.assertIn("- Pull Request: 12",body) - self.assertIn("- Source Review: 99",body) - self.assertNotIn("- Source Issue: #7",body) - self.assertNotIn("- Pull Request: #12",body) - self.assertNotIn("- Source Review: #99",body) - - def test_render_rejects_reference_hazards_in_every_safe_human_field(self): - hazards=("#123", "otherrepo#123", r"\\#123", "\\#123", "review #99") - fields=("changes","verification","reason","local_checks","runtime_browser") - # Exercise finding fields and global fields independently; references - # must not be emitted even when they resemble another repo or are escaped. - for field in fields: - for snippet in hazards: - with self.subTest(field=field,snippet=snippet): - data=json.loads(json.dumps(self.data)) - if field in ("changes","verification","reason"): - data["findings"][0][field]="unsafe payload "+snippet - else: - data[field]="unsafe payload "+snippet - self.write("hazard.json",data) - result=self.invoke("render","--no-delegated-evidence","--input",self.p("hazard.json"),"--comment",self.p("hazard.md"),"--expected",self.p("hazard-expected.json")) - self.assertNotEqual(result.returncode,0) - - def test_offline_verifier_rejects_reference_in_expected_body(self): - self.expected["marker_free_body"] += "\n- unsafe: otherrepo#123" - before_hash=self.expected["result_sha256"] - self.expected["result_sha256"]=hashlib.sha256(self.expected["marker_free_body"].encode()).hexdigest() - self.expected["marker"]=self.expected["marker"].replace(before_hash,self.expected["result_sha256"]) - self.expected["body"]=self.expected["marker_free_body"]+self.expected["marker"] - self.write("expected.json",self.expected); self.readback() - self.assertNotEqual(self.verify().returncode,0) - - def _reseal(self, value, body): - value=json.loads(json.dumps(value)); old=value["result_sha256"] - value["marker_free_body"]=body - value["result_sha256"]=hashlib.sha256(body.encode()).hexdigest() - value["marker"]=value["marker"].replace(old,value["result_sha256"]) - value["body"]=body+value["marker"] - return value - - def _legacy_expected(self): - old=json.loads(json.dumps(self.expected)) - body=old["marker_free_body"] - body=body.replace("- Source Issue: #7","- Source Issue: 7").replace("- Pull Request: #12","- Pull Request: 12").replace("- Source Review: #99","- Source Review: 99") - # Preserve an explicitly legacy migration fixture; no other old text differs. - body=body.replace("- Source Issue: 7","- Source Issue: #7").replace("- Pull Request: 12","- Pull Request: #12").replace("- Source Review: 99","- Source Review: #99") - old["marker_free_body"]=body - old["result_sha256"]=hashlib.sha256(body.encode()).hexdigest() - old["marker"]=old["marker"].replace(self.expected["result_sha256"],old["result_sha256"]) - old["body"]=body+old["marker"] - return old - - def test_prepare_correction_returns_bounded_verified_legacy_migration_plan(self): - import importlib.util - spec=importlib.util.spec_from_file_location("rework_comment_correction_test",SCRIPT) - module=importlib.util.module_from_spec(spec); spec.loader.exec_module(module) - self.assertTrue(callable(getattr(module,"prepare_correction",None)), "approved correction API missing") - plan=module.prepare_correction(self._legacy_expected(),self.expected,comment_id=501,actor_id=42) - self.assertEqual(plan["schema"],"gitea-rework-correction-plan/v1") - self.assertEqual(plan["target"],self.expected["target"]) - self.assertEqual(plan["comment_id"],501); self.assertEqual(plan["actor_id"],42) - self.assertEqual(plan["old_body"],self._legacy_expected()["body"]) - self.assertEqual(plan["new_body"],self.expected["body"]) - self.assertEqual(plan["old_body_sha256"],hashlib.sha256(plan["old_body"].encode()).hexdigest()) - self.assertEqual(plan["new_body_sha256"],hashlib.sha256(plan["new_body"].encode()).hexdigest()) - self.assertIs(plan["prior_handoff_verified"],False) - self.assertTrue(plan["immutable_source_required"]) - - def test_prepare_correction_rejects_tampered_or_unbounded_changes(self): - import importlib.util - spec=importlib.util.spec_from_file_location("rework_comment_correction_reject_test",SCRIPT) - module=importlib.util.module_from_spec(spec); spec.loader.exec_module(module) - self.assertTrue(callable(getattr(module,"prepare_correction",None)), "approved correction API missing") - old=self._legacy_expected() - cases=[] - bad=json.loads(json.dumps(old)); bad["result_sha256"]="0"*64; cases.append((bad,self.expected)) - bad=json.loads(json.dumps(old)); bad["body"]=bad["body"].replace("#7","#8",1); cases.append((bad,self.expected)) - bad=json.loads(json.dumps(old)); bad["marker_free_body"]=bad["marker_free_body"].replace("bounded fix","unsafe old text #123"); cases.append((bad,self.expected)) - bad=json.loads(json.dumps(self.expected)); bad["marker_free_body"]=bad["marker_free_body"].replace("bounded fix","different extra text"); cases.append((old,bad)) - bad=self._reseal(self.expected,self.expected["marker_free_body"].replace("bounded fix","different extra text")); cases.append((old,bad)) - unsafe_old=self._reseal(old,old["marker_free_body"].replace("bounded fix","same unsafe #123")) - unsafe_new=self._reseal(self.expected,self.expected["marker_free_body"].replace("bounded fix","same unsafe #123")); cases.append((unsafe_old,unsafe_new)) - bad=json.loads(json.dumps(self.expected)); bad["head"]="d"*40; cases.append((old,bad)) - bad=json.loads(json.dumps(self.expected)); bad["correlation"]="e"*64; cases.append((old,bad)) - for previous,new in cases: - with self.subTest(previous=previous["result_sha256"],new=new["correlation"]): - with self.assertRaises(module.Bad): module.prepare_correction(previous,new,comment_id=501,actor_id=42) - - def test_prepare_correction_cli_emits_plan_not_approval_or_receipt(self): - self.write("previous.json",self._legacy_expected()) - r=self.invoke("prepare-correction","--previous-expected",self.p("previous.json"),"--expected",self.p("expected.json"),"--comment-id","501","--actor-id","42","--output",self.p("plan.json")) - self.assertEqual(r.returncode,0,r.stderr) - plan=self.read("plan.json") - self.assertEqual(plan["schema"],"gitea-rework-correction-plan/v1") - self.assertIs(plan["approval_required"],True) - self.assertIs(plan["prior_handoff_verified"],False) - self.assertNotEqual(plan.get("status"),"verified") - self.assertEqual(self.d.joinpath("plan.json").stat().st_mode & 0o777,0o600) - - def test_prepare_correction_cli_refuses_input_alias_and_preserves_file(self): - self.write("previous.json",self._legacy_expected()); before=self.d.joinpath("expected.json").read_bytes() - r=self.invoke("prepare-correction","--previous-expected",self.p("previous.json"),"--expected",self.p("expected.json"),"--comment-id","501","--actor-id","42","--output",self.p("expected.json")) - self.assertNotEqual(r.returncode,0) - self.assertEqual(self.d.joinpath("expected.json").read_bytes(),before) - - def test_different_body_same_correlation_is_not_default_reuse(self): - old=self._legacy_expected() - self.readback([{"id":501,"html_url":"https://gitea/x/501","created_at":"2026-01-01T00:00:00Z","updated_at":"2026-01-01T00:00:00Z","body":old["body"],"user":{"id":42,"login":"bot"}}]) - self.assertNotEqual(self.verify("reuse",1).returncode,0) - -if __name__ == "__main__": unittest.main() diff --git a/harness/tests/impl/test_rework_correction.py b/harness/tests/impl/test_rework_correction.py deleted file mode 100644 index b827ea9..0000000 --- a/harness/tests/impl/test_rework_correction.py +++ /dev/null @@ -1,135 +0,0 @@ -"""Functional correction verification with provider-shaped synthetic readbacks.""" -import copy -import hashlib -import importlib.util -import json -import subprocess -import sys -import tempfile -import unittest -from pathlib import Path -from harness.tests.impl import test_rework_readback as fixtures - -DIRECTORY = Path(__file__).parents[2] / 'agents/impl/skills/gitea-issue-rework/scripts' -spec = importlib.util.spec_from_file_location('correction_verifier', DIRECTORY/'rework_correction.py') -assert spec is not None and spec.loader is not None -mod = importlib.util.module_from_spec(spec) -spec.loader.exec_module(mod) - -class CorrectionVerifierTests(unittest.TestCase): - def setUp(self): - fixture = fixtures.ReworkReadbackTests() - fixture.setUp() - fixture.contract["version"] = "1.27.3" - fixture.overrides["/version"] = ({"version":"1.27.3"}, {}, 200) - self.addCleanup(fixture.doCleanups) - self.new = copy.deepcopy(fixture.expected) - self.old = copy.deepcopy(self.new) - free = self.new['marker_free_body'] - for label in ('Source Issue', 'Pull Request', 'Source Review'): - free = free.replace('- '+label+': ', '- '+label+': #') - digest = hashlib.sha256(free.encode()).hexdigest() - marker = self.new['marker'].replace(self.new['result_sha256'], digest) - self.old.update(body=free+marker, marker=marker, marker_free_body=free, result_sha256=digest) - self.plan = fixture.legacy.prepare_correction(self.old,self.new,comment_id=501,actor_id=42) - fixture.add_result() - fixture.comments[0]['body'] = self.old['body'] - self.before = fixture.adapter().snapshot(self.new) - fixture.comments[0]['body'] = self.new['body'] - fixture.comments[0]['updated_at'] = '2026-01-03T00:00:00Z' - self.after = fixture.adapter().snapshot(self.new) - event = {'id':601,'type':'comment_ref','user':{'id':42,'login':'gitea-impl-agent'}, - 'created_at':fixtures.T1,'updated_at':fixtures.T1,'ref_action':'none', - 'issue_url':fixtures.ORIGIN+'/acme/demo/issues/7', - 'ref_issue':{'number':9,'html_url':fixtures.ORIGIN+'/acme/demo/pulls/9'}, - 'ref_comment':copy.deepcopy(self.before['raw']['comments']['rows'][0])} - self.before['source_issue_timeline'] = self.timeline(event) - changed = copy.deepcopy(event) - changed.update(ref_action='neutered',updated_at='2026-01-03T00:00:00Z', - ref_comment=copy.deepcopy(self.after['raw']['comments']['rows'][0])) - self.after['source_issue_timeline'] = self.timeline(changed) - self.approval = {'approved':True,'operation':'same_comment_correction', - 'historical_failure_baseline_sha256':'1'*64,'old_expected_document':self.old, - 'new_expected_document':self.new,'timeline_profile':'observed-comment-ref-neutralization', - 'ordinary_patch':True,'provider_version':self.before['provider']['version']} - - def timeline(self,event): - return {'complete':True,'rows':[event], 'pages':[{'page':1,'count':1,'headers':{'X-Total-Count':'1'}}]} - - def verify(self,profile='observed-comment-ref-neutralization'): - return mod.verify_correction(self.plan,self.before,self.after,self.old['body'],self.new['body'], - self.approval,approval_sha256=mod.sha(mod.canonical(self.approval)), - historical_failure_sha256='1'*64,profile=profile) - - def test_exact_approved_reference_neutralization_passes(self): - result=self.verify() - self.assertEqual(result['status'],'verified') - self.assertEqual(result['changed_timeline_ids'],[601]) - self.assertEqual(result['historical_handoff_status'],'partial_unchanged') - - def test_strict_unchanged_timeline_passes(self): - self.after['source_issue_timeline']=copy.deepcopy(self.before['source_issue_timeline']) - self.approval['timeline_profile']='strict' - self.assertEqual(self.verify('strict')['status'],'verified') - - def test_reference_profile_requires_separate_approval(self): - self.approval['timeline_profile']='strict' - with self.assertRaises(mod.Bad): self.verify() - - def test_source_and_cache_deltas_fail(self): - for name,field in [('source_issue','title'),('pr','comments'),('pr_issue','comments')]: - original=copy.deepcopy(self.after) - self.after['raw'][name][field] = 99 if field=='comments' else 'changed' - with self.subTest(name=name),self.assertRaises(mod.Bad): self.verify() - self.after=original - - def test_unrelated_timeline_field_change_fails(self): - self.after['source_issue_timeline']['rows'][0]['body']='unexpected' - with self.assertRaises(mod.Bad): self.verify() - - def test_unrelated_null_timeline_field_is_still_a_mutation(self): - self.after['source_issue_timeline']['rows'][0]['unexpected_field'] = None - with self.assertRaises(mod.Bad): self.verify() - - def test_duplicate_and_missing_timeline_rows_fail(self): - original=copy.deepcopy(self.after) - for rows in [[],self.after['source_issue_timeline']['rows']*2]: - self.after=copy.deepcopy(original) - self.after['source_issue_timeline']['rows']=rows - with self.subTest(rows=len(rows)),self.assertRaises(mod.Bad): self.verify() - - def test_wrong_actor_head_and_extra_body_change_fail(self): - for field,value in [('actor_id',99),('head','d'*40),('new_body','arbitrary')]: - original=copy.deepcopy(self.plan) - self.plan[field]=value - with self.subTest(field=field),self.assertRaises(mod.Bad): self.verify() - self.plan=original - - def test_reference_timestamp_cannot_move_backwards(self): - self.after['source_issue_timeline']['rows'][0]['updated_at']='2025-01-01T00:00:00Z' - with self.assertRaises(mod.Bad): self.verify() - - def test_real_cli_valid_inputs_and_no_overwrite(self): - with tempfile.TemporaryDirectory() as td: - root=Path(td) - args=[sys.executable,'-B',str(DIRECTORY/'rework_correction.py'),'verify'] - values={'plan':self.plan,'before':self.before,'after':self.after, - 'old-comment':{'body':self.old['body']},'new-comment':{'body':self.new['body']},'approval':self.approval} - for name,value in values.items(): - raw=json.dumps(value).encode(); path=root/(name+'.json'); path.write_bytes(raw); path.chmod(0o600) - args.extend(['--'+name,str(path),'--'+name+'-sha256',hashlib.sha256(raw).hexdigest()]) - output=root/'receipt.json' - args.extend(['--historical-failure-sha256','1'*64,'--profile','observed-comment-ref-neutralization','--output',str(output)]) - result=subprocess.run(args,capture_output=True,text=True,timeout=10) - self.assertEqual(result.returncode,0,result.stderr) - saved=output.read_bytes() - again=subprocess.run(args,capture_output=True,text=True,timeout=10) - self.assertNotEqual(again.returncode,0) - self.assertEqual(output.read_bytes(),saved) - - def test_digest_tampering_is_rejected(self): - with tempfile.TemporaryDirectory() as td: - path=Path(td)/'data.json';path.write_text('{}') - with self.assertRaises(mod.Bad):mod.load(path,'0'*64) - -if __name__=='__main__':unittest.main() diff --git a/harness/tests/impl/test_rework_input_gate.py b/harness/tests/impl/test_rework_input_gate.py deleted file mode 100644 index d3486cb..0000000 --- a/harness/tests/impl/test_rework_input_gate.py +++ /dev/null @@ -1,579 +0,0 @@ -import importlib.util -import os -import tempfile -import unittest -from pathlib import Path - -SCRIPT = Path(__file__).parents[2] / 'agents/impl/skills/gitea-issue-rework/scripts/rework_input_gate.py' - -class InputGateTests(unittest.TestCase): - def load(self): - spec = importlib.util.spec_from_file_location('rework_input_gate', SCRIPT) - module = importlib.util.module_from_spec(spec) - spec.loader.exec_module(module) - return module - - def test_valid_input_schema_is_accepted(self): - gate = self.load() - value = {'schema':'gitea-agent.rework-input/v1', - 'target':{'owner':'acme','repository':'demo','issue':7,'pr':9}, - 'head':'a'*40, 'contract_sha256':'b'*64} - self.assertEqual(gate.validate_input(value), value) - - def test_initial_output_is_immutable_and_rejected_before_credential_or_get(self): - cli_path = SCRIPT.with_name('rework_readback_cli.py') - spec = importlib.util.spec_from_file_location('rework_readback_cli', cli_path) - cli = importlib.util.module_from_spec(spec) - spec.loader.exec_module(cli) - with tempfile.TemporaryDirectory() as root: - output = Path(root) / 'decision.json' - output.write_bytes(b'old\n') - output.chmod(0o600) - mode = output.stat().st_mode - with self.assertRaisesRegex(ValueError, 'output already exists'): - cli.reject_existing_output(str(output)) - self.assertEqual(output.read_bytes(), b'old\n') - self.assertEqual(output.stat().st_mode, mode) - -class OrderedDecisionTests(unittest.TestCase): - def setUp(self): - import copy, hashlib - from harness.tests.impl import test_rework_readback as rb - from harness.tests.spec import test_implementation_spec_review_support as spec_tests - self.rb = rb.ReworkReadbackTests(); self.rb.setUp(); self.addCleanup(self.rb.doCleanups) - self.rb.repo['default_branch'] = 'main' - self.rb.source['body'] = '\nSynthetic approved source contract.\n' - self.inp = {'schema':'gitea-agent.rework-input/v1', 'target':{'owner':'acme','repository':'demo','issue':7,'pr':9}, - 'head':'b'*40, 'contract_sha256':hashlib.sha256(self.rb.source['body'].encode()).hexdigest()} - self.rows=[] - for role, rid, uid in [('gitea-spec-agent',301,33), ('gitea-review-agent',302,34)]: - row = copy.deepcopy(self.rb.reviews[0]); row.update(id=rid, commit_id=self.inp['head'], state='APPROVED', - user={'id':uid,'login':role}, stale=False, dismissed=False, - html_url=self.rb.pr['html_url']+'#pullreview-'+str(rid)) - if role == 'gitea-spec-agent': - result = spec_tests.result(); result['target'] = dict(self.inp['target']); result['target']['pull'] = result['target'].pop('pr'); result['target']['head']=self.inp['head']; result['contract_sha256']=self.inp['contract_sha256'] - row['body']=spec_tests.mod.render_review(result, {'src/a.py','tests/test_a.py'}, declaration={'mode':'none'}) - else: - row['body']=self.code_body('approve') - self.rows.append(row) - self.set_rows(self.rows) - original = self.rb.request - def request(path): - from urllib.parse import urlsplit - import re - if re.fullmatch(r'/repos/acme/demo/pulls/9/reviews/[1-9][0-9]*/comments', urlsplit(path).path): - self.rb.calls.append(path); return [], {'X-Total-Count':'0'}, 200 - return original(path) - self.rb.request=request - - def code_body(self, verdict): - return '\n- Verdict: `'+verdict+'`\nBounded native code-review fixture.\n' - - def set_rows(self, rows): - import copy - self.rb.reviews=copy.deepcopy(rows) - self.rb.pr['comments']=self.rb.underlying['comments']=self.rb.pr['review_comments']=len(rows) - - def snapshot(self): - return self.rb.adapter().input_snapshot(self.inp) - - def test_real_spec_renderer_approval_is_accepted(self): - receipt=self.snapshot() - self.assertEqual(receipt['status'],'no_op',receipt['reason_codes']) - - def test_fresh_source_contract_digest_is_enforced(self): - self.assertEqual(self.snapshot()['status'], 'no_op') - self.rb.source['body'] += 'Unexpected contract mutation.\n' - with self.assertRaisesRegex(ValueError, 'contract|digest'): - self.snapshot() - - def test_native_state_marker_contradiction_is_blocked(self): - import copy - rows = copy.deepcopy(self.rows) - code = next(row for row in rows if row['user']['login'] == 'gitea-review-agent') - code['body'] = self.code_body('request_changes') - code['state'] = 'APPROVED' - self.set_rows(rows) - receipt = self.snapshot() - self.assertEqual(receipt['status'], 'blocked') - self.assertIn('state_marker_contradiction', receipt['reason_codes']) - - def test_legacy_output_budget_is_unchanged(self): - cli_path = SCRIPT.with_name('rework_readback_cli.py') - spec = importlib.util.spec_from_file_location('legacy_budget_probe', cli_path) - cli = importlib.util.module_from_spec(spec); spec.loader.exec_module(cli) - with tempfile.TemporaryDirectory() as td: - output=Path(td)/'legacy.json' - try: - with cli.OutputFile(output) as writer: writer.write({'data':'x'*1_000_001}) - except ValueError as exc: - self.fail('legacy output below original 16MB budget must remain accepted: '+str(exc)) - self.assertGreater(output.stat().st_size, 1_000_000) - - def test_mixed_roles_permutations_require_rework_without_raw_mutation(self): - import copy - rows=copy.deepcopy(self.rows); rows[1].update(state='REQUEST_CHANGES',body=self.code_body('request_changes')) - for ordered in (rows, list(reversed(rows))): - self.set_rows(ordered); before=copy.deepcopy(self.rb.reviews) - receipt=self.snapshot() - self.assertEqual(receipt['status'],'rework',receipt['reason_codes']) - self.assertEqual(receipt['decision']['selected_review_ids'], {'gitea-spec-agent':301,'gitea-review-agent':302}) - self.assertEqual(self.rb.reviews,before) - self.assertEqual(receipt['raw_input_snapshot']['reviews']['rows'], before) - self.assertEqual([x['id'] for x in receipt['decision']['timeline']], [301,302]) - - def test_only_same_role_later_approval_supersedes_request(self): - import copy - rows=copy.deepcopy(self.rows); rows[1].update(state='REQUEST_CHANGES',body=self.code_body('request_changes')) - self.set_rows(rows); self.assertEqual(self.snapshot()['status'],'rework') - later=copy.deepcopy(self.rows[1]);later.update(id=303,submitted_at='2026-01-02T00:00:00Z',updated_at='2026-01-02T00:00:00Z',html_url=self.rb.pr['html_url']+'#pullreview-303') - self.set_rows([later,*rows]); receipt=self.snapshot() - self.assertEqual(receipt['status'],'no_op',receipt['reason_codes']) - self.assertEqual(receipt['decision']['selected_review_ids']['gitea-review-agent'],303) - - def test_missing_stale_and_dismissed_approvals_are_not_noop_eligible(self): - import copy - self.assertEqual(self.snapshot()['status'],'no_op') - for rows in ([self.rows[0]], [self.rows[1]], []): - self.set_rows(rows); self.assertEqual(self.snapshot()['status'],'blocked') - for field in ('stale','dismissed'): - rows=copy.deepcopy(self.rows);rows[1][field]=True;self.set_rows(rows) - self.assertEqual(self.snapshot()['status'],'blocked') - - def test_current_pending_and_manual_comments_never_allow_noop(self): - import copy - self.assertEqual(self.snapshot()['status'],'no_op') - for state in ('PENDING','COMMENT'): - rows=copy.deepcopy(self.rows);rows[1]['state']=state;self.set_rows(rows) - self.assertEqual(self.snapshot()['status'],'blocked') - - def test_blocked_spec_comment_dominates_code_request_changes(self): - import copy - from harness.tests.spec import test_implementation_spec_review_support as spec_tests - result=spec_tests.result('blocked_spec');result['target']={'owner':'acme','repository':'demo','issue':7,'pull':9,'head':self.inp['head']};result['contract_sha256']=self.inp['contract_sha256'];result['criteria'][0]['result']='indeterminate';result['scope']['spec_conflicts']=['src/a.py'] - rows=copy.deepcopy(self.rows);rows[0].update(state='COMMENT',body=spec_tests.mod.render_review(result,{'src/a.py','tests/test_a.py'},declaration={'mode':'none'}));rows[1].update(state='REQUEST_CHANGES',body=self.code_body('request_changes')) - self.set_rows(rows);receipt=self.snapshot();self.assertEqual(receipt['status'],'blocked') - self.assertIn('blocked_spec',receipt['reason_codes']) - - def test_marker_contract_and_target_defects_block_rework(self): - import copy - self.assertEqual(self.snapshot()['status'],'no_op') - for body in ('missing marker',self.code_body('approve').replace('issue=7','issue=8'),self.code_body('approve').replace(self.inp['contract_sha256'],'f'*64)): - rows=copy.deepcopy(self.rows);rows[1]['body']=body;self.set_rows(rows) - self.assertEqual(self.snapshot()['status'],'blocked') - - def test_unknown_role_cannot_replace_authoritative_code_review(self): - import copy - self.assertEqual(self.snapshot()['status'],'no_op') - unknown=copy.deepcopy(self.rows[1]);unknown['user']={'id':99,'login':'reader'} - self.set_rows([self.rows[0],unknown]);receipt=self.snapshot() - self.assertEqual(receipt['status'],'blocked');self.assertIn('unknown_role',receipt['decision']['timeline'][1]['reason_codes']) - - def test_native_rows_source_marker_and_default_branch_fail_closed(self): - import copy - self.assertEqual(self.snapshot()['status'],'no_op') - for field,bad in [('id',True),('state','UNRECOGNIZED'),('stale','false'),('dismissed',None),('submitted_at','not-a-timestamp')]: - rows=copy.deepcopy(self.rows);rows[1][field]=bad;self.set_rows(rows) - with self.subTest(field=field),self.assertRaises(ValueError):self.snapshot() - self.set_rows(self.rows) - original=self.rb.source['body'];self.rb.source['body']=original.replace(' -->',' --> suffix',1) - import hashlib - self.inp['contract_sha256']=hashlib.sha256(self.rb.source['body'].encode()).hexdigest() - with self.assertRaisesRegex(ValueError,'discovery'):self.snapshot() - self.rb.source['body']=original;self.inp['contract_sha256']=hashlib.sha256(original.encode()).hexdigest();self.rb.pr['base']['ref']='other' - with self.assertRaisesRegex(ValueError,'default branch'):self.snapshot() - - def test_duplicate_review_ids_and_capture_drift_fail_closed(self): - import copy - self.assertEqual(self.snapshot()['status'],'no_op') - self.set_rows([*self.rows,copy.deepcopy(self.rows[1])]) - with self.assertRaisesRegex(ValueError,'duplicate'):self.snapshot() - self.set_rows(self.rows);old=self.rb.request;users=[0] - def request(path): - if path=='/user': - users[0]+=1 - if users[0]==2:self.rb.source['title']='Changed during second capture' - return old(path) - self.rb.request=request - with self.assertRaisesRegex(ValueError,'concurrent provider drift'):self.snapshot() - self.assertEqual(users[0],2) - - def test_timezone_and_nanosecond_order_is_exact(self): - import copy - rows=copy.deepcopy(self.rows);rows[1].update(state='REQUEST_CHANGES',body=self.code_body('request_changes'),submitted_at='2026-01-01T00:00:00.000000001Z',updated_at='2026-01-01T00:00:00.000000001Z') - later=copy.deepcopy(self.rows[1]);later.update(id=303,submitted_at='2026-01-01T01:00:00.000000002+01:00',updated_at='2026-01-01T01:00:00.000000002+01:00',html_url=self.rb.pr['html_url']+'#pullreview-303') - self.set_rows([later,*rows]);receipt=self.snapshot();self.assertEqual(receipt['status'],'no_op');self.assertEqual(receipt['decision']['selected_review_ids']['gitea-review-agent'],303) - - def test_real_public_cli_runs_gate_without_selected_review(self): - import hashlib,json,sys - from unittest.mock import patch as mock - def load(name,path): - spec=importlib.util.spec_from_file_location(name,path);mod=importlib.util.module_from_spec(spec);spec.loader.exec_module(mod);return mod - cli=load('gate_cli_test',SCRIPT.with_name('rework_readback_cli.py'));command=load('gate_command_test',SCRIPT.with_name('rework_comment.py')) - with tempfile.TemporaryDirectory() as td: - root=Path(td);target=root/'target.json';provider=root/'provider.json';output=root/'decision.json' - target.write_text(json.dumps(self.inp));provider.write_text(json.dumps(self.rb.contract));target.chmod(0o600);provider.chmod(0o600) - args=['rework_comment.py','input-gate','--target',str(target),'--target-sha256',hashlib.sha256(target.read_bytes()).hexdigest(),'--provider-contract',str(provider),'--provider-contract-sha256',hashlib.sha256(provider.read_bytes()).hexdigest(),'--actor-id','42','--output',str(output)] - with mock.dict(sys.modules,{'rework_readback_cli':cli}),mock.object(sys,'argv',args),mock.dict(os.environ,{'HERMES_HOME':td}),mock.object(cli,'load_role_token',return_value='PUBLIC'+'-fixture-value') as loader,mock.object(cli,'RestTransport',return_value=self.rb.request): - self.assertEqual(command.main(),0);self.assertTrue(loader.called) - self.assertEqual(json.loads(output.read_text())['status'],'no_op');self.assertEqual(output.stat().st_mode&0o777,0o600) - import copy - rows=copy.deepcopy(self.rows);rows[1].update(state='REQUEST_CHANGES',body=self.code_body('request_changes')) - self.set_rows(list(reversed(rows)));output=root/'rework-decision.json';args[-1]=str(output) - with mock.dict(sys.modules,{'rework_readback_cli':cli}),mock.object(sys,'argv',args),mock.dict(os.environ,{'HERMES_HOME':td}),mock.object(cli,'load_role_token',return_value='PUBLIC'+'-fixture-value'),mock.object(cli,'RestTransport',return_value=self.rb.request): - self.assertEqual(command.main(),0) - actual=json.loads(output.read_text());self.assertEqual(actual['status'],'rework');self.assertEqual(actual['decision']['rework_review_ids'],[302]) - saved=output.read_bytes() - with mock.dict(sys.modules,{'rework_readback_cli':cli}),mock.object(sys,'argv',args),mock.object(cli,'load_role_token') as loader,mock.object(cli,'RestTransport') as transport: - self.assertEqual(command.main(),2);loader.assert_not_called();transport.assert_not_called() - self.assertEqual(output.read_bytes(),saved) - - def test_initial_output_atomic_publish_preserves_race_created_file(self): - from unittest.mock import patch as mock - spec=importlib.util.spec_from_file_location('no_clobber_test',SCRIPT.with_name('rework_readback_cli.py'));cli=importlib.util.module_from_spec(spec);spec.loader.exec_module(cli) - with tempfile.TemporaryDirectory() as td: - output=Path(td)/'decision.json';original_link=cli.os.link - with cli.OutputFile(output,initial_only=True) as writer: - def race_link(*args,**kwargs): - output.write_bytes(b'raced output');output.chmod(0o600);return original_link(*args,**kwargs) - with mock.object(cli.os,'link',side_effect=race_link),self.assertRaisesRegex(ValueError,'created during capture'):writer.write({'decision':'bounded'}) - self.assertEqual(output.read_bytes(),b'raced output');self.assertEqual(output.stat().st_mode&0o777,0o600) - self.assertEqual([x.name for x in Path(td).iterdir()],['decision.json']) - - def test_initial_output_total_byte_budget_rejects_without_artifact(self): - spec=importlib.util.spec_from_file_location('size_gate_test',SCRIPT.with_name('rework_readback_cli.py'));cli=importlib.util.module_from_spec(spec);spec.loader.exec_module(cli) - with tempfile.TemporaryDirectory() as td: - output=Path(td)/'decision.json' - with cli.OutputFile(output,initial_only=True) as writer,self.assertRaisesRegex(ValueError,'output size'):writer.write({'raw':'x'*1_000_001}) - self.assertFalse(output.exists()) - - def test_old_head_state_contradiction_is_history_only(self): - import copy - self.assertEqual(self.snapshot()['status'],'no_op') - old=copy.deepcopy(self.rows[1]);old.update(id=303,commit_id='a'*40, - body=self.code_body('request_changes').replace(self.inp['head'],'a'*40), - html_url=self.rb.pr['html_url']+'#pullreview-303') - self.set_rows([old,*self.rows]);receipt=self.snapshot() - self.assertEqual(receipt['status'],'no_op',receipt['reason_codes']) - history=next(x for x in receipt['decision']['timeline'] if x['id']==303) - self.assertIn('old_head',history['reason_codes']);self.assertFalse(history['eligible']) - - def test_same_instant_body_conflict_has_no_selected_review(self): - import copy - self.assertEqual(self.snapshot()['status'],'no_op') - other=copy.deepcopy(self.rows[1]);other.update(id=303,body=other['body']+'Different finding text.\n',html_url=self.rb.pr['html_url']+'#pullreview-303') - self.set_rows([other,*self.rows]);receipt=self.snapshot() - self.assertEqual(receipt['status'],'blocked');self.assertIn('conflicting_same_time_reviews',receipt['reason_codes']) - self.assertIsNone(receipt['decision']['selected_review_ids']['gitea-review-agent']) - - def test_ambiguous_code_marker_or_verdict_lines_block(self): - import copy - self.assertEqual(self.snapshot()['status'],'no_op') - for suffix in ('- Verdict: malformed\n','Quoted marker: '+self.code_body('approve').splitlines()[0]+'\n'): - rows=copy.deepcopy(self.rows);rows[1]['body']+=suffix;self.set_rows(rows) - with self.subTest(suffix=suffix):self.assertEqual(self.snapshot()['status'],'blocked') - - def test_invalid_timezone_minute_is_rejected(self): - import copy - self.assertEqual(self.snapshot()['status'],'no_op') - rows=copy.deepcopy(self.rows);rows[1]['submitted_at']='2026-01-01T00:00:00+00:99';self.set_rows(rows) - with self.assertRaisesRegex(ValueError,'timestamp|submitted_at'):self.snapshot() - - def test_rework_targets_exclude_spec_approvals(self): - import copy - self.assertEqual(self.snapshot()['status'],'no_op') - rows=copy.deepcopy(self.rows);rows[1].update(state='REQUEST_CHANGES',body=self.code_body('request_changes')) - self.set_rows(rows);receipt=self.snapshot();self.assertEqual(receipt['status'],'rework') - self.assertEqual(receipt['decision'].get('rework_review_ids'),[302]) - - def test_cli_preflight_rejects_bool_target_before_credentials(self): - import hashlib,json,sys,copy - from unittest.mock import patch as mock - spec=importlib.util.spec_from_file_location('cli_preflight_test',SCRIPT.with_name('rework_readback_cli.py'));cli=importlib.util.module_from_spec(spec);spec.loader.exec_module(cli) - spec=importlib.util.spec_from_file_location('cmd_preflight_test',SCRIPT.with_name('rework_comment.py'));cmd=importlib.util.module_from_spec(spec);spec.loader.exec_module(cmd) - with tempfile.TemporaryDirectory() as td: - root=Path(td);target=root/'target.json';provider=root/'provider.json';output=root/'decision.json' - bad=copy.deepcopy(self.inp);bad['target']['issue']=True;target.write_text(json.dumps(bad));provider.write_text(json.dumps(self.rb.contract));target.chmod(0o600);provider.chmod(0o600) - args=['command','input-gate','--target',str(target),'--target-sha256',hashlib.sha256(target.read_bytes()).hexdigest(),'--provider-contract',str(provider),'--provider-contract-sha256',hashlib.sha256(provider.read_bytes()).hexdigest(),'--actor-id','42','--output',str(output)] - with mock.dict(sys.modules,{'rework_readback_cli':cli}),mock.object(sys,'argv',args),mock.object(cli,'load_role_token') as loader,mock.object(cli,'RestTransport') as transport: - self.assertEqual(cmd.main(),2);loader.assert_not_called();transport.assert_not_called() - self.assertFalse(output.exists()) - - def test_cli_capture_race_preserves_foreign_output(self): - import hashlib,json,sys - from unittest.mock import patch as mock - spec=importlib.util.spec_from_file_location('cli_race_test',SCRIPT.with_name('rework_readback_cli.py'));cli=importlib.util.module_from_spec(spec);spec.loader.exec_module(cli) - spec=importlib.util.spec_from_file_location('cmd_race_test',SCRIPT.with_name('rework_comment.py'));cmd=importlib.util.module_from_spec(spec);spec.loader.exec_module(cmd) - with tempfile.TemporaryDirectory() as td: - root=Path(td);target=root/'target.json';provider=root/'provider.json';output=root/'decision.json' - target.write_text(json.dumps(self.inp));provider.write_text(json.dumps(self.rb.contract));target.chmod(0o600);provider.chmod(0o600) - args=['command','input-gate','--target',str(target),'--target-sha256',hashlib.sha256(target.read_bytes()).hexdigest(),'--provider-contract',str(provider),'--provider-contract-sha256',hashlib.sha256(provider.read_bytes()).hexdigest(),'--actor-id','42','--output',str(output)] - requests=[] - def transport(path): - requests.append(path) - if len(requests)==1:output.write_bytes(b'created concurrently');output.chmod(0o600) - return self.rb.request(path) - with mock.dict(sys.modules,{'rework_readback_cli':cli}),mock.object(sys,'argv',args),mock.dict(os.environ,{'HERMES_HOME':td}),mock.object(cli,'load_role_token',return_value='PUBLIC'+'-fixture-value'),mock.object(cli,'RestTransport',return_value=transport): - self.assertEqual(cmd.main(),2) - self.assertGreater(len(requests),1);self.assertEqual(output.read_bytes(),b'created concurrently');self.assertEqual(output.stat().st_mode&0o777,0o600) - - - def test_pr_comment_counter_mismatch_preserves_no_op_and_diagnostics(self): - self.assertEqual(self.snapshot()['status'], 'no_op') - self.rb.pr['comments'] += 4 - receipt = self.snapshot() - self.assertEqual(receipt['status'], 'no_op') - diagnostics = receipt['counter_diagnostics'] - self.assertEqual(set(diagnostics), {'inline', 'pr', 'pr_issue'}) - self.assertEqual(diagnostics['pr']['status'], 'mismatch') - - def test_review_counter_mismatch_preserves_rework(self): - import copy - rows = copy.deepcopy(self.rows) - rows[1].update(state='REQUEST_CHANGES', body=self.code_body('request_changes')) - self.set_rows(rows) - self.rb.pr['review_comments'] += 3 - receipt = self.snapshot() - self.assertEqual(receipt['status'], 'rework') - self.assertEqual(receipt['counter_diagnostics']['inline']['status'], 'mismatch') - self.assertEqual(receipt['decision']['rework_review_ids'], [302]) - - def test_malformed_pr_counters_are_unverified_diagnostics(self): - self.rb.pr['comments'] = 'not-a-count' - receipt = self.snapshot() - self.assertEqual(receipt['status'], 'no_op') - self.assertEqual(receipt['counter_diagnostics']['pr']['status'], 'unverified') - self.assertIn('field', receipt['counter_diagnostics']['pr']) - - def test_unavailable_inline_endpoint_is_rejected_even_with_pr_count_fallback(self): - from urllib.parse import urlsplit - original = self.rb.request - def unavailable(path): - if '/pulls/9/reviews/' in urlsplit(path).path and urlsplit(path).path.endswith('/comments'): - self.rb.calls.append(path) - return {'message': 'not available'}, {}, 404 - return original(path) - self.rb.request = unavailable - self.rb.contract['inline']['mode'] = 'per_review_or_pr_count' - with self.assertRaisesRegex(ValueError, 'inline collection unavailable'): - self.snapshot() - - def test_inline_endpoint_405_is_rejected_even_with_pr_count_fallback(self): - from urllib.parse import urlsplit - original = self.rb.request - def unavailable(path): - if '/pulls/9/reviews/' in urlsplit(path).path and urlsplit(path).path.endswith('/comments'): - self.rb.calls.append(path) - return {'message': 'method not allowed'}, {}, 405 - return original(path) - self.rb.request = unavailable - self.rb.contract['inline']['mode'] = 'per_review_or_pr_count' - with self.assertRaisesRegex(ValueError, 'inline collection unavailable'): - self.snapshot() - - def test_publication_snapshot_remains_strict_on_pr_counter_mismatch(self): - from harness.tests.impl.test_rework_readback import ReworkReadbackTests - publication = ReworkReadbackTests() - publication.setUp() - self.addCleanup(publication.doCleanups) - publication.adapter().snapshot(publication.expected) - publication.pr['comments'] += 1 - with self.assertRaisesRegex(ValueError, 'provider aggregate count mismatch'): - publication.adapter().snapshot(publication.expected) - - def test_complete_lists_with_zero_pr_issue_counter_preserve_no_op(self): - self.rb.pr['comments'] = self.rb.underlying['comments'] = 0 - receipt = self.snapshot() - self.assertEqual(receipt['status'], 'no_op') - for scope in ('pr', 'pr_issue'): - diagnostic = receipt['counter_diagnostics'][scope] - self.assertEqual(diagnostic, {'field':'comments', - 'components':['top_level','reviews','inline'], 'observed':0, - 'expected':2, 'status':'mismatch'}) - self.assertEqual(receipt['raw_input_snapshot'][scope]['comments'], 0) - - def test_missing_and_invalid_counters_do_not_replace_record_validation(self): - import copy - saved_pr, saved_issue = copy.deepcopy(self.rb.pr), copy.deepcopy(self.rb.underlying) - for scope, field in (('inline','review_comments'), ('pr','comments'), ('pr_issue','comments')): - for value in (None, True, -1, '0', 0.5, 'missing'): - with self.subTest(scope=scope, value=value): - self.rb.pr, self.rb.underlying = copy.deepcopy(saved_pr), copy.deepcopy(saved_issue) - record = self.rb.underlying if scope == 'pr_issue' else self.rb.pr - if value == 'missing': record.pop(field) - else: record[field] = value - receipt = self.snapshot() - self.assertEqual(receipt['status'], 'no_op') - self.assertEqual(receipt['counter_diagnostics'][scope]['status'], 'unverified') - raw_record = receipt['raw_input_snapshot']['pr_issue' if scope == 'pr_issue' else 'pr'] - if value == 'missing': self.assertNotIn(field, raw_record) - else: self.assertEqual(raw_record[field], value) - - def test_counter_mismatch_does_not_unblock_lone_approval(self): - self.set_rows(self.rows[:1]) - self.rb.pr['comments'] = self.rb.underlying['comments'] = 0 - receipt = self.snapshot() - self.assertEqual(receipt['status'], 'blocked') - self.assertEqual(receipt['counter_diagnostics']['pr']['status'], 'mismatch') - self.assertEqual(receipt['decision']['rework_review_ids'], []) - - def spec_body(self, verdict, head=None): - from harness.tests.spec import test_implementation_spec_review_support as spec_tests - result = spec_tests.result(verdict) - result['target'] = {'owner':'acme','repository':'demo','issue':7,'pull':9, - 'head':head or self.inp['head']} - result['contract_sha256'] = self.inp['contract_sha256'] - if verdict == 'request_changes': - result['criteria'][0].update(result='unsatisfied', reason='Bounded regression finding.') - result['conclusion'] = 'Bounded correction required.' - return spec_tests.mod.render_review(result, {'src/a.py','tests/test_a.py'}, declaration={'mode':'none'}) - - def test_lone_spec_request_changes_allows_rework(self): - rows = [dict(self.rows[0], state='REQUEST_CHANGES', body=self.spec_body('request_changes'))] - self.set_rows(rows) - receipt = self.snapshot() - self.assertEqual(receipt['status'], 'rework', receipt['reason_codes']) - self.assertEqual(receipt['decision']['selected_review_ids']['gitea-spec-agent'], 301) - self.assertIsNone(receipt['decision']['selected_review_ids']['gitea-review-agent']) - self.assertEqual(receipt['decision']['rework_review_ids'], [301]) - self.assertIn('missing_current_review', receipt['reason_codes']) - - def test_lone_code_request_changes_allows_rework(self): - import copy - row = copy.deepcopy(self.rows[1]) - row.update(state='REQUEST_CHANGES', body=self.code_body('request_changes')) - self.set_rows([row]) - receipt = self.snapshot() - self.assertEqual(receipt['status'], 'rework', receipt['reason_codes']) - self.assertIsNone(receipt['decision']['selected_review_ids']['gitea-spec-agent']) - self.assertEqual(receipt['decision']['selected_review_ids']['gitea-review-agent'], 302) - self.assertEqual(receipt['decision']['rework_review_ids'], [302]) - self.assertIn('missing_current_review', receipt['reason_codes']) - - def test_valid_request_changes_survives_invalid_counterpart_history(self): - import copy - for request_index in (0, 1): - for excluded in ('stale', 'dismissed', 'old_head'): - with self.subTest(request_role=request_index, excluded=excluded): - rows = copy.deepcopy(self.rows) - request = rows[request_index] - request.update(state='REQUEST_CHANGES', body=self.spec_body('request_changes') - if request_index == 0 else self.code_body('request_changes')) - other = rows[1-request_index] - if excluded == 'old_head': - other.update(commit_id='a'*40, body=self.spec_body('approve', head='a'*40) - if request_index == 1 else self.code_body('approve').replace(self.inp['head'], 'a'*40)) - else: other[excluded] = True - self.set_rows(rows) - receipt = self.snapshot() - self.assertEqual(receipt['status'], 'rework', receipt['reason_codes']) - self.assertEqual(receipt['decision']['rework_review_ids'], [request['id']]) - self.assertIsNone(receipt['decision']['selected_review_ids'][other['user']['login']]) - self.assertIn('missing_current_review', receipt['reason_codes']) - - def test_excluded_lone_requests_cannot_authorize_rework(self): - import copy - for request_index in (0, 1): - for excluded in ('stale', 'dismissed', 'old_head'): - with self.subTest(request_role=request_index, excluded=excluded): - request = copy.deepcopy(self.rows[request_index]) - head = 'a'*40 if excluded == 'old_head' else self.inp['head'] - request.update(state='REQUEST_CHANGES', commit_id=head, - body=self.spec_body('request_changes', head=head) if request_index == 0 - else self.code_body('request_changes').replace(self.inp['head'], head)) - if excluded != 'old_head': request[excluded] = True - self.set_rows([request]) - receipt = self.snapshot() - self.assertEqual(receipt['status'], 'blocked') - self.assertEqual(receipt['decision']['rework_review_ids'], []) - - def test_superseded_request_with_missing_counterpart_stays_blocked(self): - import copy - request = copy.deepcopy(self.rows[1]) - request.update(state='REQUEST_CHANGES', body=self.code_body('request_changes')) - approval = copy.deepcopy(self.rows[1]) - approval.update(id=303, submitted_at='2026-01-02T00:00:00Z', updated_at='2026-01-02T00:00:00Z', - html_url=self.rb.pr['html_url'] + '#pullreview-303') - self.set_rows([request, approval]) - receipt = self.snapshot() - self.assertEqual(receipt['status'], 'blocked', receipt['reason_codes']) - self.assertNotEqual(receipt['status'], 'rework') - self.assertNotEqual(receipt['status'], 'no_op') - self.assertEqual(receipt['decision']['rework_review_ids'], []) - - def test_true_blockers_override_valid_request_changes(self): - import copy - request = copy.deepcopy(self.rows[1]) - request.update(state='REQUEST_CHANGES', body=self.code_body('request_changes')) - for state in ('COMMENT', 'PENDING'): - with self.subTest(state=state): - rows = copy.deepcopy(self.rows) - rows[1] = copy.deepcopy(request) - rows[0]['state'] = state - self.set_rows(rows) - receipt = self.snapshot() - self.assertEqual(receipt['status'], 'blocked') - self.assertIn('comment_manual_classification' if state == 'COMMENT' - else 'pending_authoritative_review', receipt['reason_codes']) - rows = copy.deepcopy(self.rows) - rows[1] = request - rows[0]['body'] = 'missing marker' - self.set_rows(rows) - self.assertEqual(self.snapshot()['status'], 'blocked') - rows = copy.deepcopy(self.rows) - rows[1] = request - rows[0]['body'] = rows[0]['body'].replace(self.inp['contract_sha256'], 'f'*64) - self.set_rows(rows) - self.assertEqual(self.snapshot()['status'], 'blocked') - rows = copy.deepcopy(self.rows) - result = __import__('harness.tests.spec.test_implementation_spec_review_support', fromlist=['result']) - spec_result = result.result('blocked_spec') - spec_result['target'] = {'owner':'acme','repository':'demo','issue':7,'pull':9,'head':self.inp['head']} - spec_result['contract_sha256'] = self.inp['contract_sha256'] - spec_result['criteria'][0]['result'] = 'indeterminate' - spec_result['scope']['spec_conflicts'] = ['src/a.py'] - rows[0].update(state='COMMENT', body=result.mod.render_review(spec_result, {'src/a.py','tests/test_a.py'}, declaration={'mode':'none'})) - rows[1] = request - self.set_rows(rows) - receipt = self.snapshot() - self.assertEqual(receipt['status'], 'blocked') - self.assertIn('blocked_spec', receipt['reason_codes']) - - def test_lone_request_changes_keeps_rework_despite_counter_mismatch(self): - import copy - request = copy.deepcopy(self.rows[1]) - request.update(state='REQUEST_CHANGES', body=self.code_body('request_changes')) - self.set_rows([request]) - self.rb.pr['comments'] += 2 - receipt = self.snapshot() - self.assertEqual(receipt['status'], 'rework', receipt['reason_codes']) - self.assertEqual(receipt['decision']['rework_review_ids'], [302]) - self.assertEqual(receipt['counter_diagnostics']['pr']['status'], 'mismatch') - - def test_counter_mismatch_does_not_hide_per_review_inline_count_mismatch(self): - self.rb.reviews[0]['comments_count'] = 1 - self.rb.pr['comments'] = self.rb.underlying['comments'] = 0 - with self.assertRaisesRegex(ValueError, 'per-review inline count mismatch'): - self.snapshot() - - def test_counter_mismatch_does_not_hide_duplicate_review_records(self): - import copy - self.rb.reviews.append(copy.deepcopy(self.rb.reviews[0])) - self.rb.pr['comments'] = self.rb.underlying['comments'] = 0 - with self.assertRaisesRegex(ValueError, 'duplicate'): - self.snapshot() - - def test_counter_mismatch_does_not_hide_incomplete_review_pagination(self): - original = self.rb.request - def truncated(path): - if '/pulls/9/reviews?' in path: - payload, headers, status = original(path) - return payload, dict(headers, **{'X-Total-Count':'3'}), status - return original(path) - self.rb.request = truncated - self.rb.pr['comments'] = self.rb.underlying['comments'] = 0 - with self.assertRaisesRegex(ValueError, 'truncated collection'): - self.snapshot() - -if __name__ == '__main__': - unittest.main() diff --git a/harness/tests/impl/test_rework_publication.py b/harness/tests/impl/test_rework_publication.py deleted file mode 100644 index c401d84..0000000 --- a/harness/tests/impl/test_rework_publication.py +++ /dev/null @@ -1,163 +0,0 @@ -"""Regression tests for main-owned rework publication integration.""" -import copy -import hashlib -import sys -import unittest -from pathlib import Path -from unittest.mock import Mock - -ROOT = Path(__file__).resolve().parents[2] -SCRIPT = ROOT / "agents/impl/skills/gitea-issue-rework/scripts" -sys.path.insert(0, str(SCRIPT)) -sys.path.insert(0, str(ROOT)) - -import rework_publication as pub -from harness.lib import impl_operation as ops - -HEAD = "a" * 40 - -def expected(): - # Generate through the canonical renderer to avoid hand-authored bodies. - import rework_comment as rc - d = dict(owner="o", repository="r", issue=4, pr=5, review=6, - reviewed_head="b"*40, final_head=HEAD, contract_sha256="c"*64, - result="partial", findings=[dict(id="F1", status="resolved", classification="valid_fix", - changes="changed safely", verification="checks pass", reason="verified")], - local_checks="pass", runtime_browser="not run", commit_status="없음", actions_runs="없음") - dd, vals, fs, extras, cs, ar = rc.canon(d) - body = rc.render_body(dd, vals, fs, extras, cs, ar) - rh = hashlib.sha256(body.encode()).hexdigest() - corr = hashlib.sha256("\0".join(["o", "r", "4", "5", "6", "b"*40, HEAD, "c"*64]).encode()).hexdigest() - marker = f"\n" - return dict(body=body+marker, marker=marker, marker_free_body=body, result_sha256=rh, correlation=corr, - target=dict(owner="o", repository="r", issue=4, pr=5), review=6, - reviewed_head="b"*40, head=HEAD, contract_sha256="c"*64) - -def setup_case(op="comment.create", payload=None, approval_flags=None): - if payload is None: - e=expected(); payload={"expected":e,"target":e["target"]} - intent=ops.Intent("v1", op, "o/r", 5, HEAD, payload) - approval=ops.Approval("v1", intent.digest, "approved", **(approval_flags or {})) - declaration={"mode":"none"} - calls=[] - adapter=ops.Adapter(preflight=lambda i: {"fresh":True,"scope":"exact"}, capability=True, - mutate=lambda i: calls.append(copy.deepcopy(i)) or {"acknowledgement":"ok"}, - readback=lambda i: {"scope":"exact","head":HEAD}, verify=lambda i,p,o: True) - return intent,approval,declaration,calls,adapter - -class PublicationTests(unittest.TestCase): - def test_entrypoint_is_present_not_import_error(self): - self.assertTrue(callable(getattr(pub,"execute_rework_operation",None))) - - def test_invalid_body_target_head_approval_do_not_write(self): - cases=[] - cases.append(setup_case("pr.patch", {"body":"body", "closing":"#4"})) - cases.append(setup_case("comment.create", {"expected":expected(),"target":{"owner":"evil","repository":"r","issue":4,"pr":5}})) - p={"head":"f"*40}; cases.append(setup_case("push",p)) - for intent,approval,decl,calls,adapter in cases: - with self.subTest(intent=intent.operation), self.assertRaises(Exception): - pub.execute_rework_operation(adapter,intent=intent,approval=approval,declaration=decl, - coordinator=ops.Coordinator(),on_attempt=lambda x:None) - self.assertEqual(calls,[]) - - def test_single_mutation_and_fresh_adoption_at_boundary(self): - intent,approval,decl,calls,adapter=setup_case() - old=pub.require_adoption - try: - adopted=Mock(return_value=None); pub.require_adoption=adopted - attempts=[] - result=pub.execute_rework_operation(adapter,intent=intent,approval=approval,declaration=decl, - coordinator=ops.Coordinator(),on_attempt=attempts.append) - self.assertEqual(result.status,"verified") - self.assertEqual(len(calls),1); self.assertEqual(len(attempts),1) - adopted.assert_called_once() - finally: pub.require_adoption=old - - def test_adoption_and_preflight_failure_have_zero_mutations(self): - intent,approval,decl,calls,adapter=setup_case() - old=pub.require_adoption - try: - pub.require_adoption=Mock(side_effect=ValueError("stale")) - r=pub.execute_rework_operation(adapter,intent=intent,approval=approval,declaration=decl, - coordinator=ops.Coordinator(),on_attempt=lambda x:None) - self.assertEqual(r.status,"partial"); self.assertEqual(calls,[]) - adapter.preflight=lambda i: {"fresh":False} - r=pub.execute_rework_operation(adapter,intent=intent,approval=approval,declaration=decl, - coordinator=ops.Coordinator(),on_attempt=lambda x:None) - self.assertEqual(r.status,"partial"); self.assertEqual(calls,[]) - finally: pub.require_adoption=old - - def test_uncertain_ack_readback_once_literal_true_required(self): - intent,approval,decl,calls,adapter=setup_case() - old=pub.require_adoption - try: - pub.require_adoption=Mock(return_value=None) - mutations=[]; reads=[] - adapter.mutate=lambda i: mutations.append(i) or (_ for _ in ()).throw(ops.UncertainAcknowledgement()) - adapter.readback=lambda i: reads.append(i) or {"head":HEAD} - adapter.verify=lambda i,p,o: 1 - result=pub.execute_rework_operation(adapter,intent=intent,approval=approval,declaration=decl, - coordinator=ops.Coordinator(),on_attempt=lambda x:None) - self.assertEqual(result.status,"partial") - self.assertEqual((len(mutations),len(reads)),(1,1)) - finally: pub.require_adoption=old - - def test_reuse_collects_fresh_state_without_any_mutation(self): - intent, approval, declaration, calls, adapter = setup_case() - reads = [] - adapter.preflight = lambda value: reads.append("preflight") or {"fresh":True} - adapter.readback = lambda value: reads.append("readback") or {"exact":True} - adapter.verify_reuse = lambda value, before, after: after.get("exact") is True - result = pub.verify_rework_reuse(adapter,intent=intent,approval=approval,declaration=declaration) - self.assertEqual(result['action'],'reuse') - self.assertEqual(reads,['preflight','readback']) - self.assertEqual(calls,[]) - - def test_reuse_stale_preflight_blocks_even_when_callback_would_accept(self): - intent, approval, declaration, calls, adapter = setup_case() - reads=[] - adapter.preflight=lambda value:{"fresh":False} - adapter.readback=lambda value:reads.append(True) - adapter.verify_reuse=lambda *args:True - with self.assertRaises(ValueError): - pub.verify_rework_reuse(adapter,intent=intent,approval=approval,declaration=declaration) - self.assertEqual(reads,[]) - self.assertEqual(calls,[]) - - def test_canonical_comment_cannot_publish_to_different_intent_scope(self): - intent, approval, declaration, calls, adapter = setup_case() - for target, target_id in [('evil/r', 5), ('o/r', 999)]: - changed = ops.Intent('v1', intent.operation, target, target_id, HEAD, intent.payload) - approved = ops.Approval('v1', changed.digest, 'approved') - with self.subTest(target=target, target_id=target_id), self.assertRaises(ValueError): - pub.execute_rework_operation(adapter,intent=changed,approval=approved,declaration=declaration, - coordinator=ops.Coordinator(),on_attempt=lambda record:None) - self.assertEqual(calls, []) - - def test_unimplemented_client_cas_correction_blocks_before_write(self): - import rework_comment as rc - new = expected() - old = copy.deepcopy(new) - body = new['marker_free_body'] - for label in ('Source Issue','Pull Request','Source Review'): - body = body.replace('- '+label+': ','- '+label+': #') - digest = hashlib.sha256(body.encode()).hexdigest() - marker = new['marker'].replace(new['result_sha256'],digest) - old.update(body=body+marker,marker=marker,marker_free_body=body,result_sha256=digest) - payload = {'old_expected':old,'new_expected':new,'comment_id':7,'actor_id':42} - intent = ops.Intent('v1','correction','o/r',7,HEAD,payload) - approval = ops.Approval('v1',intent.digest,'approved',correction=True) - calls = [] - adapter = ops.Adapter(lambda value:{'fresh':True},True, - lambda value:calls.append('write'),lambda value:{'exact':True},lambda *args:True) - with self.assertRaises(ValueError): - pub.execute_rework_operation(adapter,intent=intent,approval=approval,declaration={'mode':'none'}, - coordinator=ops.Coordinator(),on_attempt=lambda record:None) - self.assertEqual(calls,[]) - - def test_generic_comment_patch_rejected_and_correction_requires_explicit_approval(self): - intent,approval,decl,calls,adapter=setup_case("comment.patch",{"body":"arbitrary"}) - with self.assertRaises(ValueError): pub.execute_rework_operation(adapter,intent=intent,approval=approval,declaration=decl,coordinator=ops.Coordinator(),on_attempt=lambda x:None) - self.assertEqual(calls,[]) - -if __name__ == "__main__": unittest.main() diff --git a/harness/tests/impl/test_rework_readback.py b/harness/tests/impl/test_rework_readback.py deleted file mode 100644 index b0880c1..0000000 --- a/harness/tests/impl/test_rework_readback.py +++ /dev/null @@ -1,707 +0,0 @@ -import copy -import hashlib -import importlib.util -import json -import tempfile -import unittest -from argparse import Namespace -from pathlib import Path -from urllib.parse import parse_qs, urlsplit -from harness.tests.common.impl_contract_docs import read_contract - -DIRECTORY = Path(__file__).parents[2] / 'agents/impl/skills/gitea-issue-rework/scripts' -ORIGIN = 'https://gitea.jsfamily.quest' -T0 = '2026-01-01T00:00:00Z' -T1 = '2026-01-02T00:00:00Z' - - -class ReworkReadbackTests(unittest.TestCase): - def load(self, name): - path = DIRECTORY / (name + '.py') - self.assertTrue(path.is_file(), 'canonical readback core missing') - spec = importlib.util.spec_from_file_location(name, path) - assert spec is not None and spec.loader is not None - module = importlib.util.module_from_spec(spec) - spec.loader.exec_module(module) - return module - - def setUp(self): - self.legacy = self.load('rework_comment') - self.temp = tempfile.TemporaryDirectory() - self.addCleanup(self.temp.cleanup) - root = Path(self.temp.name) - data = {'owner':'acme','repository':'demo','issue':7,'pr':9,'review':301, - 'reviewed_head':'a'*40,'final_head':'b'*40,'contract_sha256':'c'*64, - 'result':'partial','findings':[{'id':'F-1','status':'resolved','classification':'valid_fix', - 'changes':'bounded fix','verification':'local checks pass','reason':'regression'}], - 'local_checks':'pass','runtime_browser':'not applicable', - 'commit_status':'success','actions_runs':'없음'} - (root/'input.json').write_text(json.dumps(data)) - self.legacy.do_render(Namespace(input=str(root/'input.json'), no_delegated_evidence=True, - delegation_adoption=None, comment=str(root/'comment.md'), expected=str(root/'expected.json'))) - self.expected = json.loads((root/'expected.json').read_text()) - self.contract = {'schema':'gitea-agent.rework-provider/v1','version':'1.24.6', - 'aggregate':{'pr':{'field':'comments','components':['top_level','reviews','inline']}, - 'pr_issue':{'field':'comments','components':['top_level','reviews','inline']}}, - 'inline':{'mode':'per_review','field':'review_comments','components':['reviews','inline']}} - self.repo = {'id':3,'full_name':'acme/demo','html_url':ORIGIN+'/acme/demo'} - self.source = {'id':707,'number':7,'html_url':ORIGIN+'/acme/demo/issues/7', - 'body':'source contract','title':'Source','state':'open','comments':0,'updated_at':T0} - self.pr = {'id':99,'number':9,'html_url':ORIGIN+'/acme/demo/pulls/9', - 'url':ORIGIN+'/acme/demo/pulls/9','state':'open', - 'head':{'sha':'b'*40,'repo':{'id':12,'full_name':'contributor/demo'}}, - 'base':{'repo':copy.deepcopy(self.repo)},'title':'PR','body':'Closes #7', - 'comments':1,'review_comments':1,'updated_at':T0} - self.underlying = {'id':99,'number':9,'html_url':self.pr['html_url'], - 'pull_request':{'html_url':self.pr['html_url'],'merged':False}, - 'title':'PR','body':'Closes #7','state':'open','comments':1,'updated_at':T0} - self.reviews = [{'id':301,'commit_id':'a'*40,'comments_count':0,'state':'REQUEST_CHANGES', - 'user':{'id':33,'login':'gitea-review-agent'}}] - self.comments = [] - self.source_comments = [] - self.inline = [] - self.overrides = {} - self.calls = [] - for record in (self.source, self.pr, self.underlying): - record.update(created_at=T0, labels=[], assignees=[], milestone=None, - user={'id':22,'login':'genichin'} if record is self.source - else {'id':42,'login':'gitea-impl-agent'}) - self.pr['base'].update(ref='main', label='main', sha='d'*40, repo_id=3) - self.pr['head'].update(ref='fix/issue-7', label='contributor:fix/issue-7', repo_id=12) - self.reviews[0].update(body='Fix F-1', submitted_at=T0, updated_at=T0, - stale=True, dismissed=False, html_url=self.pr['html_url']+'#pullreview-301', - pull_request_url=self.pr['html_url']) - - def request(self, path): - self.calls.append(path) - if path in self.overrides: return copy.deepcopy(self.overrides[path]) - parsed = urlsplit(path) - objects = {'/user':{'id':42,'login':'gitea-impl-agent'}, '/version':{'version':'1.24.6'}, - '/repos/acme/demo':self.repo, '/repos/acme/demo/issues/7':self.source, - '/repos/acme/demo/pulls/9':self.pr, '/repos/acme/demo/issues/9':self.underlying} - if parsed.path in objects: return copy.deepcopy(objects[parsed.path]), {}, 200 - collections = {'/repos/acme/demo/issues/7/comments':self.source_comments, - '/repos/acme/demo/issues/9/comments':self.comments, - '/repos/acme/demo/pulls/9/reviews':self.reviews, - '/repos/acme/demo/pulls/9/reviews/301/comments':self.inline} - if parsed.path not in collections: raise AssertionError('unexpected GET endpoint') - rows = collections[parsed.path] - page = int(parse_qs(parsed.query)['page'][0]) - return copy.deepcopy(rows[(page-1)*50:page*50]), {'X-Total-Count':str(len(rows))}, 200 - - def adapter(self): - module = self.load('rework_readback') - try: - return module.ReworkReadbackAdapter(self.request, origin=ORIGIN+'/api/v1', - credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=42, - provider_contract=self.contract) - except Exception as error: - self.fail('canonical CLI transport contract rejected: '+str(error)) - - def add_result(self): - self.comments.append({'id':501,'html_url':self.pr['html_url']+'#issuecomment-501', - 'body':self.expected['body'],'created_at':T1,'updated_at':T1, - 'user':{'id':42,'login':'gitea-impl-agent'}}) - self.pr['comments'] += 1 - self.underlying['comments'] += 1 - self.pr['updated_at'] = self.underlying['updated_at'] = T1 - - def test_review_only_projection_preserves_inline_records_without_counting_them(self): - module = self.load('rework_readback') - contract = copy.deepcopy(self.contract) - contract['inline']['components'] = ['reviews'] - adapter = module.ReworkReadbackAdapter(self.request, origin=ORIGIN+'/api/v1', - credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=42, provider_contract=contract) - self.pr['review_comments'] = 1 - self.reviews[0]['comments_count'] = 0 - snapshot = adapter.snapshot(self.expected) - self.assertEqual(snapshot['normalized_counts']['inline'], 0) - self.reviews[0]['comments_count'] = 1 - self.inline = [{'id':801,'body':'inline','created_at':T0,'updated_at':T0, - 'user':{'id':33,'login':'gitea-review-agent'}}] - self.pr['review_comments'] = 1 - self.pr['comments'] = self.underlying['comments'] = 2 - snapshot = adapter.snapshot(self.expected) - self.assertEqual(snapshot['normalized_counts']['inline'], 1) - - def test_review_only_projection_rejects_counter_mismatch(self): - module = self.load('rework_readback') - contract = copy.deepcopy(self.contract) - contract['inline']['components'] = ['reviews'] - adapter = module.ReworkReadbackAdapter(self.request, origin=ORIGIN+'/api/v1', - credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=42, provider_contract=contract) - self.reviews[0]['comments_count'] = 1 - self.inline = [{'id':801,'body':'inline','created_at':T0,'updated_at':T0, - 'user':{'id':33,'login':'gitea-review-agent'}}] - self.pr['review_comments'] = 0 - with self.assertRaisesRegex(ValueError, 'inline provider counter crosscheck failed'): - adapter.snapshot(self.expected) - - def test_inline_component_contract_validation_remains_strict(self): - module = self.load('rework_readback') - for parts in ([], None, 'reviews', ['inline','inline'], ['reviews','reviews'], - ['unknown'], ['reviews','unknown'], ['reviews','inline','reviews']): - contract = copy.deepcopy(self.contract) - contract['inline']['components'] = parts - with self.subTest(components=parts), self.assertRaises(ValueError): - module.ReworkReadbackAdapter(self.request, origin=ORIGIN+'/api/v1', - credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=42, provider_contract=contract) - - def test_review_only_projection_retains_inline_evidence_rejections(self): - module = self.load('rework_readback') - contract = copy.deepcopy(self.contract) - contract['inline']['components'] = ['reviews'] - adapter = module.ReworkReadbackAdapter(self.request, origin=ORIGIN+'/api/v1', - credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=42, provider_contract=contract) - self.reviews[0]['comments_count'] = 1 - self.inline = [{'id':801,'body':'inline','created_at':T0,'updated_at':T0, - 'user':{'id':33,'login':'gitea-review-agent'}}] - self.pr['review_comments'] = 2 - with self.assertRaises(ValueError): adapter.snapshot(self.expected) - self.pr['review_comments'] = 1 - self.inline = [] - with self.assertRaisesRegex(ValueError, 'per-review inline count mismatch'): - adapter.snapshot(self.expected) - self.inline = [{'id':801,'body':'inline','created_at':T0,'updated_at':T0, - 'user':{'id':33,'login':'gitea-review-agent'}}, - {'id':801,'body':'duplicate','created_at':T0,'updated_at':T0, - 'user':{'id':33,'login':'gitea-review-agent'}}] - with self.assertRaisesRegex(ValueError, 'duplicate'): - adapter.snapshot(self.expected) - self.inline = [{'id':801,'body':'inline','created_at':T0,'updated_at':T0, - 'user':{'id':33,'login':'gitea-review-agent'}}] - self.overrides = {'/repos/acme/demo/pulls/9/reviews/301/comments?page=1&limit=50': - (self.inline, {'X-Total-Count':'2'}, 200)} - with self.assertRaises(ValueError): adapter.snapshot(self.expected) - - def test_snapshot_rejects_wrong_provider_bindings(self): - cases = [('repo',['id'],True), ('repo',['full_name'],'elsewhere/demo'), - ('repo',['html_url'],'https://foreign.invalid/acme/demo'), - ('source',['number'],8), ('source',['id'],True), - ('source',['html_url'],ORIGIN+'/elsewhere/demo/issues/7'), - ('source',['pull_request'],{}), ('pr',['number'],10), ('pr',['id'],True), - ('pr',['url'],'https://foreign.invalid/api/v1/repos/acme/demo/pulls/9'), - ('pr',['html_url'],ORIGIN+'/elsewhere/demo/pulls/9'), - ('pr',['head','sha'],'c'*40), ('pr',['base','repo','id'],4), - ('pr',['state'],'closed'), ('underlying',['number'],10), - ('underlying',['pull_request'],True), ('reviews',[0,'commit_id'],'c'*40)] - error = self.load('rework_readback').ReadbackError - self.assertTrue(issubclass(error, ValueError)) - for attribute, keys, value in cases: - with self.subTest(attribute=attribute, keys=keys): - original = copy.deepcopy(getattr(self, attribute)) - item = getattr(self, attribute) - for key in keys[:-1]: item = item[key] - item[keys[-1]] = value - try: - with self.assertRaises(ValueError): self.adapter().snapshot(self.expected) - finally: setattr(self, attribute, original) - - def test_pagination_complete_51_and_rejects_bad_pages(self): - self.comments = [{'id':600+i,'body':'existing '+str(i),'created_at':T0,'updated_at':T0, - 'user':{'id':8,'login':'other'}} for i in range(51)] - self.pr['comments'] = self.underlying['comments'] = 52 - snapshot = self.adapter().snapshot(self.expected) - self.assertEqual(snapshot['normalized_counts']['top_level'], 51) - self.assertEqual(len(snapshot['raw']['comments']['pages']), 2) - first = '/repos/acme/demo/issues/9/comments?page=1&limit=50' - second = '/repos/acme/demo/issues/9/comments?page=2&limit=50' - invalid = [(second,[self.comments[0]],{'X-Total-Count':'51'},200), - (second,[self.comments[-1]],{'X-Total-Count':'52'},200), - (first,None,{},200), (first,[],{'X-Total-Count':'0','x-total-count':'0'},200), - (first,[],{'X-Total-Count':'0','Link':'; rel="next"'},200), - (first,self.comments[:50],{},200), - (first,[{'id':True}],{'X-Total-Count':'1'},200), - (first,[],{'X-Total-Count':'0','Link':'not a Link'},200), - (first,[],{},True), (first,[],{},403), (first,[],{},302)] - for path, rows, headers, status in invalid: - with self.subTest(path=path, headers=headers, status=status): - self.overrides = {path:(rows,headers,status)} - with self.assertRaises(ValueError): self.adapter().snapshot(self.expected) - self.overrides = {} - - def test_counts_only_inline_fallback_is_explicit_and_crosschecked(self): - self.contract['inline']['mode'] = 'per_review_or_pr_count' - self.reviews[0]['comments_count'] = 3 - self.pr['comments'] = self.underlying['comments'] = self.pr['review_comments'] = 4 - endpoint = '/repos/acme/demo/pulls/9/reviews/301/comments?page=1&limit=50' - self.overrides = {endpoint:(None,{},404)} - try: snapshot = self.adapter().snapshot(self.expected) - except ValueError as error: self.fail('configured counts-only fallback missing: '+str(error)) - self.assertEqual(snapshot['normalized_counts']['inline'], 3) - self.assertEqual(snapshot['fallback'], 'counts_only') - self.assertEqual(snapshot['raw']['inline']['301']['rows'], []) - self.pr['review_comments'] = 5 - with self.assertRaises(ValueError): self.adapter().snapshot(self.expected) - self.pr['review_comments'] = 4 - for status, payload in ((403,None),(302,None),(200,None)): - self.overrides = {endpoint:(payload,{},status)} - with self.subTest(status=status): - with self.assertRaises(ValueError): self.adapter().snapshot(self.expected) - self.contract['inline']['mode'] = 'per_review' - self.overrides = {endpoint:(None,{},404)} - with self.assertRaises(ValueError): self.adapter().snapshot(self.expected) - - def test_preflight_rejects_unbound_contract_and_target_before_GET(self): - module = self.load('rework_readback') - changes = [(['schema'],'unsupported'), (['version'],True), - (['aggregate','pr','components'],['inline']), - (['aggregate','pr','components'],['top_level','top_level']), - (['aggregate','pr_issue','field'],'other'), - (['inline','mode'],'automatic'), (['inline','components'],['other'])] - for keys, value in changes: - contract = copy.deepcopy(self.contract) - item = contract - for key in keys[:-1]: item = item[key] - item[keys[-1]] = value - with self.subTest(contract=keys): - with self.assertRaises(ValueError): - module.ReworkReadbackAdapter(self.request, origin=ORIGIN+'/api/v1', - credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=42, - provider_contract=contract) - self.assertEqual(self.calls, []) - for keys, value in ((['target','owner'],'acme/../elsewhere'), - (['target','issue'],9), (['review'],True), (['head'],'B'*40), - (['body'],'forged body')): - expected = copy.deepcopy(self.expected) - item = expected - for key in keys[:-1]: item = item[key] - item[keys[-1]] = value - self.calls = [] - with self.subTest(expected=keys): - with self.assertRaises(ValueError): self.adapter().snapshot(expected) - self.assertEqual(self.calls, [], 'invalid expectation reached provider') - - def test_snapshot_detects_drift_during_collection(self): - original = self.request - endpoints = ['/user','/version','/repos/acme/demo','/repos/acme/demo/issues/7', - '/repos/acme/demo/pulls/9','/repos/acme/demo/issues/9', - '/repos/acme/demo/pulls/9/reviews'] - for endpoint in endpoints: - counts = {} - def changing(path): - value, headers, status = original(path) - key = urlsplit(path).path - counts[key] = counts.get(key, 0) + 1 - if key == endpoint and counts[key] > 1: - if type(value) is list: value[0]['state'] = 'APPROVED' - else: value['unexpected_drift'] = True - return value, headers, status - self.request = changing - try: - with self.subTest(endpoint=endpoint): - with self.assertRaisesRegex(ValueError, 'drift'): - self.adapter().snapshot(self.expected) - finally: self.request = original - - def test_normalization_rejects_unrelated_delta_and_forged_pages(self): - adapter = self.adapter() - baseline = adapter.snapshot(self.expected) - self.add_result() - after = adapter.snapshot(self.expected) - adapter.normalized_inputs(self.expected, baseline, after, 'create') - cases = [(['schema'],'forged'), (['actor_id'],True), (['origin'],'https://foreign.invalid/api/v1'), - (['target','owner'],'other'), (['provider','version'],'different'), - (['expected_sha256'],'0'*64), (['raw','identity','login'],'other'), - (['raw','source_issue','body'],'unrelated edit'), - (['raw','reviews','rows',0,'state'],'APPROVED'), - (['raw','pr','head','sha'],'c'*40), (['raw','pr','title'],'unrelated edit'), - (['raw','pr','updated_at'],'2026-01-03T00:00:00Z'), - (['raw','comments','pages',0,'path'],'/repos/elsewhere/demo/issues/9/comments?page=1&limit=50'), - (['raw','comments','pages',0,'count'],0), - (['raw','comments','rows',0,'body'],'forged body'), - (['raw','comments','rows',0,'user','id'],8), - (['raw','comments','rows',0,'created_at'],'2025-01-01T00:00:00Z'), - (['raw','comments','rows',0,'updated_at'],'2026-01-03T00:00:00Z')] - for keys, value in cases: - changed = copy.deepcopy(after) - item = changed - for key in keys[:-1]: item = item[key] - item[keys[-1]] = value - with self.subTest(keys=keys): - with self.assertRaises(ValueError): - adapter.normalized_inputs(self.expected, baseline, changed, 'create') - with self.assertRaises(ValueError): adapter.normalized_inputs(self.expected, baseline, after, 'reuse') - - def test_existing_comment_rows_are_immutable(self): - self.comments = [{'id':400,'body':'prior comment','created_at':T0,'updated_at':T0, - 'user':{'id':8,'login':'other'}}] - self.pr['comments'] = self.underlying['comments'] = 2 - adapter = self.adapter() - baseline = adapter.snapshot(self.expected) - self.add_result() - after = adapter.snapshot(self.expected) - adapter.normalized_inputs(self.expected, baseline, after, 'create') - after['raw']['comments']['rows'][0]['body'] = 'changed prior comment' - with self.assertRaisesRegex(ValueError, 'existing comment'): - adapter.normalized_inputs(self.expected, baseline, after, 'create') - - def test_snapshot_validates_captured_rows_before_persisting(self): - self.add_result() - self.adapter().snapshot(self.expected) - for keys, value in ((['body'],True), (['user','id'],True), - (['created_at'],'2026-01-02T00:00:00'), - (['updated_at'],T0)): - saved = copy.deepcopy(self.comments) - item = self.comments[0] - for key in keys[:-1]: item = item[key] - item[keys[-1]] = value - with self.subTest(keys=keys): - with self.assertRaises(ValueError): self.adapter().snapshot(self.expected) - self.comments = saved - self.source['updated_at'] = 'not a timestamp' - with self.assertRaises(ValueError): self.adapter().snapshot(self.expected) - - def test_live_bridge_round_trip_over_local_HTTP(self): - import contextlib - import http.server - import io - import os - import threading - import urllib.request - from unittest.mock import patch - cli = self.load('rework_readback_cli') - root = Path(self.temp.name) - profile = root/'profile' - profile.mkdir(mode=0o700) - secret = 'synthetic-fixture-' + 'q'*40 - (profile/'.env').write_text('GITEA_TOKEN='+secret+'\n') - (profile/'.env').chmod(0o600) - (root/'provider.json').write_text(json.dumps(self.contract)) - contract_hash = hashlib.sha256((root/'provider.json').read_bytes()).hexdigest() - requests = [] - fixture = self - class Handler(http.server.BaseHTTPRequestHandler): - def log_message(self, format, *args): pass - def do_GET(self): - requests.append((self.command, self.headers.get('Authorization') == 'token '+secret)) - data, headers, status = fixture.request(self.path[len('/api/v1'):]) - self.send_response(status) - self.send_header('Content-Type','application/json') - for key, value in headers.items(): self.send_header(key,value) - self.end_headers() - self.wfile.write(json.dumps(data).encode()) - server = http.server.ThreadingHTTPServer(('127.0.0.1',0), Handler) - thread = threading.Thread(target=server.serve_forever, daemon=True) - thread.start() - inner = urllib.request.build_opener(cli.NoRedirect()) - class Proxy: - def open(self, request, timeout): - forwarded = urllib.request.Request('http://127.0.0.1:'+str(server.server_port)+request.selector, - headers=dict(request.header_items()), method=request.get_method()) - response = inner.open(forwarded, timeout=timeout) - class Response: - status = response.status - headers = response.headers - def geturl(self): return request.full_url - def read1(self, n): return response.read1(n) - def __enter__(self): return self - def __exit__(self, *args): response.close() - return Response() - transport = cli.RestTransport - common = dict(expected=str(root/'expected.json'), provider_contract=str(root/'provider.json'), - provider_contract_sha256=contract_hash, actor_id=42) - baseline_path, receipt_path = root/'baseline.json', root/'receipt.json' - try: - with patch.dict(os.environ, {'HERMES_HOME':str(profile)}), patch.object(cli,'RestTransport', - side_effect=lambda credential: transport(credential, opener=Proxy())), contextlib.redirect_stdout(io.StringIO()) as stdout: - cli.run_live(Namespace(cmd='snapshot', output=str(baseline_path), **common), self.legacy.verify_readbacks) - baseline_hash = hashlib.sha256(baseline_path.read_bytes()).hexdigest() - self.add_result() - args = Namespace(cmd='verify-live', baseline=str(baseline_path), baseline_sha256=baseline_hash, - action='create', output=str(receipt_path), **common) - cli.run_live(args, self.legacy.verify_readbacks) - receipt = json.loads(receipt_path.read_text()) - self.assertEqual(receipt['status'],'verified') - self.assertEqual(receipt['comment_id'],501) - self.assertEqual(receipt['readback_provenance']['kind'],'fresh_authenticated_GET') - original = receipt_path.read_bytes() - self.source['title'] = 'unrelated provider edit' - with self.assertRaises(ValueError): cli.run_live(args, self.legacy.verify_readbacks) - self.assertEqual(receipt_path.read_bytes(), original) - self.assertNotIn(secret, stdout.getvalue()) - self.assertNotIn(secret, receipt_path.read_text()) - self.assertTrue(requests) - self.assertEqual(set(method for method, _ in requests), {'GET'}) - self.assertTrue(all(authenticated for _, authenticated in requests)) - finally: - server.shutdown() - server.server_close() - thread.join(timeout=3) - - def test_underlying_relation_requires_native_html_url(self): - self.adapter().snapshot(self.expected) - self.underlying['pull_request'].pop('html_url') - with self.assertRaises(ValueError): self.adapter().snapshot(self.expected) - - def test_non_next_Link_relations_are_coherent_and_preserved(self): - endpoint = '/repos/acme/demo/issues/7/comments' - path = endpoint+'?page=1&limit=50' - for relation, page in (('last',21), ('prev',1), ('first',2)): - self.overrides = {path:([],{'X-Total-Count':'0','Link': - '; rel="'+relation+'"'},200)} - with self.subTest(relation=relation): - with self.assertRaises(ValueError): self.adapter().snapshot(self.expected) - header = '; rel="last"' - self.overrides = {path:([],{'X-Total-Count':'0','Link':header},200)} - snapshot = self.adapter().snapshot(self.expected) - self.assertEqual(snapshot['raw']['source_comments']['pages'][0].get('link'), header) - - def test_pagination_link_rejects_unbalanced_relation_quotes(self): - endpoint = '/repos/acme/demo/issues/7/comments' - path = endpoint+'?page=1&limit=50' - for relation in ('"last', 'last"'): - self.overrides = {path:([],{'X-Total-Count':'0','Link': - '; rel='+relation},200)} - with self.subTest(relation=relation): - with self.assertRaises(ValueError): self.adapter().snapshot(self.expected) - - def test_native_Gitea_PR_URL_and_omitted_proven_zero_inline_count(self): - self.pr['url'] = self.pr['html_url'] - self.contract['inline']['components'] = ['inline'] - self.pr.pop('review_comments') - try: snapshot = self.adapter().snapshot(self.expected) - except ValueError as error: self.fail('native Gitea wire shape rejected: '+str(error)) - self.assertEqual(snapshot['normalized_counts']['inline'], 0) - self.assertEqual(snapshot['fallback'], 'none') - self.contract['inline']['mode'] = 'per_review_or_pr_count' - self.overrides = {'/repos/acme/demo/pulls/9/reviews/301/comments?page=1&limit=50':(None,{},404)} - with self.assertRaises(ValueError): self.adapter().snapshot(self.expected) - - def test_boolean_actor_is_rejected_before_other_GETs(self): - module = self.load('rework_readback') - self.overrides = {'/user':({'id':True,'login':'gitea-impl-agent'}, {}, 200)} - adapter = module.ReworkReadbackAdapter(self.request, origin=ORIGIN+'/api/v1', - credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=1, provider_contract=self.contract) - with self.assertRaises(ValueError): adapter.snapshot(self.expected) - self.assertEqual(self.calls, ['/user']) - - def test_required_native_metadata_and_PR_author_are_verified(self): - self.adapter().snapshot(self.expected) - cases = [('pr',['user'],None), ('pr',['user','id'],43), - ('pr',['user','login'],'different'), ('source',['labels'],True), - ('source',['assignees'],True), ('source',['milestone'],[]), - ('pr',['base','ref'],None), ('pr',['head','repo_id'],99), - ('reviews',[0,'state'],None), ('reviews',[0,'body'],None), - ('reviews',[0,'user','login'],'unapproved'), - ('reviews',[0,'submitted_at'],None), ('reviews',[0,'dismissed'],True), - ('reviews',[0,'stale'],'true')] - for name, keys, value in cases: - original = copy.deepcopy(getattr(self,name)) - item = getattr(self,name) - for key in keys[:-1]: item = item[key] - item[keys[-1]] = value - with self.subTest(name=name, keys=keys): - with self.assertRaises(ValueError): self.adapter().snapshot(self.expected) - setattr(self,name,original) - for name, field in (('source','labels'), ('source','assignees'), ('source','milestone'), - ('underlying','user'), ('reviews','submitted_at')): - original = copy.deepcopy(getattr(self,name)) - record = getattr(self,name)[0] if name == 'reviews' else getattr(self,name) - record.pop(field) - with self.subTest(missing=name+'.'+field): - with self.assertRaises(ValueError): self.adapter().snapshot(self.expected) - setattr(self,name,original) - - def test_source_timestamp_change_is_not_admitted_as_causal_proof(self): - adapter = self.adapter() - baseline = adapter.snapshot(self.expected) - self.add_result() - after = adapter.snapshot(self.expected) - adapter.normalized_inputs(self.expected, baseline, after, 'create') - after['raw']['source_issue']['updated_at'] = T1 - with self.assertRaises(ValueError): adapter.normalized_inputs(self.expected, baseline, after, 'create') - - def test_native_nullable_assignees_and_milestone_without_number(self): - for record in (self.source,self.pr,self.underlying): - record['assignees'] = None - record['milestone'] = {'id':55,'title':'Native milestone','state':'open'} - try: self.adapter().snapshot(self.expected) - except ValueError as error: self.fail('native nullable-assignee/milestone shape rejected: '+str(error)) - self.source.pop('assignees') - with self.assertRaises(ValueError): self.adapter().snapshot(self.expected) - - def test_closing_keyword_must_match_canonical_Closes(self): - self.adapter().snapshot(self.expected) - self.pr['body'] = 'Fixes #7' - with self.assertRaises(ValueError): self.adapter().snapshot(self.expected) - - def test_renderer_to_existing_verifier_create_and_reuse(self): - adapter = self.adapter() - try: baseline = adapter.snapshot(self.expected) - except Exception as error: self.fail('actual renderer expected rejected: '+str(error)) - self.assertEqual(self.calls[0], '/user') - self.add_result() - after = adapter.snapshot(self.expected) - inputs = adapter.normalized_inputs(self.expected, baseline, after, 'create') - receipt = self.legacy.verify_readbacks(self.expected, inputs['pr'], inputs['comments'], - inputs['identity'], action='create', baseline_count=inputs['baseline_count']) - self.assertEqual(receipt['status'], 'verified') - self.assertEqual(receipt['comment_id'], 501) - reused = adapter.normalized_inputs(self.expected, after, adapter.snapshot(self.expected), 'reuse') - self.assertEqual(self.legacy.verify_readbacks(self.expected, reused['pr'], reused['comments'], - reused['identity'], action='reuse', baseline_count=reused['baseline_count'])['status'], 'verified') - - def _cache_adapter(self): - self.contract['version'] = '1.27.3' - self.overrides['/version'] = ({'version':'1.27.3'}, {}, 200) - self.contract['aggregate']['pr'] = {'field':'comments','components':['top_level','reviews'], - 'cache_mode':'gitea-conversation-v1'} - self.contract['aggregate']['pr_issue'] = {'field':'comments','components':['top_level','reviews'], - 'cache_mode':'gitea-conversation-v1'} - return self.adapter() - - def _stale_baseline(self): - self.pr['comments'] = self.underlying['comments'] = 0 - adapter = self._cache_adapter() - return adapter, adapter.snapshot(self.expected) - - def test_cache_opt_in_captures_and_validates_bounded_stale_baseline(self): - adapter, baseline = self._stale_baseline() - self.assertEqual(baseline['normalized_counts']['top_level'], 0) - self.assertEqual(baseline['normalized_counts']['reviews'], 1) - self.assertEqual(baseline['raw']['pr']['comments'], 0) - self.assertEqual(baseline['raw']['pr_issue']['comments'], 0) - adapter.validate_snapshot(self.expected, baseline) - self.overrides['/version'] = ({'version':'1.24.6'}, {}, 200) - with self.assertRaisesRegex(ValueError, 'provider version mismatch'): - adapter.snapshot(self.expected) - - def test_cache_create_recomputes_post_conversation_count_and_guards_deltas(self): - adapter, baseline = self._stale_baseline() - self.add_result() - self.pr['comments'] = self.underlying['comments'] = 2 - after = adapter.snapshot(self.expected) - inputs = adapter.normalized_inputs(self.expected, baseline, after, 'create') - receipt = self.legacy.verify_readbacks(self.expected, inputs['pr'], inputs['comments'], - inputs['identity'], action='create', baseline_count=inputs['baseline_count']) - self.assertEqual(receipt['status'], 'verified') - # Capture a genuine pre-create synchronized baseline, not the post-create rows. - self.comments.clear() - self.pr['comments'] = self.underlying['comments'] = 1 - self.pr['updated_at'] = self.underlying['updated_at'] = T0 - synchronized = adapter.snapshot(self.expected) - self.add_result() - synchronized_after = adapter.snapshot(self.expected) - adapter.normalized_inputs(self.expected, synchronized, synchronized_after, 'create') - for bad in (1, None, -1, 3): - changed = copy.deepcopy(after) - changed['raw']['pr']['comments'] = bad - changed['raw']['pr_issue']['comments'] = bad - with self.subTest(after_counter=bad), self.assertRaises(ValueError): - adapter.normalized_inputs(self.expected, baseline, changed, 'create') - for mutation in ('old_comment','duplicate','native_review','source_issue'): - changed = copy.deepcopy(after) - if mutation == 'old_comment': - changed['raw']['comments']['rows'] = [{'id':400,'body':'mutated','created_at':T0, - 'updated_at':T0,'user':{'id':8,'login':'other'}}] + changed['raw']['comments']['rows'] - elif mutation == 'duplicate': - changed['raw']['comments']['rows'].append(copy.deepcopy(changed['raw']['comments']['rows'][0])) - elif mutation == 'native_review': changed['raw']['reviews']['rows'][0]['state'] = 'APPROVED' - else: changed['raw']['source_issue']['title'] = 'drift' - with self.subTest(mutation=mutation), self.assertRaises(ValueError): - adapter.normalized_inputs(self.expected, baseline, changed, 'create') - - def test_cache_reuse_allows_same_lag_but_rejects_refresh(self): - self.add_result() - self.pr['comments'] = self.underlying['comments'] = 0 - adapter = self._cache_adapter() - baseline = adapter.snapshot(self.expected) - same = adapter.snapshot(self.expected) - inputs = adapter.normalized_inputs(self.expected, baseline, same, 'reuse') - receipt = self.legacy.verify_readbacks(self.expected, inputs['pr'], inputs['comments'], - inputs['identity'], action='reuse', baseline_count=inputs['baseline_count']) - self.assertEqual(receipt['status'], 'verified') - self.pr['comments'] = self.underlying['comments'] = 2 - refreshed = adapter.snapshot(self.expected) - with self.assertRaises(ValueError): adapter.normalized_inputs(self.expected, baseline, refreshed, 'reuse') - - def test_cache_opt_in_rejects_invalid_mode_version_and_components(self): - module = self.load('rework_readback') - for mode, version, components in ( - ('unknown','1.27.3',['top_level','reviews']), - (None,'1.27.3',['top_level','reviews']), - (True,'1.27.3',['top_level','reviews']), - ('gitea-conversation-v1','1.27.2',['top_level','reviews']), - ('gitea-conversation-v1','1.27.3',['reviews']), - ('gitea-conversation-v1','1.27.3',['top_level']), - ('gitea-conversation-v1','1.27.3',['top_level','reviews','inline'])): - contract = copy.deepcopy(self.contract) - contract['version'] = version - contract['aggregate']['pr'] = {'field':'comments','components':components,'cache_mode':mode} - contract['aggregate']['pr_issue'] = {'field':'comments','components':components,'cache_mode':mode} - with self.subTest(mode=mode,version=version,components=components), self.assertRaises(ValueError): - module.ReworkReadbackAdapter(self.request, origin=ORIGIN+'/api/v1', - credential_source='$HERMES_HOME/.env:GITEA_TOKEN', actor_id=42, provider_contract=contract) - - def test_strict_legacy_counter_semantics_are_unchanged_without_cache_mode(self): - self.pr['comments'] = self.underlying['comments'] = 0 - with self.assertRaises(ValueError): self.adapter().snapshot(self.expected) - - def test_cache_baseline_does_not_count_inline_records_as_conversation(self): - adapter = self._cache_adapter() - self.contract['inline']['components'] = ['reviews'] - adapter = self.adapter() - self.reviews[0]['comments_count'] = 1 - self.inline = [{'id':801,'body':'inline','created_at':T0,'updated_at':T0, - 'user':{'id':33,'login':'gitea-review-agent'}}] - self.pr['comments'] = self.underlying['comments'] = 2 - with self.assertRaisesRegex(ValueError, 'cached provider aggregate count exceeds collection'): - adapter.snapshot(self.expected) - - def test_cache_baseline_rejects_malformed_missing_and_ahead_counters(self): - adapter = self._cache_adapter() - for scope in (self.pr, self.underlying): - for bad in (None, -1, True, '0', 1.0, 2): - scope['comments'] = bad - with self.subTest(counter=bad), self.assertRaises(ValueError): - adapter.snapshot(self.expected) - scope.pop('comments') - with self.assertRaises(ValueError): adapter.snapshot(self.expected) - scope['comments'] = 1 - - def test_cache_transition_rejects_mutation_of_a_genuine_existing_comment(self): - old = {'id':400, 'html_url':self.pr['html_url']+'#issuecomment-400', - 'body':'preserved previous comment', 'created_at':T0, 'updated_at':T0, - 'user':{'id':33,'login':'gitea-review-agent'}} - self.comments.append(old) - adapter, baseline = self._stale_baseline() - self.add_result() - self.pr['comments'] = self.underlying['comments'] = 3 - after = adapter.snapshot(self.expected) - adapter.normalized_inputs(self.expected, baseline, after, 'create') - changed = copy.deepcopy(after) - changed['raw']['comments']['rows'][0]['body'] = 'unauthorized mutation' - with self.assertRaisesRegex(ValueError, 'existing comment changed or missing'): - adapter.normalized_inputs(self.expected, baseline, changed, 'create') - - def test_cache_opt_in_does_not_relax_an_undeclared_aggregate_scope(self): - self._cache_adapter() - self.contract['aggregate']['pr_issue'].pop('cache_mode') - adapter = self.adapter() - self.pr['comments'] = self.underlying['comments'] = 0 - with self.assertRaisesRegex(ValueError, 'provider aggregate count mismatch'): - adapter.snapshot(self.expected) - self.underlying['comments'] = 1 - adapter.snapshot(self.expected) - - def test_cache_provider_documentation_example_matches_executable_contract(self): - import re - doc = (DIRECTORY.parent/'references/readback.md').read_text() - match = re.search(r'```json\s*(.*?)\s*```', doc, re.S) - if match is None: self.fail('provider documentation requires a JSON example') - example = json.loads(match.group(1)) - self._cache_adapter() - self.contract = example - self.adapter().snapshot(self.expected) - - def test_shared_commit_message_safety_rule_is_linked_and_amend_is_narrow(self): - common = (DIRECTORY.parents[4]/'operations/repository-cache.md').read_text() - skill = read_contract(DIRECTORY.parent / 'SKILL.md') - self.assertIn('## Commit-message reference safety', common) - for clause in ('complete subject and body', 'cross-repository', - 'non-default branch', 'explicit user authorization', - 'commit is unpublished', 'tree is identical', - 'PR body', 'not general permission to amend'): - self.assertIn(clause, common) - self.assertIn('repository-cache.md#commit-message-reference-safety', skill) diff --git a/harness/tests/review/contracts/__init__.py b/harness/tests/review/contracts/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/harness/tests/review/test_native_journal_contract_docs.py b/harness/tests/review/contracts/test_review_document_contracts.py similarity index 58% rename from harness/tests/review/test_native_journal_contract_docs.py rename to harness/tests/review/contracts/test_review_document_contracts.py index 9153731..2fc65db 100644 --- a/harness/tests/review/test_native_journal_contract_docs.py +++ b/harness/tests/review/contracts/test_review_document_contracts.py @@ -1,41 +1,42 @@ -import unittest from pathlib import Path +import unittest +from harness.tests._support.paths import ROOT -ROOT = Path(__file__).resolve().parents[3] REFERENCE = ROOT / 'harness/agents/review/skills/gitea-code-review/references/native-review.md' SKILL = ROOT / 'harness/agents/review/skills/gitea-code-review/SKILL.md' CACHE = ROOT / 'harness/operations/repository-cache.md' - +OBSERVABILITY_HEADING = '## 조건부 observability 정책 확인과 종료 기록' +TIMEOUT_HEADING = '### 장시간 suite의 timeout evidence recipe' class NativeJournalDocumentationTests(unittest.TestCase): def test_canonical_registry_and_legacy_policy_are_explicit(self): for path in (REFERENCE, SKILL, CACHE): text = path.read_text() - for token in ('harness/.runtime/native-review-journals', - 'canonical_journal_path', 'register_legacy_journal', - 'binding', 'read-only', 'cross-host'): + for token in ('harness/.runtime/native-review-journals', 'canonical_journal_path', 'register_legacy_journal'): with self.subTest(path=str(path), token=token): self.assertIn(token, text) - def test_journal_parent_preflight_precedes_expensive_execution(self): text = SKILL.read_text() entry = text.split('## 진입 조건', 1)[1].split('## 리뷰 관점', 1)[0] - for requirement in ( - '고비용 리뷰·테스트 실행 전에', - '$HARNESS_ROOT/harness/.runtime', 'lstat', 'symlink', - '현재 review role 소유', '정확한 mode `0700`', - '읽기 전용', 'blocker', '운영자 승인', 'local-only', - '자동 chmod/chown', 'journal 이동', 'guard 완화', - '기존 생성 규칙', '게시 승인'): + for requirement in ('고비용 리뷰·테스트 실행 전에', '$HARNESS_ROOT/harness/.runtime', 'lstat', 'symlink', '정확한 mode `0700`'): with self.subTest(requirement=requirement): self.assertIn(requirement, entry) - def test_no_caller_selected_journal_in_live_example(self): text = REFERENCE.read_text() self.assertNotIn('result = native_review.publish(plan, baseline, api, lease, journal_directory)', text) self.assertIn('result = native_review.publish(plan, baseline, api, lease)', text) - self.assertIn('기존 journal을 자동 이동·삭제', text) - self.assertIn('pending', text) + + + + + +class ReviewProbeDocumentationTests(unittest.TestCase): + def test_documentation_contract_requires_compile_before_execution_and_preserves_authority(self): + section = SKILL.read_text().split('### Programmatic read control response와 script 준비', 1)[1].split('\n## ', 1)[0] + # Retain the required preparation order, not general authority prose. + for rule in ('compile', '실행·로그 append·외부 action 전에'): + with self.subTest(rule=rule): + self.assertIn(rule, section) if __name__ == '__main__': diff --git a/harness/tests/review/evidence/__init__.py b/harness/tests/review/evidence/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/harness/tests/review/test_native_evidence_recipe.py b/harness/tests/review/evidence/test_native_evidence_recipe.py similarity index 84% rename from harness/tests/review/test_native_evidence_recipe.py rename to harness/tests/review/evidence/test_native_evidence_recipe.py index 23c2757..a6ccbb8 100644 --- a/harness/tests/review/test_native_evidence_recipe.py +++ b/harness/tests/review/evidence/test_native_evidence_recipe.py @@ -8,9 +8,9 @@ import sys import unittest from pathlib import Path -from harness.tests.impl import test_evidence_manifest as fixtures +from harness.tests._support.evidence import EvidenceManifestFixture, run_git +from harness.tests._support.paths import ROOT -ROOT = Path(__file__).resolve().parents[3] REFERENCE = ROOT / "harness/agents/review/skills/gitea-code-review/references/native-review.md" HELPER_RELATIVE = "harness/agents/impl/skills/gitea-issue-rework/scripts/evidence_manifest.py" HEADING = "## Reviewer verification evidence recipe" @@ -46,11 +46,8 @@ class NativeEvidenceRecipeTests(unittest.TestCase): def test_documented_commands_run_validate_and_reject_absolute_artifact_paths(self): section = self.recipe() - fixture = fixtures.EvidenceManifestTests() + fixture = EvidenceManifestFixture().setup() try: - fixture.setUp() - # Install exact current helper bytes into an isolated canonical-shaped - # synthetic harness. No fork, wrapper, live lease or provider is used. source = ROOT / HELPER_RELATIVE helper = fixture.root / HELPER_RELATIVE helper.parent.mkdir(parents=True) @@ -77,7 +74,7 @@ class NativeEvidenceRecipeTests(unittest.TestCase): manifest_path = fixture.artifacts / "evidence-manifest.json" manifest = json.loads(manifest_path.read_text()) self.assertEqual(manifest["schema"], "evidence-manifest/v3") - self.assertEqual(manifest["source_revision"], fixtures.run_git(fixture.repo, "rev-parse", "HEAD").stdout.strip()) + self.assertEqual(manifest["source_revision"], run_git(fixture.repo, "rev-parse", "HEAD").stdout.strip()) self.assertEqual(manifest["plan"]["path"], "evidence-plan.json") self.assertEqual(manifest["commands"][0]["argv"], ["git", "show", "HEAD:input.txt"]) self.assertEqual(manifest["commands"][0]["exit_code"], 0) @@ -87,15 +84,13 @@ class NativeEvidenceRecipeTests(unittest.TestCase): before = manifest_path.read_bytes() mode = stat.S_IMODE(manifest_path.stat().st_mode) self.assertEqual(mode, 0o600) - for flag, base_command, filename in ( - ("--plan", commands[0], "evidence-plan.json"), - ("--output", commands[0], "bad-output.json"), - ("--manifest", commands[1], "evidence-manifest.json")): + for flag, base_command, filename in (("--plan", commands[0], "evidence-plan.json"), + ("--output", commands[0], "bad-output.json"), + ("--manifest", commands[1], "evidence-manifest.json")): with self.subTest(flag=flag): invalid = list(base_command) invalid[invalid.index(flag) + 1] = str(fixture.artifacts / filename) if flag == "--plan": - # Use an absent output so collision cannot mask the path rejection. invalid[invalid.index("--output") + 1] = "bad-plan.json" rejected = invoke(invalid) self.assertEqual(rejected.returncode, 2) @@ -104,7 +99,10 @@ class NativeEvidenceRecipeTests(unittest.TestCase): self.assertEqual(stat.S_IMODE(manifest_path.stat().st_mode), mode) self.assertFalse((fixture.artifacts / "bad-plan.json").exists()) self.assertFalse((fixture.artifacts / "bad-output.json").exists()) - self.assertEqual(fixtures.run_git(fixture.repo, "status", "--porcelain").stdout, "") + self.assertEqual(run_git(fixture.repo, "status", "--porcelain").stdout, "") finally: - if hasattr(fixture, "tmp"): - fixture.tearDown() + fixture.cleanup() + + +if __name__ == "__main__": + unittest.main() diff --git a/harness/tests/review/journal/__init__.py b/harness/tests/review/journal/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/harness/tests/review/test_publication_journal_safety.py b/harness/tests/review/journal/test_journal_filesystem_safety.py similarity index 97% rename from harness/tests/review/test_publication_journal_safety.py rename to harness/tests/review/journal/test_journal_filesystem_safety.py index f7f2670..e862c13 100644 --- a/harness/tests/review/test_publication_journal_safety.py +++ b/harness/tests/review/journal/test_journal_filesystem_safety.py @@ -7,7 +7,9 @@ import tempfile import unittest from pathlib import Path -SCRIPT = Path(__file__).resolve().parents[2] / 'agents/review/skills/gitea-code-review/scripts/publication_journal.py' +from harness.tests._support.paths import ROOT + +SCRIPT = ROOT / 'harness/agents/review/skills/gitea-code-review/scripts/publication_journal.py' def load(): spec = importlib.util.spec_from_file_location('publication_journal_safety', SCRIPT) diff --git a/harness/tests/review/journal/test_journal_safety.py b/harness/tests/review/journal/test_journal_safety.py new file mode 100644 index 0000000..3585d4a --- /dev/null +++ b/harness/tests/review/journal/test_journal_safety.py @@ -0,0 +1,78 @@ +import copy +import importlib.util +import os +import tempfile +import unittest +from pathlib import Path + +from harness.tests._support.paths import ROOT +SCRIPT = ROOT / 'harness/agents/review/skills/gitea-code-review/scripts/native_review.py' +FIXTURES = ROOT / 'harness/tests/_support/review.py' + +def load(path, name): + spec = importlib.util.spec_from_file_location(name, path) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + + +class JournalSafetyTests(unittest.TestCase): + def test_binding_path_is_stable_and_head_scoped(self): + m = load(SCRIPT, 'native_review_canonical_test') + fixtures = load(FIXTURES, 'native_review_fixture_test') + plan = fixtures.plan_fixture() + first = m.canonical_journal_path(plan) + again = m.canonical_journal_path(copy.deepcopy(plan)) + self.assertEqual(first, again) + changed = copy.deepcopy(plan) + changed['target']['head'] = 'b' * 40 + changed['body'] = changed['body'].replace('head=' + plan['target']['head'], 'head=' + changed['target']['head']) + self.assertNotEqual(first, m.canonical_journal_path(changed)) + changed = copy.deepcopy(plan) + changed['reviewer']['id'] = 43 + self.assertNotEqual(first, m.canonical_journal_path(changed)) + self.assertEqual(first.parts[-2], 'v1') + self.assertEqual(first.parts[-3], 'native-review-journals') + def test_missing_entry_with_presence_anchor_fails_closed(self): + m = load(SCRIPT, 'native_review_anchor_test') + fixtures = load(FIXTURES, 'native_review_anchor_fixture_test') + plan = fixtures.plan_fixture() + with tempfile.TemporaryDirectory() as td: + root = Path(td) / 'native-review-journals' + old = m._journal.root_for + m._journal.root_for = lambda anchor: root + try: + path = m.canonical_journal_path(plan) + entry = m._journal.open_binding(path, plan, anchor_file=str(SCRIPT), + origin=m.ORIGIN, validate=m.validate_plan) + self.assertTrue((entry / 'binding.json').exists()) + import shutil + shutil.rmtree(entry) + with self.assertRaises(ValueError): + m._journal.open_binding(path, plan, anchor_file=str(SCRIPT), + origin=m.ORIGIN, validate=m.validate_plan) + finally: + m._journal.root_for = old + def test_deleted_root_with_persistent_anchor_fails_closed(self): + m = load(SCRIPT, 'native_review_root_loss_test') + fixtures = load(FIXTURES, 'native_review_root_loss_fixture_test') + plan = fixtures.plan_fixture() + with tempfile.TemporaryDirectory() as td: + root = Path(td) / 'native-review-journals' + old = m._journal.root_for + m._journal.root_for = lambda anchor: root + try: + path = m.canonical_journal_path(plan) + m._journal.open_binding(path, plan, anchor_file=str(SCRIPT), + origin=m.ORIGIN, validate=m.validate_plan) + import shutil + shutil.rmtree(root) + with self.assertRaises(ValueError): + m._journal.open_binding(path, plan, anchor_file=str(SCRIPT), + origin=m.ORIGIN, validate=m.validate_plan) + finally: + m._journal.root_for = old + +if __name__ == '__main__': + unittest.main() diff --git a/harness/tests/review/journal/test_legacy_journal_contract.py b/harness/tests/review/journal/test_legacy_journal_contract.py new file mode 100644 index 0000000..48c503b --- /dev/null +++ b/harness/tests/review/journal/test_legacy_journal_contract.py @@ -0,0 +1,220 @@ +import hashlib +import importlib.util +import json +import os +import stat +import tempfile +import unittest +from pathlib import Path + +from harness.tests._support.paths import ROOT +SCRIPT = ROOT / 'harness/agents/review/skills/gitea-code-review/scripts/native_review.py' +FIXTURES = ROOT / 'harness/tests/_support/review.py' + + +def load(path, name): + spec = importlib.util.spec_from_file_location(name, path) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +class LegacyJournalContractTests(unittest.TestCase): + + def test_legacy_inventory_rejects_unknown_artifact_before_import(self): + m = load(SCRIPT, 'legacy_inventory_contract') + f = load(FIXTURES, 'legacy_inventory_fixture') + plan, before, after, exact = f.publication_fixture() + with tempfile.TemporaryDirectory(dir='/opt/data/cache/scratch') as td: + m._journal.root_for = lambda anchor: Path(td) / 'native-review-journals' + source = Path(td) / 'legacy'; source.mkdir(mode=0o700) + with m.Journal(source) as journal: + journal.write('before.json', before) + journal.write('intent.json', m._expected_intent(plan, before)) + extra = source / 'unknown.json'; extra.write_bytes(b'{"unknown":true}') + extra.chmod(0o600) + with self.assertRaises(m.ValidationError): + m.legacy_journal_digest(source) + + def test_import_preserves_raw_bytes_and_archives_verified_separately(self): + m = load(SCRIPT, 'legacy_raw_import_contract') + f = load(FIXTURES, 'legacy_raw_import_fixture') + plan, before, after, exact = f.publication_fixture() + with tempfile.TemporaryDirectory(dir='/opt/data/cache/scratch') as td: + m._journal.root_for = lambda anchor: Path(td) / 'native-review-journals' + source = Path(td) / 'legacy'; source.mkdir(mode=0o700) + intent = m._expected_intent(plan, before) + before_raw = (json.dumps(before, sort_keys=True, indent=1) + '\n').encode() + intent_raw = (json.dumps(intent, sort_keys=True, indent=1) + '\n').encode() + verified_raw = b'{\n "verified": true,\n "legacy": "exact"\n}\n' + for name, raw in [('before.json', before_raw), ('intent.json', intent_raw), + ('verified.json', verified_raw)]: + path = source / name; path.write_bytes(raw); path.chmod(0o600) + with m.Journal(source): + pass + original = {name: (source / name).read_bytes() for name in + ['before.json', 'intent.json', 'verified.json']} + digest = m.legacy_journal_digest(source) + result = m.register_legacy_journal( + plan, source, expected_legacy_sha256=digest, + api=f.FakeAPI(plan, before, after, exact), lease=f.FakeLease(), + declaration={'mode': 'none'}) + self.assertTrue(result['read_only']) + target = m.canonical_journal_path(plan) + self.assertEqual((target / 'before.json').read_bytes(), original['before.json']) + self.assertEqual((target / 'intent.json').read_bytes(), original['intent.json']) + self.assertEqual((target / 'legacy-verified.json').read_bytes(), original['verified.json']) + self.assertFalse((target / 'verified.json').exists()) + self.assertEqual(stat.S_IMODE((target / 'legacy-verified.json').stat().st_mode), 0o600) + api = f.FakeAPI(plan, before, after, exact); api.created = True + for _ in range(2): + result = m.publish(plan, before, api, f.FakeLease(), declaration={'mode': 'none'}) + self.assertFalse(result['wrote']) + self.assertEqual(api.posts, 0) + self.assertTrue((target / 'verified.json').exists()) + self.assertEqual((target / 'legacy-verified.json').read_bytes(), verified_raw) + (target / 'legacy-verified.json').write_bytes(b'{"verified": false}') + with self.assertRaisesRegex(m.ValidationError, 'integrity mismatch'): + m.publish(plan, before, api, f.FakeLease(), declaration={'mode': 'none'}) + self.assertEqual(api.posts, 0) + + + + def test_receipt_cannot_omit_imported_verified_archive(self): + m = load(SCRIPT, 'legacy_receipt_contract'); f = load(FIXTURES, 'legacy_receipt_fixture') + plan, before, after, exact = f.publication_fixture() + with tempfile.TemporaryDirectory(dir='/opt/data/cache/scratch') as td: + m._journal.root_for = lambda anchor: Path(td) / 'native-review-journals' + source = Path(td) / 'legacy' + with m.Journal(source) as journal: + journal.write('before.json', before) + journal.write('intent.json', m._expected_intent(plan, before)) + journal.write('verified.json', {'verified': True}) + m.register_legacy_journal(plan, source, expected_legacy_sha256=m.legacy_journal_digest(source), + api=f.FakeAPI(plan, before, after, exact), lease=f.FakeLease(), declaration={'mode': 'none'}) + target = m.canonical_journal_path(plan) + receipt = json.loads((target / 'registration.json').read_text()) + receipt['imported_files'].pop() + with m.Journal(target) as journal: + journal.write('registration.json', receipt) + with self.assertRaises(m.ValidationError): m._check_registration(journal, plan) + +class LegacyJournalProcessContractTests(unittest.TestCase): + def test_valid_legacy_registration_is_read_only_and_preserves_source_bytes(self): + p, before, after, exact = self.plan_data(); source = self._legacy_files(p, before) + original = {name: (source / name).read_bytes() for name in ['before.json', 'intent.json', 'verified.json']} + modes = {name: stat.S_IMODE((source / name).stat().st_mode) for name in original} + api = self.fixtures.FakeAPI(p, before, after, exact) + result = self.m.register_legacy_journal(p, source, expected_legacy_sha256=self._legacy_digest(source), + api=api, lease=self.fixtures.FakeLease(), declaration=self.declaration()) + self.assertTrue(result['read_only']); self.assertEqual(api.posts, 0) + for name, data in original.items(): + self.assertEqual((source / name).read_bytes(), data) + self.assertEqual(stat.S_IMODE((source / name).stat().st_mode), modes[name]) + target = self.m.canonical_journal_path(p) + self.assertEqual((target / 'before.json').read_bytes(), original['before.json']) + self.assertFalse((target / 'import.pending.json').exists()) + def test_legacy_source_mutation_digest_mismatch_rejects_without_post_or_target_intent(self): + p, before, after, exact = self.plan_data(); source = self._legacy_files(p, before) + digest = self._legacy_digest(source) + mutated = json.loads((source / 'before.json').read_text()); mutated['issue']['body'] = 'changed' + (source / 'before.json').write_text(json.dumps(mutated, separators=(',', ':'))) + api = self.fixtures.FakeAPI(p, before, after, exact) + with self.assertRaises(self.m.ValidationError): + self.m.register_legacy_journal(p, source, expected_legacy_sha256=digest, api=api, + lease=self.fixtures.FakeLease(), declaration=self.declaration()) + self.assertEqual(api.posts, 0) + self.assertFalse((self.m.canonical_journal_path(p) / 'intent.json').exists()) + + def test_legacy_extra_intent_fields_union_rejects_without_post_or_target_intent(self): + p, before, after, exact = self.plan_data(); source = self._legacy_files(p, before) + intent = json.loads((source / 'intent.json').read_text()); intent['unexpected'] = True + (source / 'intent.json').write_text(json.dumps(intent, separators=(',', ':'))) + api = self.fixtures.FakeAPI(p, before, after, exact) + with self.assertRaises(self.m.ValidationError): + self.m.register_legacy_journal(p, source, expected_legacy_sha256=self._legacy_digest(source), api=api, + lease=self.fixtures.FakeLease(), declaration=self.declaration()) + self.assertEqual(api.posts, 0) + self.assertFalse((self.m.canonical_journal_path(p) / 'intent.json').exists()) + + m = load(SCRIPT, 'legacy_contract_publisher') + f = load(FIXTURES, 'legacy_contract_fixture') + plan, before, after, exact = f.publication_fixture() + with tempfile.TemporaryDirectory(dir='/opt/data/cache/scratch') as td: + m._journal.root_for = lambda anchor: Path(td) / 'native-review-journals' + source = Path(td) / 'legacy' + payload = m.build_payload(plan) + intent = {'schema': 'gitea-agent.native-review-intent/v1', + 'fingerprint': hashlib.sha256(m.canonical(plan)).hexdigest(), + 'before_sha256': hashlib.sha256(m.canonical(before)).hexdigest(), + 'head': plan['target']['head'], 'event': payload['event'], + 'body_sha256': hashlib.sha256(payload['body'].encode()).hexdigest(), + 'unexpected': 'cannot silently import this'} + with m.Journal(source) as journal: + journal.write('before.json', before) + journal.write('intent.json', intent) + digest = m.legacy_journal_digest(source) + api = f.FakeAPI(plan, before, after, exact) + with self.assertRaises(m.ValidationError): + m.register_legacy_journal(plan, source, expected_legacy_sha256=digest, + api=api, lease=f.FakeLease(), declaration={'mode': 'none'}) + self.assertEqual(api.posts, 0) + self.assertFalse((m.canonical_journal_path(plan) / 'intent.json').exists()) + def test_legacy_symlink_hardlink_and_duplicate_json_are_rejected(self): + p, before, after, exact = self.plan_data() + source = self._legacy_files(p, before) + (source / 'before.json').unlink(); (source / 'before.json').symlink_to(Path(self.tmp.name) / 'elsewhere') + with self.assertRaises(self.m.ValidationError): + self.m.register_legacy_journal(p, source, expected_legacy_sha256='0' * 64, api=self.fixtures.FakeAPI(p, before, after, exact), + lease=self.fixtures.FakeLease(), declaration=self.declaration()) + source = self._legacy_files(p, before) + os.link(source / 'before.json', source / 'duplicate.json') + with self.assertRaises(self.m.ValidationError): + self.m.register_legacy_journal(p, source, expected_legacy_sha256=self._legacy_digest(source), api=self.fixtures.FakeAPI(p, before, after, exact), + lease=self.fixtures.FakeLease(), declaration=self.declaration()) + def test_registration_rejects_existing_canonical_binding_without_post(self): + p, before, after, exact = self.plan_data(); source = self._legacy_files(p, before) + target = self.m.canonical_journal_path(p) + target.parent.mkdir(parents=True, exist_ok=True); target.mkdir(mode=0o700) + (target / 'intent.json').write_text('{}'); (target / 'intent.json').chmod(0o600) + api = self.fixtures.FakeAPI(p, before, after, exact) + with self.assertRaises(self.m.ValidationError): + self.m.register_legacy_journal(p, source, expected_legacy_sha256=self._legacy_digest(source), api=api, + lease=self.fixtures.FakeLease(), declaration=self.declaration()) + self.assertEqual(api.posts, 0) + def setUp(self): + self.m = load(SCRIPT, 'native_review_process_' + str(id(self))) + self.fixtures = load(FIXTURES, 'native_review_process_fixture_' + str(id(self))) + self.tmp = tempfile.TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + self.root = Path(self.tmp.name) / 'native-review-journals' + self.m._journal.root_for = lambda anchor, root=self.root: root + def plan_data(self): + p, before, after, exact = self.fixtures.publication_fixture() + return p, before, after, exact + def declaration(self): + return {'mode': 'none'} + + def _legacy_files(self, p, before, *, verified=True): + directory = Path(tempfile.mkdtemp(prefix='legacy-', dir=self.tmp.name)) + directory.chmod(0o700) + canonical = self.m.canonical + intent = {'schema': 'gitea-agent.native-review-intent/v1', + 'fingerprint': hashlib.sha256(self.m.canonical(p)).hexdigest(), + 'before_sha256': hashlib.sha256(self.m.canonical(before)).hexdigest(), + 'head': p['target']['head'], 'event': 'APPROVED', + 'body_sha256': hashlib.sha256(p['body'].encode()).hexdigest()} + with self.m.Journal(directory) as journal: + journal.write('before.json', before) + journal.write('intent.json', intent) + if verified: + journal.write('verified.json', {'verified': True}) + return directory + def _legacy_digest(self, directory): + helper = getattr(self.m, 'legacy_journal_digest', None) + self.assertTrue(callable(helper), 'public legacy journal digest helper is required') + return helper(directory) + + +if __name__ == '__main__': + unittest.main() diff --git a/harness/tests/review/journal/test_publication_processes.py b/harness/tests/review/journal/test_publication_processes.py new file mode 100644 index 0000000..9d2170e --- /dev/null +++ b/harness/tests/review/journal/test_publication_processes.py @@ -0,0 +1,210 @@ +import copy +import hashlib +import importlib.util +import json +import os +import stat +import subprocess +import sys +import tempfile +import textwrap +import time +import unittest +from pathlib import Path + +from harness.tests._support.paths import ROOT +SCRIPT = ROOT / 'harness/agents/review/skills/gitea-code-review/scripts/native_review.py' +FIXTURES = ROOT / 'harness/tests/_support/review.py' + + +def load(path, name): + spec = importlib.util.spec_from_file_location(name, path) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module +class PublicationJournalProcessTests(unittest.TestCase): + def test_pending_intent_survives_process_crash_and_blocks_second_post(self): + p, before, after, exact = self.plan_data() + plan_file = Path(self.tmp.name) / 'plan.json'; plan_file.write_text(json.dumps(p, ensure_ascii=False)) + counter = Path(self.tmp.name) / 'posts' + crash = self.child_prelude() + textwrap.dedent(''' + class CrashAPI(API): + def create_review(self, target, body): + Path(sys.argv[4]).write_text('create-entered') + os._exit(17) + m.publish(p, before, CrashAPI(), Lease(), m.canonical_journal_path(p), declaration={'mode': 'none'}) + ''') + entered = Path(self.tmp.name) / 'entered' + crashed = self.run_child(crash, self.root, plan_file, counter, entered) + self.assertEqual(crashed.returncode, 17) + self.assertEqual(entered.read_text(), 'create-entered') + resume = self.child_prelude() + textwrap.dedent(''' + try: + m.publish(p, before, API(), Lease(), m.canonical_journal_path(p), declaration={'mode': 'none'}) + except Exception as exc: + print(type(exc).__name__) + ''') + resumed = self.run_child(resume, self.root, plan_file, counter) + self.assertEqual(resumed.returncode, 0, resumed.stderr) + self.assertIn('UncertainPublication', resumed.stdout) + self.assertFalse(counter.exists(), 'resume must not issue a second POST') + def test_process_lock_contention_fails_fast_without_post(self): + p, before, after, exact = self.plan_data() + plan_file = Path(self.tmp.name) / 'plan.json'; plan_file.write_text(json.dumps(p)) + counter = Path(self.tmp.name) / 'posts' + ready = Path(self.tmp.name) / 'ready' + holder = self.child_prelude() + textwrap.dedent(''' + entry = m._journal.open_binding(m.canonical_journal_path(p), p, anchor_file=str(script), origin=m.ORIGIN, validate=m.validate_plan) + fd = m._journal.acquire_lock(entry); Path(sys.argv[4]).write_text('ready'); time.sleep(3); os.close(fd) + ''') + proc = subprocess.Popen([sys.executable, '-c', holder, str(self.root), str(plan_file), str(counter), str(ready)], + env={'PATH': os.environ['PATH'], 'PYTHONPATH': str(ROOT)}, text=True, + stdout=subprocess.PIPE, stderr=subprocess.PIPE) + try: + deadline = time.monotonic() + 3 + while not ready.exists() and time.monotonic() < deadline: time.sleep(.02) + self.assertTrue(ready.exists()) + contender = self.child_prelude() + textwrap.dedent(''' + try: + m.publish(p, before, API(), Lease(), m.canonical_journal_path(p), declaration={'mode': 'none'}) + except Exception as exc: + print(type(exc).__name__) + ''') + started = time.monotonic(); result = self.run_child(contender, self.root, plan_file, counter, timeout=2) + self.assertLess(time.monotonic() - started, 2) + self.assertEqual(result.returncode, 0, result.stderr) + self.assertIn('ValidationError', result.stdout) + self.assertFalse(counter.exists()) + finally: + proc.kill(); proc.communicate(timeout=3) + def test_crash_after_transport_write_reconciles_without_second_write(self): + p, _, _, _ = self.plan_data() + plan_file = Path(self.tmp.name) / 'plan.json'; plan_file.write_text(json.dumps(p)) + counter = Path(self.tmp.name) / 'posts' + crash = self.child_prelude() + textwrap.dedent(""" + class CrashAfterWrite(API): + def create_review(self, target, body): + super().create_review(target, body) + os._exit(17) + m.publish(p, before, CrashAfterWrite(), Lease(), declaration={'mode': 'none'}) + """) + result = self.run_child(crash, self.root, plan_file, counter) + self.assertEqual(result.returncode, 17, result.stderr) + self.assertEqual(counter.read_text(), '1') + resume = self.child_prelude() + textwrap.dedent(""" + result = m.publish(p, before, API(), Lease(), declaration={'mode': 'none'}) + print(result['mode']) + """) + result = self.run_child(resume, self.root, plan_file, counter) + self.assertEqual(result.returncode, 0, result.stderr) + self.assertIn('no_op', result.stdout) + self.assertEqual(counter.read_text(), '1') + def test_two_publishing_processes_create_only_once(self): + p, _, _, _ = self.plan_data() + plan_file = Path(self.tmp.name) / 'plan.json'; plan_file.write_text(json.dumps(p)) + counter = Path(self.tmp.name) / 'posts' + entered = Path(self.tmp.name) / 'create-entered'; release = Path(self.tmp.name) / 'release' + first = self.child_prelude() + textwrap.dedent(""" + class GatedAPI(API): + def create_review(self, target, body): + result = super().create_review(target, body) + Path(sys.argv[4]).write_text('entered') + deadline = time.monotonic() + 8 + while not Path(sys.argv[5]).exists(): + if time.monotonic() >= deadline: raise TimeoutError('test watchdog') + time.sleep(.02) + return result + result = m.publish(p, before, GatedAPI(), Lease(), declaration={'mode': 'none'}) + print(result['mode']) + """) + proc = subprocess.Popen([sys.executable, '-B', '-c', first, str(self.root), str(plan_file), str(counter), str(entered), str(release)], + env={'PATH': os.environ['PATH'], 'PYTHONPATH': str(ROOT)}, + text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + try: + deadline = time.monotonic() + 4 + while not entered.exists() and time.monotonic() < deadline: time.sleep(.02) + self.assertTrue(entered.exists(), 'first publisher must reach transport') + self.assertEqual(counter.read_text(), '1') + contender = self.child_prelude() + textwrap.dedent(""" + try: m.publish(p, before, API(), Lease(), declaration={'mode': 'none'}) + except m.ValidationError: print('blocked') + """) + result = self.run_child(contender, self.root, plan_file, counter, timeout=3) + self.assertEqual(result.returncode, 0, result.stderr) + self.assertIn('blocked', result.stdout) + self.assertEqual(counter.read_text(), '1') + release.write_text('release') + stdout, stderr = proc.communicate(timeout=3) + self.assertEqual(proc.returncode, 0, stderr) + self.assertIn('created', stdout) + self.assertEqual(counter.read_text(), '1') + finally: + if proc.poll() is None: proc.kill() + proc.communicate(timeout=3) + def setUp(self): + self.m = load(SCRIPT, 'native_review_process_' + str(id(self))) + self.fixtures = load(FIXTURES, 'native_review_process_fixture_' + str(id(self))) + self.tmp = tempfile.TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + self.root = Path(self.tmp.name) / 'native-review-journals' + self.m._journal.root_for = lambda anchor, root=self.root: root + def plan_data(self): + p, before, after, exact = self.fixtures.publication_fixture() + return p, before, after, exact + def declaration(self): + return {'mode': 'none'} + def run_child(self, script, *args, timeout=8): + env = {'PATH': os.environ['PATH'], 'PYTHONPATH': str(ROOT), 'PYTHONDONTWRITEBYTECODE': '1'} + return subprocess.run([sys.executable, '-c', script, *map(str, args)], env=env, + capture_output=True, text=True, timeout=timeout) + def child_prelude(self): + return textwrap.dedent(f''' + import copy, importlib.util, json, os, sys, time + from pathlib import Path + root = Path(sys.argv[1]); payload = json.loads(Path(sys.argv[2]).read_text()) + script = Path({str(SCRIPT)!r}); fixtures_path = Path({str(FIXTURES)!r}) + def load(path, name): + spec = importlib.util.spec_from_file_location(name, path) + m = importlib.util.module_from_spec(spec); spec.loader.exec_module(m); return m + m = load(script, 'native_child_' + str(os.getpid())) + f = load(fixtures_path, 'fixture_child_' + str(os.getpid())) + m._journal.root_for = lambda anchor: root + p, before, after, exact = f.publication_fixture() + class Lease: + def check(self, plan): return None + class API: + def __init__(self): self.created = Path(sys.argv[3]).exists() + def identity(self): return copy.deepcopy(p['reviewer']) + def snapshot(self, target): return copy.deepcopy(after if self.created else before) + def read_review(self, target, rid): return copy.deepcopy(exact) + def create_review(self, target, body): + counter = Path(sys.argv[3]); n = int(counter.read_text() or '0') if counter.exists() else 0 + counter.write_text(str(n + 1)) + with counter.open('r+') as fd: + fd.flush(); os.fsync(fd.fileno()) + self.created = True + return copy.deepcopy(exact) + api = API() + ''') + def _legacy_files(self, p, before, *, verified=True): + directory = Path(tempfile.mkdtemp(prefix='legacy-', dir=self.tmp.name)) + directory.chmod(0o700) + canonical = self.m.canonical + intent = {'schema': 'gitea-agent.native-review-intent/v1', + 'fingerprint': hashlib.sha256(self.m.canonical(p)).hexdigest(), + 'before_sha256': hashlib.sha256(self.m.canonical(before)).hexdigest(), + 'head': p['target']['head'], 'event': 'APPROVED', + 'body_sha256': hashlib.sha256(p['body'].encode()).hexdigest()} + with self.m.Journal(directory) as journal: + journal.write('before.json', before) + journal.write('intent.json', intent) + if verified: + journal.write('verified.json', {'verified': True}) + return directory + def _legacy_digest(self, directory): + helper = getattr(self.m, 'legacy_journal_digest', None) + self.assertTrue(callable(helper), 'public legacy journal digest helper is required') + return helper(directory) + +if __name__ == '__main__': + unittest.main() diff --git a/harness/tests/review/native/__init__.py b/harness/tests/review/native/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/harness/tests/review/native/test_native_lease.py b/harness/tests/review/native/test_native_lease.py new file mode 100644 index 0000000..1391224 --- /dev/null +++ b/harness/tests/review/native/test_native_lease.py @@ -0,0 +1,205 @@ +import copy +import importlib.util +import json +import os +import subprocess +import sys +import tempfile +import time +import unittest +from pathlib import Path + +from harness.tests._support.paths import ROOT +SCRIPT = ROOT / 'harness/agents/review/skills/gitea-code-review/scripts/native_review.py' + + +def load(path, name): + spec = importlib.util.spec_from_file_location(name, path) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +from harness.tests._support.review import ( + FakeAPI, + FakeLease, + plan_fixture, + publication_fixture, + support, +) +class TestNativeLease(unittest.TestCase): + def module(self): + self.assertTrue(SCRIPT.is_file(), 'shared native review adapter is missing') + m = load(SCRIPT, 'native_review_test_' + str(id(self))) + # Every test gets a private registry root; production has no test-only + # root override, so isolate through the journal module seam. + private_root = tempfile.TemporaryDirectory() + self.addCleanup(private_root.cleanup) + root = Path(private_root.name) / 'native-review-journals' + m._journal.root_for = lambda anchor, root=root: root + return m + @staticmethod + def canonical(m, plan): + return m.canonical_journal_path(plan) + def test_held_lease_checks_real_registered_worktree_and_expiry(self): + import fcntl + m = self.module() + self.assertTrue(callable(getattr(m, 'HeldIssueLease', None)), 'real Issue lease guard is missing') + with tempfile.TemporaryDirectory() as td: + root = Path(td); policy = root / 'harness/operations/repository-cache.md' + policy.parent.mkdir(parents=True); policy.write_text('canonical cache fixture') + rs = root / 'repositories'; owner = rs / 'acme'; cache = owner / 'repo'; locks = cache / 'locks' + for directory in [rs, owner, cache, locks]: directory.mkdir(mode=0o700) + bare = cache / 'repository.git'; seed = root / 'seed' + def git(*args): + out = subprocess.run(['git', '-c', 'core.hooksPath=/dev/null', *map(str, args)], capture_output=True, text=True, + env={'PATH': os.environ['PATH'], 'GIT_CONFIG_GLOBAL': '/dev/null', 'GIT_CONFIG_NOSYSTEM': '1'}) + self.assertEqual(out.returncode, 0, out.stderr) + return out.stdout.strip() + git('init', '--bare', bare); git('init', seed) + git('-C', seed, 'config', 'user.name', 'Synthetic'); git('-C', seed, 'config', 'user.email', 'fixture@example.test') + (seed / 'a.txt').write_text('synthetic') + git('-C', seed, 'add', 'a.txt'); git('-C', seed, 'commit', '-m', 'fixture') + git('--git-dir=' + str(bare), 'fetch', seed, 'HEAD:refs/heads/main') + wt = cache / 'worktrees/issue-3/checkout'; wt.parent.mkdir(parents=True, mode=0o700) + git('--git-dir=' + str(bare), 'worktree', 'add', '--detach', '--lock', wt, 'main') + head = git('-C', wt, 'rev-parse', 'HEAD') + p = plan_fixture(); p['target']['head'] = head; p['body'] = p['body'].replace('a' * 40, head) + state = wt.parent / 'state.json' + state.write_text(json.dumps({'schema': 'gitea-agent.worktree/v1', 'owner': 'acme', 'repository': 'repo', + 'issue': 3, 'created_by_role': 'gitea-impl-agent', 'mode': 'detached'})) + state.chmod(0o600) + path = locks / 'issue-3.lock'; fd = os.open(path, os.O_RDWR | os.O_CREAT, 0o600) + try: + fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB) + lease = m.HeldIssueLease(fd, policy, time.monotonic() + 30) + canonical_bytes = state.read_bytes(); canonical_stat = state.stat() + lease.check(p) + self.assertEqual(state.read_bytes(), canonical_bytes) + self.assertEqual((state.stat().st_mode, state.stat().st_mtime_ns, state.stat().st_ino), + (canonical_stat.st_mode, canonical_stat.st_mtime_ns, canonical_stat.st_ino)) + # The bounded compatibility admission is read-only: mutate only + # this scratch fixture from canonical producer state to legacy impl. + state.write_text(json.dumps({'schema': 'gitea-agent.worktree/v1', 'owner': 'acme', 'repository': 'repo', + 'issue': 3, 'created_by_role': 'impl', 'mode': 'detached'})) + state.chmod(0o600) + legacy_bytes = state.read_bytes(); legacy_stat = state.stat() + try: + lease.check(p) + except m.ValidationError as exc: + self.fail(f'valid legacy impl registration must be admitted: {exc}') + after_stat = state.stat() + self.assertEqual(state.read_bytes(), legacy_bytes) + self.assertEqual((after_stat.st_mode, after_stat.st_mtime_ns, after_stat.st_ino), + (legacy_stat.st_mode, legacy_stat.st_mtime_ns, legacy_stat.st_ino)) + for bad_role in [None, True, False, 1, [], {}, '', ' impl', 'impl ', 'Impl', 'IMPL', 'gitea-impl-agent ', 'review', 'gitea-review-agent', 'gitea-spec-agent', 'gitea-release-agent', 'other']: + state.write_text(json.dumps({'schema': 'gitea-agent.worktree/v1', 'owner': 'acme', 'repository': 'repo', + 'issue': 3, 'created_by_role': bad_role, 'mode': 'detached'})) + state.chmod(0o600) + with self.subTest(bad_role=repr(bad_role)), self.assertRaisesRegex(m.ValidationError, 'worktree registration identity mismatch'): + lease.check(p) + missing_role = json.loads(legacy_bytes); missing_role.pop('created_by_role') + state.write_text(json.dumps(missing_role)); state.chmod(0o600) + with self.assertRaisesRegex(m.ValidationError, 'worktree registration identity mismatch'): lease.check(p) + bool_issue = json.loads(legacy_bytes); bool_issue['issue'] = True + state.write_text(json.dumps(bool_issue)); state.chmod(0o600) + with self.assertRaisesRegex(m.ValidationError, 'worktree registration identity mismatch'): lease.check(p) + state.write_text(json.dumps({'schema': 'gitea-agent.worktree/v1', 'owner': 'other', 'repository': 'repo', + 'issue': 3, 'created_by_role': 'impl', 'mode': 'detached'})); state.chmod(0o600) + with self.assertRaisesRegex(m.ValidationError, 'worktree registration identity mismatch'): lease.check(p) + state.write_text(json.dumps({'schema': 'gitea-agent.worktree/v1', 'owner': 'acme', 'repository': 'other', + 'issue': 3, 'created_by_role': 'impl', 'mode': 'detached'})); state.chmod(0o600) + with self.assertRaisesRegex(m.ValidationError, 'worktree registration identity mismatch'): lease.check(p) + state.write_text(json.dumps({'schema': 'gitea-agent.worktree/v1', 'owner': 'acme', 'repository': 'repo', + 'issue': 4, 'created_by_role': 'impl', 'mode': 'detached'})); state.chmod(0o600) + with self.assertRaisesRegex(m.ValidationError, 'worktree registration identity mismatch'): lease.check(p) + state.write_text(json.dumps({'schema': 'wrong/v1', 'owner': 'acme', 'repository': 'repo', + 'issue': 3, 'created_by_role': 'impl', 'mode': 'detached'})); state.chmod(0o600) + with self.assertRaisesRegex(m.ValidationError, 'worktree registration identity mismatch'): lease.check(p) + state.write_text(json.dumps({'schema': 'gitea-agent.worktree/v1', 'owner': 'acme', 'repository': 'repo', + 'issue': 3, 'created_by_role': 'impl', 'mode': 'attached'})); state.chmod(0o600) + with self.assertRaisesRegex(m.ValidationError, 'worktree registration identity mismatch'): lease.check(p) + state.write_text(json.dumps({'schema': 'gitea-agent.worktree/v1', 'owner': 'acme', 'repository': 'repo', + 'issue': 3, 'created_by_role': 'impl', 'mode': 'detached'})); state.chmod(0o600) + wrong_head = copy.deepcopy(p); wrong_head['target']['head'] = 'b' * 40 + wrong_head['body'] = wrong_head['body'].replace(head, 'b' * 40) + with self.assertRaisesRegex(m.ValidationError, 'worktree HEAD mismatch'): lease.check(wrong_head) + fcntl.flock(fd, fcntl.LOCK_UN) + with self.assertRaises(m.ValidationError): lease.check(p) + self.assertNotIn('lock:', Path(f'/proc/self/fdinfo/{fd}').read_text()) + fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB) + from unittest.mock import patch + fakebin = root / 'fakebin'; fakebin.mkdir() + marker = root / 'path-git-executed' + fakegit = fakebin / 'git' + fakegit.write_text(f'#!/bin/sh\n/usr/bin/touch "{marker}"\nexit 1\n'); fakegit.chmod(0o700) + with patch.dict(os.environ, {'PATH': str(fakebin)}): lease.check(p) + self.assertFalse(marker.exists()) + lease.deadline_monotonic = time.monotonic() - 1 + with self.assertRaises(m.ValidationError): lease.check(p) + lease.deadline_monotonic = time.monotonic() + 30 + (wt / 'a.txt').write_text('dirty') + with self.assertRaises(m.ValidationError): lease.check(p) + # Every failed source validation must close all transient FDs. + before_fds = len(list((Path('/proc/self/fd')).iterdir())) + for _ in range(20): + with self.assertRaises(m.ValidationError): lease.check(p) + after_fds = len(list((Path('/proc/self/fd')).iterdir())) + self.assertLessEqual(after_fds, before_fds + 1) + (wt / 'a.txt').unlink() + for directory in [wt.parent.parent, wt.parent, wt]: + directory.chmod(0o770) + with self.subTest(directory=directory), self.assertRaises(m.ValidationError): + lease.check(p) + directory.chmod(0o700) + finally: + os.close(fd) + def test_false_lease_result_does_not_authorize_publication(self): + m = self.module(); p, before, after, exact = publication_fixture(); api = FakeAPI(p, before, after, exact) + lease = FakeLease(); lease.check = lambda plan: False + with tempfile.TemporaryDirectory() as td: + with self.assertRaises(m.ValidationError): m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'}) + self.assertEqual(api.posts, 0) + def test_expired_lease_does_not_create_new_journal_directory(self): + m = self.module(); p, before, after, exact = publication_fixture() + lease = FakeLease(); lease.valid = False + with tempfile.TemporaryDirectory() as td: + directory = Path(td) / 'not-created' + with self.assertRaises(m.ValidationError): m.publish(p, before, FakeAPI(p, before, after, exact), lease, directory) + self.assertFalse(directory.exists()) + def test_broker_issue_lease_accepts_authenticated_matching_status(self): + m = self.module(); p, before, after, exact = publication_fixture() + now = time.monotonic() + lease = m.BrokerIssueLease('/trusted/handle.json', '/trusted/policy.md', now + 30) + handle = {'owner': 'acme', 'repository': 'repo', 'issue': 3, 'role': 'gitea-review-agent', + 'pid': 1234, 'deadline': now + 20, 'token': 'not-returned', 'socket': '/trusted/broker.sock'} + status = {'status': 'held', 'owner': 'acme', 'repository': 'repo', 'issue': 3, + 'role': 'gitea-review-agent', 'pid': 1234, 'deadline': handle['deadline']} + from unittest.mock import patch + with patch.object(m._broker, 'read_handle_metadata', return_value={k: v for k, v in handle.items() if k != 'token'}), \ + patch.object(m._broker, 'request_handle', return_value=status), \ + patch.object(m, '_validate_registered_source') as source: + self.assertIsNone(lease.check(p)) + source.assert_called_once_with(p, '/trusted/policy.md', lease.deadline_monotonic) + def test_broker_issue_lease_rejects_wrong_identity_status_or_deadline(self): + m = self.module(); p, before, after, exact = publication_fixture(); now = time.monotonic() + lease = m.BrokerIssueLease('/trusted/handle.json', '/trusted/policy.md', now + 30) + handle = {'pid': 1234, 'deadline': now + 20, 'token': 'hidden', 'socket': '/trusted/broker.sock'} + cases = [ + {'status': 'held', 'owner': 'acme', 'repository': 'repo', 'issue': 3, 'role': 'gitea-spec-agent', 'pid': 1234, 'deadline': handle['deadline']}, + {'status': 'held', 'owner': 'other', 'repository': 'repo', 'issue': 3, 'role': 'gitea-review-agent', 'pid': 1234, 'deadline': handle['deadline']}, + {'status': 'released', 'owner': 'acme', 'repository': 'repo', 'issue': 3, 'role': 'gitea-review-agent', 'pid': 1234, 'deadline': handle['deadline']}, + {'status': 'held', 'owner': 'acme', 'repository': 'repo', 'issue': 3, 'role': 'gitea-review-agent', 'pid': 9999, 'deadline': handle['deadline']}, + {'status': 'held', 'owner': 'acme', 'repository': 'repo', 'issue': 3, 'role': 'gitea-review-agent', 'pid': 1234, 'deadline': now + 40}, + ] + from unittest.mock import patch + for status in cases: + with self.subTest(status=status): + with patch.object(m._broker, 'read_handle_metadata', return_value={k: v for k, v in handle.items() if k != 'token'}), \ + patch.object(m._broker, 'request_handle', return_value=status), \ + patch.object(m, '_validate_registered_source') as source: + with self.assertRaises(m.ValidationError): lease.check(p) + source.assert_not_called() + +if __name__ == '__main__': + unittest.main() diff --git a/harness/tests/review/native/test_native_publication.py b/harness/tests/review/native/test_native_publication.py new file mode 100644 index 0000000..bb6218c --- /dev/null +++ b/harness/tests/review/native/test_native_publication.py @@ -0,0 +1,383 @@ +import copy +import importlib.util +import json +import os +import subprocess +import sys +import tempfile +import time +import unittest +from pathlib import Path + +from harness.tests._support.paths import ROOT +SCRIPT = ROOT / 'harness/agents/review/skills/gitea-code-review/scripts/native_review.py' + + +def load(path, name): + spec = importlib.util.spec_from_file_location(name, path) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +from harness.tests._support.review import ( + FakeAPI, + FakeLease, + plan_fixture, + publication_fixture, + support, +) +class TestNativePublication(unittest.TestCase): + def module(self): + self.assertTrue(SCRIPT.is_file(), 'shared native review adapter is missing') + m = load(SCRIPT, 'native_review_test_' + str(id(self))) + # Every test gets a private registry root; production has no test-only + # root override, so isolate through the journal module seam. + private_root = tempfile.TemporaryDirectory() + self.addCleanup(private_root.cleanup) + root = Path(private_root.name) / 'native-review-journals' + m._journal.root_for = lambda anchor, root=root: root + return m + @staticmethod + def canonical(m, plan): + return m.canonical_journal_path(plan) + def test_payload_is_exact_bound_native_create(self): + m = self.module() + p = plan_fixture() + self.assertEqual(m.build_payload(p), {'commit_id': p['target']['head'], + 'body': p['body'], 'event': 'APPROVED', 'comments': []}) + bad = copy.deepcopy(p); bad['target']['pull'] = True + with self.assertRaises(m.ValidationError): + m.build_payload(bad) + bad = copy.deepcopy(p); bad['body'] = bad['body'].replace('a' * 40, 'd' * 40) + with self.assertRaises(m.ValidationError): + m.build_payload(bad) + def test_verify_exact_target_and_bound_provider_changes(self): + m = self.module() + self.assertTrue(callable(getattr(m, 'verify_publication', None)), 'readback verifier is missing') + p, before, after, exact = publication_fixture() + previous = after['reviews'][0] + previous.update(official=False, dismissed=True, updated_at=exact['submitted_at']) + after['commits'][0]['files'].reverse() + result = m.verify_publication(p, before, after, exact) + self.assertEqual(result['review_id'], 10) + self.assertEqual(result['projected_review_ids'], [8]) + self.assertTrue(result['commit_file_order_normalized']) + for mutation in ['head', 'body', 'unrelated_review', 'commit_file']: + bad = copy.deepcopy(after) + if mutation == 'head': bad['pr']['head']['sha'] = 'e' * 40 + elif mutation == 'body': bad['reviews'][-1]['body'] += ' altered' + elif mutation == 'unrelated_review': bad['reviews'][1]['body'] += ' altered' + else: bad['commits'][0]['files'][0]['status'] = 'removed' + with self.subTest(mutation=mutation), self.assertRaises(m.ValidationError): + m.verify_publication(p, before, bad, exact) + def test_publish_once_and_reconcile_durable_attempt_without_reposting(self): + m = self.module() + self.assertTrue(callable(getattr(m, 'publish', None)), 'shared publisher is missing') + p, before, after, exact = publication_fixture() + api = FakeAPI(p, before, after, exact); lease = FakeLease() + with tempfile.TemporaryDirectory() as td: + result = m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'}) + self.assertEqual(result['mode'], 'created') + self.assertEqual(api.posts, 1) + self.assertEqual(m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'})['mode'], 'no_op') + self.assertEqual(api.posts, 1) + self.assertNotIn('body', json.loads((self.canonical(m, p) / 'intent.json').read_text())) + for applies in [True, False]: + m = self.module(); p, before, after, exact = publication_fixture() + api = FakeAPI(p, before, after, exact); api.timeout = True; api.apply = applies + with self.subTest(applies=applies), tempfile.TemporaryDirectory() as td: + if applies: + self.assertEqual(m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'})['mode'], 'reconciled') + self.assertEqual(m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'})['mode'], 'no_op') + else: + for _ in range(2): + with self.assertRaises(m.UncertainPublication) as error: + m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'}) + self.assertNotIn('secret transport detail', str(error.exception)) + self.assertEqual(api.posts, 1) + def test_empty_review_comments_no_header_contract_is_explicit(self): + text = (ROOT / 'harness/agents/review/skills/gitea-code-review/references/native-review.md').read_text() + self.assertIn('### Empty review-comments readback without a count header', text) + section = text.split('### Empty review-comments readback without a count header', 1)[1].split('\n### ', 1)[0] + for rule in ['X-Total-Count', 'comments_count=0', 'JSON `[]`', 'declared_total=None', 'rel=next']: + self.assertIn(rule, section) + def test_cli_preflight_and_saved_wire_verification_are_read_only(self): + m = self.module(); p, before, after, exact = publication_fixture() + with tempfile.TemporaryDirectory() as td: + root = Path(td) + for name, value in [('plan', p), ('before', before), ('after', after), ('exact', exact)]: + (root / (name + '.json')).write_text(json.dumps(value, ensure_ascii=False)) + base = [sys.executable, str(SCRIPT)] + env = {'PATH': os.environ['PATH'], 'PYTHONDONTWRITEBYTECODE': '1'} + pre = subprocess.run(base + ['preflight', '--plan', str(root / 'plan.json'), + '--baseline', str(root / 'before.json')], capture_output=True, text=True, env=env) + self.assertEqual(pre.returncode, 0, pre.stderr) + self.assertIn('"ready": true', pre.stdout, 'read-only preflight CLI is missing') + verified = subprocess.run(base + ['verify', '--plan', str(root / 'plan.json'), + '--before', str(root / 'before.json'), '--after', str(root / 'after.json'), + '--exact', str(root / 'exact.json')], capture_output=True, text=True, env=env) + self.assertEqual(verified.returncode, 0, verified.stderr) + self.assertTrue(json.loads(verified.stdout)['read_only']) + self.assertEqual(json.loads(verified.stdout)['review_id'], 10) + unsupported = subprocess.run(base + ['publish'], capture_output=True, text=True, env=env) + self.assertNotEqual(unsupported.returncode, 0) + def test_spec_approval_may_reference_unchanged_surrounding_test_paths(self): + m = self.module(); p, before, after, exact = publication_fixture() + p['spec_paths'] = ['src/a.py', 'tests/existing.py'] + spec_result = support.parse_review(before['reviews'][1]['body'], {'src/a.py'}) + spec_result['criteria'][0]['test_paths'] = ['tests/existing.py'] + before['reviews'][1]['body'] = support.render_review(spec_result, set(p['spec_paths']), declaration={'mode': 'none'}) + try: + m.validate_snapshot(p, before) + except m.ValidationError: + self.fail('spec approval must use its declared scope, not only changed files') + def test_malformed_historical_actor_and_missing_exact_review_flags_fail_closed(self): + m = self.module(); p, before, after, exact = publication_fixture() + bad = copy.deepcopy(before); bad['reviews'][0]['user'] = {'id': True, 'login': 'x'} + with self.assertRaises(m.ValidationError): + m.validate_snapshot(p, bad) + p, before, after, exact = publication_fixture() + del exact['official']; del after['reviews'][-1]['official'] + with self.assertRaises(m.ValidationError): + m.verify_publication(p, before, after, exact) + def test_invalid_plan_bindings_are_not_normalized(self): + m = self.module() + mutations = [lambda p: p['target'].__setitem__('issue', True), + lambda p: p['target'].__setitem__('head', 'A' * 40), + lambda p: p['target'].__setitem__('owner', 'acme/other'), + lambda p: p.__setitem__('verdict', 'blocked'), + lambda p: p.__setitem__('body', p['body'] + '- Verdict: `approve`\n'), + lambda p: p['provider_contract'].__setitem__('evidence', []), + lambda p: p.__setitem__('spec_paths', ['../secret']), + lambda p: p.__setitem__('spec_paths', ['src/a.py', 'src/a.py']), + lambda p: p['reviewer'].__setitem__('id', p['spec_reviewer']['id'])] + for i, mutate in enumerate(mutations): + with self.subTest(case=i): + p = plan_fixture(); mutate(p) + with self.assertRaises(m.ValidationError): m.build_payload(p) + def test_wrong_role_and_expired_lease_prevent_any_post(self): + m = self.module() + for kind in ['wrong_role', 'expired']: + with self.subTest(kind=kind), tempfile.TemporaryDirectory() as td: + p, before, after, exact = publication_fixture(); api = FakeAPI(p, before, after, exact); lease = FakeLease() + if kind == 'expired': lease.valid = False + else: api.identity = lambda: p['spec_reviewer'] + with self.assertRaises(m.ValidationError): m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'}) + self.assertEqual(api.posts, 0) + self.assertFalse((self.canonical(m, p) / 'intent.json').exists()) + def test_fresh_head_contract_and_source_linkage_drift_prevent_post(self): + m = self.module() + changes = [lambda s: s['pr']['head'].__setitem__('sha', 'b' * 40), + lambda s: s['issue'].__setitem__('body', 'changed contract'), + lambda s: s['pr'].__setitem__('body', 'Closes #4'), + lambda s: s['pr']['base'].__setitem__('ref', 'other'), + lambda s: s['pr'].__setitem__('merged', True)] + for i, change in enumerate(changes): + with self.subTest(case=i), tempfile.TemporaryDirectory() as td: + p, before, after, exact = publication_fixture(); api = FakeAPI(p, copy.deepcopy(before), after, exact) + change(api.before) + with self.assertRaises(m.ValidationError): m.publish(p, before, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'}) + self.assertEqual(api.posts, 0) + def test_pagination_total_duplicate_and_truncation_are_hard_errors(self): + m = self.module() + changes = [lambda s: s['pagination']['files'][0].__setitem__('declared_total', 2), + lambda s: s['pagination']['reviews'][0].__setitem__('link', '; rel="next"'), + lambda s: s['pagination']['commits'][0].__setitem__('count', True), + lambda s: s['files'].append(copy.deepcopy(s['files'][0])), + lambda s: s['review_comments'].pop('8')] + for i, change in enumerate(changes): + with self.subTest(case=i): + p, before, after, exact = publication_fixture(); change(before) + with self.assertRaises(m.ValidationError): m.validate_snapshot(p, before) + def test_spec_approval_body_head_role_and_state_are_bound(self): + m = self.module() + changes = [lambda s: s['reviews'][1].__setitem__('state', 'REQUEST_CHANGES'), + lambda s: s['reviews'][1].__setitem__('commit_id', 'b' * 40), + lambda s: s['reviews'][1].__setitem__('dismissed', True), + lambda s: s['reviews'][1].__setitem__('body', s['reviews'][1]['body'].replace('충족', '임의 변경', 1)), + lambda s: s['reviews'][1]['user'].__setitem__('id', 23)] + for i, change in enumerate(changes): + with self.subTest(case=i): + p, before, after, exact = publication_fixture(); change(before) + with self.assertRaises(m.ValidationError): m.validate_snapshot(p, before) + def test_normal_supersession_is_allowed_without_projection_opt_in(self): + for already_dismissed in (False, True): + with self.subTest(already_dismissed=already_dismissed): + m = self.module() + p, before, after, exact = publication_fixture() + p['provider_contract'].update(supersede_same_author=False, + retouch_dismissed=False, + unordered_commit_files=False, evidence=[]) + if already_dismissed: + before['reviews'][0].update(official=False, dismissed=True) + after['reviews'][0].update(official=False, dismissed=True, + updated_at=exact['submitted_at']) + saved_plan = copy.deepcopy(p) + saved_before = copy.deepcopy(before) + try: + result = m.verify_publication(p, before, after, exact) + except m.ValidationError as exc: + self.fail(f'normal same-author replacement was rejected: {exc}') + self.assertTrue(result['verified']) + self.assertEqual(result['projected_review_ids'], [8]) + self.assertEqual(p, saved_plan) + self.assertEqual(before, saved_before) + def test_legacy_replacement_flags_do_not_require_external_evidence(self): + m = self.module() + p, before, after, exact = publication_fixture() + p['provider_contract'].update(unordered_commit_files=False, evidence=[]) + after['reviews'][0].update(official=False, dismissed=True, + updated_at=exact['submitted_at']) + try: + result = m.verify_publication(p, before, after, exact) + except m.ValidationError as exc: + self.fail(f'compatibility-only replacement flags required an opt-in: {exc}') + self.assertTrue(result['verified']) + self.assertEqual(m.build_payload(p)['body'], p['body']) + # Unordered file comparisons remain a separate, evidence-backed exception. + p['provider_contract']['unordered_commit_files'] = True + with self.assertRaisesRegex(m.ValidationError, 'provider projection requires evidence'): + m.validate_plan(p) + def test_normal_replacement_does_not_allow_historical_content_mutation(self): + m = self.module() + for field, value in [('body', 'altered'), ('commit_id', 'e' * 40), + ('state', 'APPROVED'), ('user', {'id': 72, 'login': 'other'}), + ('submitted_at', '2026-10-04T00:03:00Z'), + ('official', True), ('dismissed', False), + ('updated_at', '2026-10-04T00:03:00Z')]: + with self.subTest(field=field): + p, before, after, exact = publication_fixture() + p['provider_contract'].update(supersede_same_author=False, + retouch_dismissed=False, + unordered_commit_files=False, evidence=[]) + after['reviews'][0].update(official=False, dismissed=True, + updated_at=exact['submitted_at']) + after['reviews'][0][field] = value + with self.assertRaises(m.ValidationError): + m.verify_publication(p, before, after, exact) + p, before, after, exact = publication_fixture() + p['provider_contract'].update(supersede_same_author=False, retouch_dismissed=False, + unordered_commit_files=False, evidence=[]) + after['reviews'][0].update(official=False, dismissed=True, updated_at=exact['submitted_at']) + for snap in (before, after): + snap['review_comments']['8'] = [{'id': 71, 'body': 'original'}] + snap['reviews'][0]['comments_count'] = 1 + snap['pagination']['review_comments_8'][0].update(count=1, declared_total=1) + after['review_comments']['8'][0]['body'] = 'altered' + with self.assertRaisesRegex(m.ValidationError, 'historical review comments changed'): + m.verify_publication(p, before, after, exact) + def test_normal_replacement_reconciles_original_intent_without_second_post(self): + m = self.module() + p, before, after, exact = publication_fixture() + p['provider_contract'].update(supersede_same_author=False, retouch_dismissed=False, + unordered_commit_files=False, evidence=[]) + after['reviews'][0].update(official=False, dismissed=True, updated_at=exact['submitted_at']) + api = FakeAPI(p, before, after, exact) + original_read = api.read_review + api.read_review = lambda target, rid: (_ for _ in ()).throw(TimeoutError('unavailable')) + with self.assertRaises(m.ValidationError): + m.publish(p, before, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'}) + self.assertEqual(api.posts, 1) + journal = self.canonical(m, p) + intent = (journal / 'intent.json').read_bytes() + self.assertFalse((journal / 'verified.json').exists()) + api.read_review = original_read + result = m.publish(p, before, api, FakeLease(), journal, declaration={'mode': 'none'}) + self.assertTrue(result['verified']) + self.assertFalse(result['wrote']) + self.assertEqual(result['projected_review_ids'], [8]) + self.assertEqual(api.posts, 1) + self.assertEqual((journal / 'intent.json').read_bytes(), intent) + self.assertEqual(json.loads(intent)['fingerprint'], + __import__('hashlib').sha256(m.canonical(p)).hexdigest()) + self.assertTrue(json.loads((journal / 'verified.json').read_text())['verified']) + def test_provider_projection_requires_explicit_bound_evidence(self): + m = self.module() + for kind in ['wrong_initial_flags', 'wrong_timestamp', 'wrong_author']: + with self.subTest(kind=kind): + p, before, after, exact = publication_fixture() + after['reviews'][0].update(official=False, dismissed=True, updated_at=exact['submitted_at']) + if kind == 'wrong_initial_flags': before['reviews'][0].update(official=True, dismissed=True) + elif kind == 'wrong_timestamp': after['reviews'][0]['updated_at'] = '2026-10-04T00:02:00Z' + else: + before['reviews'][0]['user'] = {'id': 72, 'login': 'other'} + after['reviews'][0]['user'] = {'id': 72, 'login': 'other'} + with self.assertRaises(m.ValidationError): m.verify_publication(p, before, after, exact) + def test_order_normalization_does_not_mask_file_contents_or_parent_order(self): + m = self.module() + p, before, after, exact = publication_fixture() + after['commits'][0]['files'].reverse() + self.assertTrue(m.verify_publication(p, before, after, exact)['commit_file_order_normalized']) + p['provider_contract']['unordered_commit_files'] = False + with self.assertRaises(m.ValidationError): m.verify_publication(p, before, after, exact) + p, before, after, exact = publication_fixture() + after['commits'][0]['files'][0]['status'] = 'removed' + with self.assertRaises(m.ValidationError): m.verify_publication(p, before, after, exact) + p, before, after, exact = publication_fixture() + before['commits'][0]['parents'] = [{'sha': 'e' * 40}, {'sha': 'f' * 40}] + after['commits'][0]['parents'] = list(reversed(before['commits'][0]['parents'])) + with self.assertRaises(m.ValidationError): m.verify_publication(p, before, after, exact) + def test_historical_head_state_body_and_comments_remain_immutable(self): + m = self.module() + changes = [lambda s: s['reviews'][0].__setitem__('body', 'altered'), + lambda s: s['reviews'][0].__setitem__('commit_id', 'e' * 40), + lambda s: s['reviews'][0].__setitem__('state', 'APPROVED')] + for i, change in enumerate(changes): + with self.subTest(case=i): + p, before, after, exact = publication_fixture(); change(after) + with self.assertRaises(m.ValidationError): m.verify_publication(p, before, after, exact) + p, before, after, exact = publication_fixture() + for snap in [before, after]: + snap['review_comments']['8'] = [{'id': 71, 'body': 'original'}] + snap['reviews'][0]['comments_count'] = 1 + snap['pagination']['review_comments_8'][0].update(count=1, declared_total=1) + after['review_comments']['8'][0]['body'] = 'altered' + with self.assertRaises(m.ValidationError): m.verify_publication(p, before, after, exact) + def test_existing_canonical_review_is_write_free_no_op(self): + m = self.module(); p, before, after, exact = publication_fixture() + api = FakeAPI(p, before, after, exact); api.created = True + with tempfile.TemporaryDirectory() as td: + result = m.publish(p, after, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'}) + self.assertEqual(result['mode'], 'no_op'); self.assertFalse(result['wrote']) + self.assertEqual(api.posts, 0) + self.assertFalse((self.canonical(m, p) / 'intent.json').exists()) + def test_duplicate_or_conflicting_canonical_review_is_not_reposted(self): + m = self.module() + for kind in ['body', 'duplicate']: + with self.subTest(kind=kind), tempfile.TemporaryDirectory() as td: + p, before, after, exact = publication_fixture() + if kind == 'body': after['reviews'][-1]['body'] += 'other' + else: + extra = copy.deepcopy(exact); extra['id'] = 11; after['reviews'].append(extra) + after['review_comments']['11'] = [] + after['pagination']['reviews'][0].update(count=4, declared_total=4) + after['pagination']['review_comments_11'] = [{'page': 1, 'count': 0, 'declared_total': 0, 'link': None}] + api = FakeAPI(p, before, after, exact); api.created = True + with self.assertRaises(m.ValidationError): m.publish(p, before, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'}) + self.assertEqual(api.posts, 0) + def test_issue_repository_mismatch_is_not_accepted(self): + m = self.module(); p, before, after, exact = publication_fixture() + before['issue']['repository']['full_name'] = 'other/repo' + with self.assertRaises(m.ValidationError): m.validate_snapshot(p, before) + def test_fork_head_and_nonofficial_native_states_preserve_accepted_contract(self): + m = self.module(); p, before, after, exact = publication_fixture() + for snapshot in [before, after]: + snapshot['pr']['head']['repo']['full_name'] = 'contributor/repo' + snapshot['reviews'][1]['official'] = False + after['reviews'][-1]['official'] = exact['official'] = False + self.assertTrue(m.verify_publication(p, before, after, exact)['verified']) + after['pr']['head']['repo']['full_name'] = 'different/repo' + with self.assertRaises(m.ValidationError): m.verify_publication(p, before, after, exact) + + def test_alternate_path_is_rejected_before_publication(self): + m = load(SCRIPT, 'native_review_alternate_test') + fixtures = load(ROOT / 'harness/tests/_support/review.py', 'native_review_alternate_fixture_test') + plan, before, after, exact = fixtures.publication_fixture() + with tempfile.TemporaryDirectory() as td: + with self.assertRaises(m.ValidationError): + m.publish(plan, before, fixtures.FakeAPI(plan, before, after, exact), + fixtures.FakeLease(), td, declaration={'mode': 'none'}) + +if __name__ == '__main__': + unittest.main() diff --git a/harness/tests/review/native/test_native_resume.py b/harness/tests/review/native/test_native_resume.py new file mode 100644 index 0000000..41fab7b --- /dev/null +++ b/harness/tests/review/native/test_native_resume.py @@ -0,0 +1,83 @@ +import copy +import importlib.util +import json +import os +import subprocess +import sys +import tempfile +import time +import unittest +from pathlib import Path + +from harness.tests._support.paths import ROOT +SCRIPT = ROOT / 'harness/agents/review/skills/gitea-code-review/scripts/native_review.py' + + +def load(path, name): + spec = importlib.util.spec_from_file_location(name, path) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +from harness.tests._support.review import ( + FakeAPI, + FakeLease, + plan_fixture, + publication_fixture, + support, +) +class TestNativeResume(unittest.TestCase): + def module(self): + self.assertTrue(SCRIPT.is_file(), 'shared native review adapter is missing') + m = load(SCRIPT, 'native_review_test_' + str(id(self))) + # Every test gets a private registry root; production has no test-only + # root override, so isolate through the journal module seam. + private_root = tempfile.TemporaryDirectory() + self.addCleanup(private_root.cleanup) + root = Path(private_root.name) / 'native-review-journals' + m._journal.root_for = lambda anchor, root=root: root + return m + @staticmethod + def canonical(m, plan): + return m.canonical_journal_path(plan) + def test_readback_failure_and_changed_journal_plan_never_retry_post(self): + m = self.module(); p, before, after, exact = publication_fixture(); api = FakeAPI(p, before, after, exact) + api.read_review = lambda target, rid: (_ for _ in ()).throw(TimeoutError('DO_NOT_ECHO_TRANSPORT_SECRET')) + with tempfile.TemporaryDirectory() as td: + for _ in range(2): + with self.assertRaises(Exception) as raised: m.publish(p, before, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'}) + self.assertNotIn('DO_NOT_ECHO_TRANSPORT_SECRET', str(raised.exception)) + self.assertEqual(api.posts, 1) + different = copy.deepcopy(p); different['body'] += 'different' + with self.assertRaises(m.ValidationError): m.publish(different, before, api, FakeLease(), td, declaration={'mode': 'none'}) + self.assertEqual(api.posts, 1) + def test_journal_symlinks_hardlinks_and_changed_baseline_are_rejected(self): + m = self.module(); p, before, after, exact = publication_fixture() + with tempfile.TemporaryDirectory() as td: + root = Path(td); link = root / 'link'; link.symlink_to(root, target_is_directory=True) + with self.assertRaises(m.ValidationError): m.Journal(link) + api = FakeAPI(p, before, after, exact); api.timeout = True; api.apply = False + journal = self.canonical(m, p) + with self.assertRaises(m.UncertainPublication): m.publish(p, before, api, FakeLease(), journal, declaration={'mode': 'none'}) + saved = json.loads((journal / 'before.json').read_text()); saved['issue']['body'] = 'changed' + (journal / 'before.json').write_text(json.dumps(saved)) + with self.assertRaises(m.ValidationError): m.publish(p, before, api, FakeLease(), journal, declaration={'mode': 'none'}) + self.assertEqual(api.posts, 1) + with m.Journal(root / 'hardlinks') as j: + j.write('before.json', {'safe': True}); os.link(j.path / 'before.json', j.path / 'b.json') + with self.assertRaises(m.ValidationError): j.read('before.json') + def test_journal_intent_fields_are_validated_on_resume(self): + for field in ['schema', 'head', 'event', 'body_sha256']: + m = self.module() + with self.subTest(field=field), tempfile.TemporaryDirectory() as td: + p, before, after, exact = publication_fixture(); api = FakeAPI(p, before, after, exact) + api.timeout = True; api.apply = False + with self.assertRaises(m.ValidationError): m.publish(p, before, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'}) + path = self.canonical(m, p) / 'intent.json'; intent = json.loads(path.read_text()); intent[field] = 'corrupt' + path.write_text(json.dumps(intent)); api.created = True + with self.assertRaises(m.ValidationError): m.publish(p, before, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'}) + self.assertEqual(api.posts, 1) + +if __name__ == '__main__': + unittest.main() diff --git a/harness/tests/review/native/test_native_transport.py b/harness/tests/review/native/test_native_transport.py new file mode 100644 index 0000000..f54dd3c --- /dev/null +++ b/harness/tests/review/native/test_native_transport.py @@ -0,0 +1,213 @@ +import copy +import importlib.util +import json +import os +import subprocess +import sys +import tempfile +import time +import unittest +from pathlib import Path + +from harness.tests._support.paths import ROOT +SCRIPT = ROOT / 'harness/agents/review/skills/gitea-code-review/scripts/native_review.py' + + +def load(path, name): + spec = importlib.util.spec_from_file_location(name, path) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +from harness.tests._support.review import ( + FakeAPI, + FakeLease, + plan_fixture, + publication_fixture, + support, +) +class TestNativeTransport(unittest.TestCase): + def module(self): + self.assertTrue(SCRIPT.is_file(), 'shared native review adapter is missing') + m = load(SCRIPT, 'native_review_test_' + str(id(self))) + # Every test gets a private registry root; production has no test-only + # root override, so isolate through the journal module seam. + private_root = tempfile.TemporaryDirectory() + self.addCleanup(private_root.cleanup) + root = Path(private_root.name) / 'native-review-journals' + m._journal.root_for = lambda anchor, root=root: root + return m + @staticmethod + def canonical(m, plan): + return m.canonical_journal_path(plan) + def test_api_adapter_empty_comments_without_header_verifies_publication(self): + m = self.module() + p, before, after, exact = publication_fixture() + + def collect(source, comments=None, headers=None): + calls = [] + def request(path, payload=None): + self.assertIsNone(payload, 'readback regression must be GET-only') + calls.append(path) + direct = {'/user': p['reviewer'], '/users/gitea-spec-agent': p['spec_reviewer'], + '/repos/acme/repo': source['repository'], + '/repos/acme/repo/issues/3': source['issue'], + '/repos/acme/repo/pulls/7': source['pr'], + '/repos/acme/repo/pulls/7/reviews/10': exact} + if path in direct: + return direct[path], {}, 200 + from urllib.parse import urlsplit + endpoint = urlsplit(path).path + for name, suffix in [('files', '/pulls/7/files'), ('commits', '/pulls/7/commits'), + ('reviews', '/pulls/7/reviews'), ('issue_comments', '/issues/3/comments'), + ('pr_comments', '/issues/7/comments'), + ('statuses', '/statuses/' + p['target']['head'])]: + if endpoint == '/repos/acme/repo' + suffix: + return copy.deepcopy(source[name]), {'X-Total-Count': str(len(source[name]))}, 200 + if '/reviews/' in endpoint and endpoint.endswith('/comments'): + return ([] if comments is None else comments), ({} if headers is None else headers), 200 + raise AssertionError('unexpected synthetic readback endpoint') + api = m.ApiAdapter(request, origin=m.ORIGIN + '/api/v1', + credential_source='$HERMES_HOME/.env:GITEA_TOKEN') + snapshot = api.snapshot(p['target']) + self.assertEqual(calls[0], '/user') + self.assertTrue(any('/reviews/' in path and '/comments?' in path for path in calls)) + return api, snapshot + + _, initial = collect(before) + api, final = collect(after) + m.validate_snapshot(p, initial) + m.validate_snapshot(p, final) + authoritative = api.read_review(p['target'], exact['id']) + m.verify_publication(p, initial, final, authoritative) + for snapshot in [initial, final]: + for rid in snapshot['review_comments']: + self.assertEqual(snapshot['review_comments'][rid], []) + self.assertEqual(snapshot['pagination']['review_comments_' + rid], + [{'page': 1, 'count': 0, 'declared_total': None, 'link': None}]) + + # The admitted control above distinguishes intentional rejection from + # a broken fixture. Missing headers never waive the review count or + # the complete-collection and newly-created no-inline-comment checks. + for count in [1, None, True]: + with self.subTest(count=count): + drift = copy.deepcopy(final) + drift['reviews'][-1]['comments_count'] = count + with self.assertRaisesRegex(m.ValidationError, 'review comment cardinality mismatch'): + m.validate_snapshot(p, drift) + _, unexpected = collect(after, comments=[{'id': 999}]) + for review in unexpected['reviews']: + review['comments_count'] = 1 + m.validate_snapshot(p, unexpected) + exact_with_inline = copy.deepcopy(authoritative) + exact_with_inline['comments_count'] = 1 + with self.assertRaisesRegex(m.ValidationError, 'exact review binding mismatch'): + m.verify_publication(p, initial, unexpected, exact_with_inline) + with self.assertRaisesRegex(m.ValidationError, 'truncated pages'): + collect(after, headers={'X-Total-Count': '1'}) + truncated = copy.deepcopy(final) + truncated['pagination']['review_comments_' + str(exact['id'])][0]['link'] = '; rel=next' + with self.assertRaisesRegex(m.ValidationError, 'truncated pagination'): + m.validate_snapshot(p, truncated) + with self.assertRaisesRegex(m.ValidationError, 'truncated pagination'): + m.check_pages([{'id': i + 1} for i in range(50)], + [{'page': 1, 'count': 50, 'declared_total': None, 'link': None}]) + def test_api_adapter_checks_identity_first_and_collects_all_pages(self): + m = self.module() + self.assertTrue(callable(getattr(m, 'ApiAdapter', None)), 'REST bridge is missing') + p, before, after, exact = publication_fixture() + calls = [] + def request(path, payload=None): + calls.append((path, payload)) + if path == '/user': return p['reviewer'], {}, 200 + if path == '/users/gitea-spec-agent': return p['spec_reviewer'], {}, 200 + if path == '/repos/acme/repo': return before['repository'], {}, 200 + if path == '/repos/acme/repo/issues/3': return before['issue'], {}, 200 + if path == '/repos/acme/repo/pulls/7': return before['pr'], {}, 200 + from urllib.parse import parse_qs, urlsplit + parsed = urlsplit(path); page = int(parse_qs(parsed.query).get('page', ['1'])[0]) + for name, suffix in [('files', '/pulls/7/files'), ('commits', '/pulls/7/commits'), + ('reviews', '/pulls/7/reviews'), ('issue_comments', '/issues/3/comments'), + ('pr_comments', '/issues/7/comments'), ('statuses', '/statuses/' + p['target']['head'])]: + if parsed.path == '/repos/acme/repo' + suffix: + if name == 'reviews' and payload is not None: return exact, {}, 201 + return before[name], {'x-total-count': str(len(before[name]))}, 200 + if parsed.path.endswith('/comments'): return [], {'X-Total-Count': '0'}, 200 + raise AssertionError('unexpected request') + adapter = m.ApiAdapter(request, origin=m.ORIGIN + '/api/v1', + credential_source='$HERMES_HOME/.env:GITEA_TOKEN') + snapshot = adapter.snapshot(p['target']) + m.validate_snapshot(p, snapshot) + self.assertEqual(calls[0][0], '/user') + self.assertEqual(adapter.create_review(p['target'], m.build_payload(p))['id'], 10) + self.assertEqual(calls[-1][1]['comments'], []) + with self.assertRaises(m.ValidationError): + m.ApiAdapter(request, origin='https://other.test/api/v1', credential_source='unknown') + def test_transport_origin_credential_source_and_exception_text_are_guarded(self): + m = self.module() + with self.assertRaises(m.ValidationError): m.ApiAdapter(lambda *args: None, origin='https://other/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN') + with self.assertRaises(m.ValidationError): m.ApiAdapter(lambda *args: None, origin=m.ORIGIN + '/api/v1', credential_source='other-profile') + def bad(*args): raise TimeoutError('DO_NOT_ECHO_TRANSPORT_SECRET') + api = m.ApiAdapter(bad, origin=m.ORIGIN + '/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN') + with self.assertRaises(m.ValidationError) as raised: api.identity() + self.assertNotIn('DO_NOT_ECHO_TRANSPORT_SECRET', str(raised.exception)) + self.assertTrue(raised.exception.__suppress_context__) + def test_live_rest_bridge_over_loopback_publishes_once_then_no_ops(self): + import threading + from http.server import BaseHTTPRequestHandler, HTTPServer + from urllib.request import Request, urlopen + m = self.module(); p, before, after, exact = publication_fixture() + after['reviews'][0].update(official=False, dismissed=True, updated_at=exact['submitted_at']) + after['commits'][0]['files'].reverse() + state = {'posted': False, 'posts': 0} + class Handler(BaseHTTPRequestHandler): + def log_message(self, *args): pass + def respond(self, value, status=200, total=None): + body = json.dumps(value, ensure_ascii=False).encode() + self.send_response(status); self.send_header('Content-Type', 'application/json') + if total is not None: self.send_header('X-Total-Count', str(total)) + self.send_header('Content-Length', str(len(body))); self.end_headers(); self.wfile.write(body) + def do_GET(self): + path = self.path.split('?')[0]; s = after if state['posted'] else before + if path == '/user': return self.respond(p['reviewer']) + if path == '/users/gitea-spec-agent': return self.respond(p['spec_reviewer']) + prefix = '/repos/acme/repo'; pull = prefix + '/pulls/7' + singles = {prefix: s['repository'], prefix + '/issues/3': s['issue'], pull: s['pr']} + if path in singles: return self.respond(singles[path]) + if path == pull + '/reviews/10': return self.respond(exact) + arrays = {pull + '/files': s['files'], pull + '/commits': s['commits'], pull + '/reviews': s['reviews'], + prefix + '/issues/3/comments': s['issue_comments'], prefix + '/issues/7/comments': s['pr_comments'], + prefix + '/statuses/' + p['target']['head']: s['statuses']} + for rid, comments in s['review_comments'].items(): arrays[pull + '/reviews/' + rid + '/comments'] = comments + if path in arrays: return self.respond(arrays[path], total=len(arrays[path])) + self.respond({}, 404) + def do_POST(self): + self.server.received = json.loads(self.rfile.read(int(self.headers['Content-Length']))) + state['posts'] += 1; state['posted'] = True + self.respond(exact, 201) + server = HTTPServer(('127.0.0.1', 0), Handler) + thread = threading.Thread(target=server.serve_forever, daemon=True); thread.start() + def request(path, payload=None): + wire = None if payload is None else json.dumps(payload, ensure_ascii=False).encode() + req = Request('http://127.0.0.1:' + str(server.server_port) + path, data=wire, + headers={'Content-Type': 'application/json'}) + with urlopen(req, timeout=5) as response: + return json.loads(response.read()), dict(response.headers), response.status + try: + # Controlled local fixture transport, not a real role credential/origin assertion. + api = m.ApiAdapter(request, origin=m.ORIGIN + '/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN') + baseline = api.snapshot(p['target']) + with tempfile.TemporaryDirectory() as td: + result = m.publish(p, baseline, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'}) + self.assertEqual(result['mode'], 'created'); self.assertEqual(result['projected_review_ids'], [8]) + self.assertTrue(result['commit_file_order_normalized']) + second = m.publish(p, baseline, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'}) + self.assertEqual(second['mode'], 'no_op'); self.assertFalse(second['wrote']) + self.assertEqual(state['posts'], 1); self.assertEqual(server.received, m.build_payload(p)) + finally: + server.shutdown(); server.server_close(); thread.join(timeout=5) + self.assertFalse(thread.is_alive()) + +if __name__ == '__main__': + unittest.main() diff --git a/harness/tests/review/test_legacy_journal_contract.py b/harness/tests/review/test_legacy_journal_contract.py deleted file mode 100644 index 700ed5e..0000000 --- a/harness/tests/review/test_legacy_journal_contract.py +++ /dev/null @@ -1,126 +0,0 @@ -import hashlib -import importlib.util -import json -import stat -import tempfile -import unittest -from pathlib import Path - -ROOT = Path(__file__).resolve().parents[3] -SCRIPT = ROOT / 'harness/agents/review/skills/gitea-code-review/scripts/native_review.py' -FIXTURE = ROOT / 'harness/tests/review/test_native_review.py' - - -def load(path, name): - spec = importlib.util.spec_from_file_location(name, path) - module = importlib.util.module_from_spec(spec) - spec.loader.exec_module(module) - return module - - -class LegacyJournalContractTests(unittest.TestCase): - def test_legacy_intent_extra_fields_are_not_registered(self): - m = load(SCRIPT, 'legacy_contract_publisher') - f = load(FIXTURE, 'legacy_contract_fixture') - plan, before, after, exact = f.publication_fixture() - with tempfile.TemporaryDirectory(dir='/opt/data/cache/scratch') as td: - m._journal.root_for = lambda anchor: Path(td) / 'native-review-journals' - source = Path(td) / 'legacy' - payload = m.build_payload(plan) - intent = {'schema': 'gitea-agent.native-review-intent/v1', - 'fingerprint': hashlib.sha256(m.canonical(plan)).hexdigest(), - 'before_sha256': hashlib.sha256(m.canonical(before)).hexdigest(), - 'head': plan['target']['head'], 'event': payload['event'], - 'body_sha256': hashlib.sha256(payload['body'].encode()).hexdigest(), - 'unexpected': 'cannot silently import this'} - with m.Journal(source) as journal: - journal.write('before.json', before) - journal.write('intent.json', intent) - digest = m.legacy_journal_digest(source) - api = f.FakeAPI(plan, before, after, exact) - with self.assertRaises(m.ValidationError): - m.register_legacy_journal(plan, source, expected_legacy_sha256=digest, - api=api, lease=f.FakeLease(), declaration={'mode': 'none'}) - self.assertEqual(api.posts, 0) - self.assertFalse((m.canonical_journal_path(plan) / 'intent.json').exists()) - - def test_legacy_inventory_rejects_unknown_artifact_before_import(self): - m = load(SCRIPT, 'legacy_inventory_contract') - f = load(FIXTURE, 'legacy_inventory_fixture') - plan, before, after, exact = f.publication_fixture() - with tempfile.TemporaryDirectory(dir='/opt/data/cache/scratch') as td: - m._journal.root_for = lambda anchor: Path(td) / 'native-review-journals' - source = Path(td) / 'legacy'; source.mkdir(mode=0o700) - with m.Journal(source) as journal: - journal.write('before.json', before) - journal.write('intent.json', m._expected_intent(plan, before)) - extra = source / 'unknown.json'; extra.write_bytes(b'{"unknown":true}') - extra.chmod(0o600) - with self.assertRaises(m.ValidationError): - m.legacy_journal_digest(source) - - def test_import_preserves_raw_bytes_and_archives_verified_separately(self): - m = load(SCRIPT, 'legacy_raw_import_contract') - f = load(FIXTURE, 'legacy_raw_import_fixture') - plan, before, after, exact = f.publication_fixture() - with tempfile.TemporaryDirectory(dir='/opt/data/cache/scratch') as td: - m._journal.root_for = lambda anchor: Path(td) / 'native-review-journals' - source = Path(td) / 'legacy'; source.mkdir(mode=0o700) - intent = m._expected_intent(plan, before) - before_raw = (json.dumps(before, sort_keys=True, indent=1) + '\n').encode() - intent_raw = (json.dumps(intent, sort_keys=True, indent=1) + '\n').encode() - verified_raw = b'{\n "verified": true,\n "legacy": "exact"\n}\n' - for name, raw in [('before.json', before_raw), ('intent.json', intent_raw), - ('verified.json', verified_raw)]: - path = source / name; path.write_bytes(raw); path.chmod(0o600) - with m.Journal(source): - pass - original = {name: (source / name).read_bytes() for name in - ['before.json', 'intent.json', 'verified.json']} - digest = m.legacy_journal_digest(source) - result = m.register_legacy_journal( - plan, source, expected_legacy_sha256=digest, - api=f.FakeAPI(plan, before, after, exact), lease=f.FakeLease(), - declaration={'mode': 'none'}) - self.assertTrue(result['read_only']) - target = m.canonical_journal_path(plan) - self.assertEqual((target / 'before.json').read_bytes(), original['before.json']) - self.assertEqual((target / 'intent.json').read_bytes(), original['intent.json']) - self.assertEqual((target / 'legacy-verified.json').read_bytes(), original['verified.json']) - self.assertFalse((target / 'verified.json').exists()) - self.assertEqual(stat.S_IMODE((target / 'legacy-verified.json').stat().st_mode), 0o600) - api = f.FakeAPI(plan, before, after, exact); api.created = True - for _ in range(2): - result = m.publish(plan, before, api, f.FakeLease(), declaration={'mode': 'none'}) - self.assertFalse(result['wrote']) - self.assertEqual(api.posts, 0) - self.assertTrue((target / 'verified.json').exists()) - self.assertEqual((target / 'legacy-verified.json').read_bytes(), verified_raw) - (target / 'legacy-verified.json').write_bytes(b'{"verified": false}') - with self.assertRaisesRegex(m.ValidationError, 'integrity mismatch'): - m.publish(plan, before, api, f.FakeLease(), declaration={'mode': 'none'}) - self.assertEqual(api.posts, 0) - - - - def test_receipt_cannot_omit_imported_verified_archive(self): - m = load(SCRIPT, 'legacy_receipt_contract'); f = load(FIXTURE, 'legacy_receipt_fixture') - plan, before, after, exact = f.publication_fixture() - with tempfile.TemporaryDirectory(dir='/opt/data/cache/scratch') as td: - m._journal.root_for = lambda anchor: Path(td) / 'native-review-journals' - source = Path(td) / 'legacy' - with m.Journal(source) as journal: - journal.write('before.json', before) - journal.write('intent.json', m._expected_intent(plan, before)) - journal.write('verified.json', {'verified': True}) - m.register_legacy_journal(plan, source, expected_legacy_sha256=m.legacy_journal_digest(source), - api=f.FakeAPI(plan, before, after, exact), lease=f.FakeLease(), declaration={'mode': 'none'}) - target = m.canonical_journal_path(plan) - receipt = json.loads((target / 'registration.json').read_text()) - receipt['imported_files'].pop() - with m.Journal(target) as journal: - journal.write('registration.json', receipt) - with self.assertRaises(m.ValidationError): m._check_registration(journal, plan) - -if __name__ == '__main__': - unittest.main() diff --git a/harness/tests/review/test_native_review.py b/harness/tests/review/test_native_review.py deleted file mode 100644 index 9794b52..0000000 --- a/harness/tests/review/test_native_review.py +++ /dev/null @@ -1,869 +0,0 @@ -import copy -import importlib.util -import json -import os -import subprocess -import sys -import tempfile -import time -import unittest -from pathlib import Path - -ROOT = Path(__file__).resolve().parents[3] -SCRIPT = ROOT / 'harness/agents/review/skills/gitea-code-review/scripts/native_review.py' -SPEC_SCRIPT = ROOT / 'harness/agents/spec/skills/gitea-implementation-spec-review/scripts/review_support.py' - - -def load(path, name): - spec = importlib.util.spec_from_file_location(name, path) - module = importlib.util.module_from_spec(spec) - spec.loader.exec_module(module) - return module - - -support = load(SPEC_SCRIPT, 'spec_support_fixture') - - -def plan_fixture(): - target = {'owner': 'acme', 'repository': 'repo', 'issue': 3, 'pull': 7, - 'head': 'a' * 40, 'base': 'c' * 40} - digest = __import__('hashlib').sha256(b'approved contract').hexdigest() - marker = ('') - return {'schema': 'gitea-agent.code-review-publication/v1', 'target': target, - 'contract_sha256': digest, 'reviewer': {'id': 42, 'login': 'gitea-review-agent'}, - 'spec_reviewer': {'id': 5, 'login': 'gitea-spec-agent'}, 'spec_review_id': 9, 'spec_paths': ['src/a.py'], - 'verdict': 'approve', 'body': marker + '\n- Verdict: `approve`\n검토 완료.\n', - 'provider_contract': {'supersede_same_author': True, 'retouch_dismissed': True, - 'unordered_commit_files': True, - 'evidence': ['saved observed Gitea projection and ordering']}} - - -def publication_fixture(): - p = plan_fixture(); t = p['target'] - spec_result = {'schema': support.SCHEMA, - 'target': {k: t[k] for k in ['owner', 'repository', 'issue', 'pull', 'head']}, - 'contract_sha256': p['contract_sha256'], 'verdict': 'approve', - 'criteria': [{'criterion': 'REQ-1', 'contract': '계약', - 'implementation_paths': ['src/a.py'], 'test_paths': [], - 'evidence': '직접 검증', 'result': 'satisfied', 'reason': '충족'}], - 'scope': {'missing': [], 'excess': [], 'spec_conflicts': []}, 'conclusion': '충족'} - url = 'https://gitea.jsfamily.quest/acme/repo/pulls/7' - old_time = '2026-10-04T00:00:00Z'; new_time = '2026-10-04T00:01:00Z' - def review(rid, user, body, head, state): - return {'id': rid, 'user': copy.deepcopy(user), 'body': body, 'commit_id': head, 'state': state, - 'official': True, 'dismissed': False, 'stale': head != t['head'], - 'submitted_at': old_time, 'updated_at': old_time, 'comments_count': 0, - 'pull_request_url': url} - sr = review(9, p['spec_reviewer'], support.render_review(spec_result, {'src/a.py'}, declaration={'mode': 'none'}), - t['head'], 'APPROVED') - previous = review(8, p['reviewer'], 'old review body', 'd' * 40, 'REQUEST_CHANGES') - s = {'identity': copy.deepcopy(p['reviewer']), 'spec_user': copy.deepcopy(p['spec_reviewer']), - 'repository': {'full_name': 'acme/repo', 'default_branch': 'main'}, - 'issue': {'number': 3, 'body': 'approved contract', 'state': 'open', - 'updated_at': old_time, 'repository': {'full_name': 'acme/repo'}}, - 'pr': {'number': 7, 'html_url': url, 'body': 'Closes #3', 'user': {'id': 99}, - 'state': 'open', 'merged': False, 'updated_at': old_time, - 'review_comments': 2, 'requested_reviewers': [], - 'base': {'sha': t['base'], 'ref': 'main', 'repo': {'full_name': 'acme/repo'}}, - 'head': {'sha': t['head'], 'ref': 'feature', 'repo': {'full_name': 'acme/repo'}}}, - 'files': [{'filename': 'src/a.py', 'status': 'modified'}], - 'commits': [{'sha': t['head'], 'files': [{'filename': 'src/a.py', 'status': 'modified'}, - {'filename': 'docs/x.md', 'status': 'added'}]}], - 'reviews': [previous, sr], 'review_comments': {'8': [], '9': []}, - 'issue_comments': [], 'pr_comments': [], 'statuses': [], 'pagination': {}} - exact = review(10, p['reviewer'], p['body'], t['head'], 'APPROVED') - exact['submitted_at'] = exact['updated_at'] = new_time - after = copy.deepcopy(s); after['reviews'].append(exact); after['review_comments']['10'] = [] - after['pr']['updated_at'] = new_time; after['pr']['review_comments'] += 1 - for snap in [s, after]: - for name in ['files', 'commits', 'reviews', 'issue_comments', 'pr_comments', 'statuses']: - snap['pagination'][name] = [{'page': 1, 'count': len(snap[name]), - 'declared_total': len(snap[name]), 'link': None}] - for rid, comments in snap['review_comments'].items(): - snap['pagination']['review_comments_' + rid] = [{'page': 1, 'count': len(comments), - 'declared_total': len(comments), 'link': None}] - return p, s, after, copy.deepcopy(exact) - - -class FakeAPI: - def __init__(self, plan, before, after, exact): - self.plan = plan; self.before = before; self.after = after; self.exact = exact - self.created = False; self.posts = 0; self.timeout = False; self.apply = True - self.events = [] - - def identity(self): - self.events.append('identity') - return copy.deepcopy(self.plan['reviewer']) - - def snapshot(self, target): - self.events.append('snapshot') - return copy.deepcopy(self.after if self.created else self.before) - - def create_review(self, target, payload): - self.events.append('create'); self.posts += 1 - if self.apply: self.created = True - if self.timeout: raise TimeoutError('secret transport detail must not escape') - return copy.deepcopy(self.exact) - - def read_review(self, target, rid): - self.events.append('exact') - return copy.deepcopy(self.exact) - - -class FakeLease: - def __init__(self): self.valid = True; self.checks = 0 - def check(self, plan): - self.checks += 1 - if not self.valid: raise support.ValidationError('expired lease') - - -class NativeReviewTests(unittest.TestCase): - def module(self): - self.assertTrue(SCRIPT.is_file(), 'shared native review adapter is missing') - m = load(SCRIPT, 'native_review_test_' + str(id(self))) - # Every test gets a private registry root; production has no test-only - # root override, so isolate through the journal module seam. - private_root = tempfile.TemporaryDirectory() - self.addCleanup(private_root.cleanup) - root = Path(private_root.name) / 'native-review-journals' - m._journal.root_for = lambda anchor, root=root: root - return m - - @staticmethod - def canonical(m, plan): - return m.canonical_journal_path(plan) - - def test_payload_is_exact_bound_native_create(self): - m = self.module() - p = plan_fixture() - self.assertEqual(m.build_payload(p), {'commit_id': p['target']['head'], - 'body': p['body'], 'event': 'APPROVED', 'comments': []}) - bad = copy.deepcopy(p); bad['target']['pull'] = True - with self.assertRaises(m.ValidationError): - m.build_payload(bad) - bad = copy.deepcopy(p); bad['body'] = bad['body'].replace('a' * 40, 'd' * 40) - with self.assertRaises(m.ValidationError): - m.build_payload(bad) - def test_verify_exact_target_and_bound_provider_changes(self): - m = self.module() - self.assertTrue(callable(getattr(m, 'verify_publication', None)), 'readback verifier is missing') - p, before, after, exact = publication_fixture() - previous = after['reviews'][0] - previous.update(official=False, dismissed=True, updated_at=exact['submitted_at']) - after['commits'][0]['files'].reverse() - result = m.verify_publication(p, before, after, exact) - self.assertEqual(result['review_id'], 10) - self.assertEqual(result['projected_review_ids'], [8]) - self.assertTrue(result['commit_file_order_normalized']) - for mutation in ['head', 'body', 'unrelated_review', 'commit_file']: - bad = copy.deepcopy(after) - if mutation == 'head': bad['pr']['head']['sha'] = 'e' * 40 - elif mutation == 'body': bad['reviews'][-1]['body'] += ' altered' - elif mutation == 'unrelated_review': bad['reviews'][1]['body'] += ' altered' - else: bad['commits'][0]['files'][0]['status'] = 'removed' - with self.subTest(mutation=mutation), self.assertRaises(m.ValidationError): - m.verify_publication(p, before, bad, exact) - def test_publish_once_and_reconcile_durable_attempt_without_reposting(self): - m = self.module() - self.assertTrue(callable(getattr(m, 'publish', None)), 'shared publisher is missing') - p, before, after, exact = publication_fixture() - api = FakeAPI(p, before, after, exact); lease = FakeLease() - with tempfile.TemporaryDirectory() as td: - result = m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'}) - self.assertEqual(result['mode'], 'created') - self.assertEqual(api.posts, 1) - self.assertEqual(m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'})['mode'], 'no_op') - self.assertEqual(api.posts, 1) - self.assertNotIn('body', json.loads((self.canonical(m, p) / 'intent.json').read_text())) - for applies in [True, False]: - m = self.module(); p, before, after, exact = publication_fixture() - api = FakeAPI(p, before, after, exact); api.timeout = True; api.apply = applies - with self.subTest(applies=applies), tempfile.TemporaryDirectory() as td: - if applies: - self.assertEqual(m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'})['mode'], 'reconciled') - self.assertEqual(m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'})['mode'], 'no_op') - else: - for _ in range(2): - with self.assertRaises(m.UncertainPublication) as error: - m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'}) - self.assertNotIn('secret transport detail', str(error.exception)) - self.assertEqual(api.posts, 1) - def test_publication_request_binding_is_independent_of_workflow_labels(self): - text = (ROOT / 'harness/agents/review/skills/gitea-code-review/SKILL.md').read_text() - entry = text.split('## 진입 조건', 1)[1].split('## 리뷰 관점', 1)[0] - publication = text.split('## Native review 게시와 재검토', 1)[1].split('### 재리뷰 수렴', 1)[0] - for rule in ('현재 요청의 원문', 'session/message ID', '독립적으로 대조', - '기존 workflow artifact', 'workflow label', 'run-id suffix', - '추론하지 않는다', '권한 근거가 아니다'): - with self.subTest(entry_rule=rule): - self.assertIn(rule, entry) - for rule in ('진입 시 고정한 요청 참조', '재사용', '현재 요청', - '불명확하거나 불일치하면 게시하지 않는다', - 'label이나 동일 문구의 다른 요청으로 대체하지 않는다'): - with self.subTest(publication_rule=rule): - self.assertIn(rule, publication) - - def test_empty_review_comments_no_header_contract_is_explicit(self): - text = (ROOT / 'harness/agents/review/skills/gitea-code-review/references/native-review.md').read_text() - self.assertIn('### Empty review-comments readback without a count header', text) - section = text.split('### Empty review-comments readback without a count header', 1)[1].split('\n### ', 1)[0] - for rule in ['X-Total-Count', 'comments_count=0', 'JSON `[]`', - 'declared_total=None', 'rel=next', 'blind retry']: - self.assertIn(rule, section) - - def test_api_adapter_empty_comments_without_header_verifies_publication(self): - m = self.module() - p, before, after, exact = publication_fixture() - - def collect(source, comments=None, headers=None): - calls = [] - def request(path, payload=None): - self.assertIsNone(payload, 'readback regression must be GET-only') - calls.append(path) - direct = {'/user': p['reviewer'], '/users/gitea-spec-agent': p['spec_reviewer'], - '/repos/acme/repo': source['repository'], - '/repos/acme/repo/issues/3': source['issue'], - '/repos/acme/repo/pulls/7': source['pr'], - '/repos/acme/repo/pulls/7/reviews/10': exact} - if path in direct: - return direct[path], {}, 200 - from urllib.parse import urlsplit - endpoint = urlsplit(path).path - for name, suffix in [('files', '/pulls/7/files'), ('commits', '/pulls/7/commits'), - ('reviews', '/pulls/7/reviews'), ('issue_comments', '/issues/3/comments'), - ('pr_comments', '/issues/7/comments'), - ('statuses', '/statuses/' + p['target']['head'])]: - if endpoint == '/repos/acme/repo' + suffix: - return copy.deepcopy(source[name]), {'X-Total-Count': str(len(source[name]))}, 200 - if '/reviews/' in endpoint and endpoint.endswith('/comments'): - return ([] if comments is None else comments), ({} if headers is None else headers), 200 - raise AssertionError('unexpected synthetic readback endpoint') - api = m.ApiAdapter(request, origin=m.ORIGIN + '/api/v1', - credential_source='$HERMES_HOME/.env:GITEA_TOKEN') - snapshot = api.snapshot(p['target']) - self.assertEqual(calls[0], '/user') - self.assertTrue(any('/reviews/' in path and '/comments?' in path for path in calls)) - return api, snapshot - - _, initial = collect(before) - api, final = collect(after) - m.validate_snapshot(p, initial) - m.validate_snapshot(p, final) - authoritative = api.read_review(p['target'], exact['id']) - m.verify_publication(p, initial, final, authoritative) - for snapshot in [initial, final]: - for rid in snapshot['review_comments']: - self.assertEqual(snapshot['review_comments'][rid], []) - self.assertEqual(snapshot['pagination']['review_comments_' + rid], - [{'page': 1, 'count': 0, 'declared_total': None, 'link': None}]) - - # The admitted control above distinguishes intentional rejection from - # a broken fixture. Missing headers never waive the review count or - # the complete-collection and newly-created no-inline-comment checks. - for count in [1, None, True]: - with self.subTest(count=count): - drift = copy.deepcopy(final) - drift['reviews'][-1]['comments_count'] = count - with self.assertRaisesRegex(m.ValidationError, 'review comment cardinality mismatch'): - m.validate_snapshot(p, drift) - _, unexpected = collect(after, comments=[{'id': 999}]) - for review in unexpected['reviews']: - review['comments_count'] = 1 - m.validate_snapshot(p, unexpected) - exact_with_inline = copy.deepcopy(authoritative) - exact_with_inline['comments_count'] = 1 - with self.assertRaisesRegex(m.ValidationError, 'exact review binding mismatch'): - m.verify_publication(p, initial, unexpected, exact_with_inline) - with self.assertRaisesRegex(m.ValidationError, 'truncated pages'): - collect(after, headers={'X-Total-Count': '1'}) - truncated = copy.deepcopy(final) - truncated['pagination']['review_comments_' + str(exact['id'])][0]['link'] = '; rel=next' - with self.assertRaisesRegex(m.ValidationError, 'truncated pagination'): - m.validate_snapshot(p, truncated) - with self.assertRaisesRegex(m.ValidationError, 'truncated pagination'): - m.check_pages([{'id': i + 1} for i in range(50)], - [{'page': 1, 'count': 50, 'declared_total': None, 'link': None}]) - - def test_api_adapter_checks_identity_first_and_collects_all_pages(self): - m = self.module() - self.assertTrue(callable(getattr(m, 'ApiAdapter', None)), 'REST bridge is missing') - p, before, after, exact = publication_fixture() - calls = [] - def request(path, payload=None): - calls.append((path, payload)) - if path == '/user': return p['reviewer'], {}, 200 - if path == '/users/gitea-spec-agent': return p['spec_reviewer'], {}, 200 - if path == '/repos/acme/repo': return before['repository'], {}, 200 - if path == '/repos/acme/repo/issues/3': return before['issue'], {}, 200 - if path == '/repos/acme/repo/pulls/7': return before['pr'], {}, 200 - from urllib.parse import parse_qs, urlsplit - parsed = urlsplit(path); page = int(parse_qs(parsed.query).get('page', ['1'])[0]) - for name, suffix in [('files', '/pulls/7/files'), ('commits', '/pulls/7/commits'), - ('reviews', '/pulls/7/reviews'), ('issue_comments', '/issues/3/comments'), - ('pr_comments', '/issues/7/comments'), ('statuses', '/statuses/' + p['target']['head'])]: - if parsed.path == '/repos/acme/repo' + suffix: - if name == 'reviews' and payload is not None: return exact, {}, 201 - return before[name], {'x-total-count': str(len(before[name]))}, 200 - if parsed.path.endswith('/comments'): return [], {'X-Total-Count': '0'}, 200 - raise AssertionError('unexpected request') - adapter = m.ApiAdapter(request, origin=m.ORIGIN + '/api/v1', - credential_source='$HERMES_HOME/.env:GITEA_TOKEN') - snapshot = adapter.snapshot(p['target']) - m.validate_snapshot(p, snapshot) - self.assertEqual(calls[0][0], '/user') - self.assertEqual(adapter.create_review(p['target'], m.build_payload(p))['id'], 10) - self.assertEqual(calls[-1][1]['comments'], []) - with self.assertRaises(m.ValidationError): - m.ApiAdapter(request, origin='https://other.test/api/v1', credential_source='unknown') - def test_held_lease_checks_real_registered_worktree_and_expiry(self): - import fcntl - m = self.module() - self.assertTrue(callable(getattr(m, 'HeldIssueLease', None)), 'real Issue lease guard is missing') - with tempfile.TemporaryDirectory() as td: - root = Path(td); policy = root / 'harness/operations/repository-cache.md' - policy.parent.mkdir(parents=True); policy.write_text('canonical cache fixture') - rs = root / 'repositories'; owner = rs / 'acme'; cache = owner / 'repo'; locks = cache / 'locks' - for directory in [rs, owner, cache, locks]: directory.mkdir(mode=0o700) - bare = cache / 'repository.git'; seed = root / 'seed' - def git(*args): - out = subprocess.run(['git', '-c', 'core.hooksPath=/dev/null', *map(str, args)], capture_output=True, text=True, - env={'PATH': os.environ['PATH'], 'GIT_CONFIG_GLOBAL': '/dev/null', 'GIT_CONFIG_NOSYSTEM': '1'}) - self.assertEqual(out.returncode, 0, out.stderr) - return out.stdout.strip() - git('init', '--bare', bare); git('init', seed) - git('-C', seed, 'config', 'user.name', 'Synthetic'); git('-C', seed, 'config', 'user.email', 'fixture@example.test') - (seed / 'a.txt').write_text('synthetic') - git('-C', seed, 'add', 'a.txt'); git('-C', seed, 'commit', '-m', 'fixture') - git('--git-dir=' + str(bare), 'fetch', seed, 'HEAD:refs/heads/main') - wt = cache / 'worktrees/issue-3/checkout'; wt.parent.mkdir(parents=True, mode=0o700) - git('--git-dir=' + str(bare), 'worktree', 'add', '--detach', '--lock', wt, 'main') - head = git('-C', wt, 'rev-parse', 'HEAD') - p = plan_fixture(); p['target']['head'] = head; p['body'] = p['body'].replace('a' * 40, head) - state = wt.parent / 'state.json' - state.write_text(json.dumps({'schema': 'gitea-agent.worktree/v1', 'owner': 'acme', 'repository': 'repo', - 'issue': 3, 'created_by_role': 'gitea-impl-agent', 'mode': 'detached'})) - state.chmod(0o600) - path = locks / 'issue-3.lock'; fd = os.open(path, os.O_RDWR | os.O_CREAT, 0o600) - try: - fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB) - lease = m.HeldIssueLease(fd, policy, time.monotonic() + 30) - canonical_bytes = state.read_bytes(); canonical_stat = state.stat() - lease.check(p) - self.assertEqual(state.read_bytes(), canonical_bytes) - self.assertEqual((state.stat().st_mode, state.stat().st_mtime_ns, state.stat().st_ino), - (canonical_stat.st_mode, canonical_stat.st_mtime_ns, canonical_stat.st_ino)) - # The bounded compatibility admission is read-only: mutate only - # this scratch fixture from canonical producer state to legacy impl. - state.write_text(json.dumps({'schema': 'gitea-agent.worktree/v1', 'owner': 'acme', 'repository': 'repo', - 'issue': 3, 'created_by_role': 'impl', 'mode': 'detached'})) - state.chmod(0o600) - legacy_bytes = state.read_bytes(); legacy_stat = state.stat() - try: - lease.check(p) - except m.ValidationError as exc: - self.fail(f'valid legacy impl registration must be admitted: {exc}') - after_stat = state.stat() - self.assertEqual(state.read_bytes(), legacy_bytes) - self.assertEqual((after_stat.st_mode, after_stat.st_mtime_ns, after_stat.st_ino), - (legacy_stat.st_mode, legacy_stat.st_mtime_ns, legacy_stat.st_ino)) - for bad_role in [None, True, False, 1, [], {}, '', ' impl', 'impl ', 'Impl', 'IMPL', 'gitea-impl-agent ', 'review', 'gitea-review-agent', 'gitea-spec-agent', 'gitea-release-agent', 'other']: - state.write_text(json.dumps({'schema': 'gitea-agent.worktree/v1', 'owner': 'acme', 'repository': 'repo', - 'issue': 3, 'created_by_role': bad_role, 'mode': 'detached'})) - state.chmod(0o600) - with self.subTest(bad_role=repr(bad_role)), self.assertRaisesRegex(m.ValidationError, 'worktree registration identity mismatch'): - lease.check(p) - missing_role = json.loads(legacy_bytes); missing_role.pop('created_by_role') - state.write_text(json.dumps(missing_role)); state.chmod(0o600) - with self.assertRaisesRegex(m.ValidationError, 'worktree registration identity mismatch'): lease.check(p) - bool_issue = json.loads(legacy_bytes); bool_issue['issue'] = True - state.write_text(json.dumps(bool_issue)); state.chmod(0o600) - with self.assertRaisesRegex(m.ValidationError, 'worktree registration identity mismatch'): lease.check(p) - state.write_text(json.dumps({'schema': 'gitea-agent.worktree/v1', 'owner': 'other', 'repository': 'repo', - 'issue': 3, 'created_by_role': 'impl', 'mode': 'detached'})); state.chmod(0o600) - with self.assertRaisesRegex(m.ValidationError, 'worktree registration identity mismatch'): lease.check(p) - state.write_text(json.dumps({'schema': 'gitea-agent.worktree/v1', 'owner': 'acme', 'repository': 'other', - 'issue': 3, 'created_by_role': 'impl', 'mode': 'detached'})); state.chmod(0o600) - with self.assertRaisesRegex(m.ValidationError, 'worktree registration identity mismatch'): lease.check(p) - state.write_text(json.dumps({'schema': 'gitea-agent.worktree/v1', 'owner': 'acme', 'repository': 'repo', - 'issue': 4, 'created_by_role': 'impl', 'mode': 'detached'})); state.chmod(0o600) - with self.assertRaisesRegex(m.ValidationError, 'worktree registration identity mismatch'): lease.check(p) - state.write_text(json.dumps({'schema': 'wrong/v1', 'owner': 'acme', 'repository': 'repo', - 'issue': 3, 'created_by_role': 'impl', 'mode': 'detached'})); state.chmod(0o600) - with self.assertRaisesRegex(m.ValidationError, 'worktree registration identity mismatch'): lease.check(p) - state.write_text(json.dumps({'schema': 'gitea-agent.worktree/v1', 'owner': 'acme', 'repository': 'repo', - 'issue': 3, 'created_by_role': 'impl', 'mode': 'attached'})); state.chmod(0o600) - with self.assertRaisesRegex(m.ValidationError, 'worktree registration identity mismatch'): lease.check(p) - state.write_text(json.dumps({'schema': 'gitea-agent.worktree/v1', 'owner': 'acme', 'repository': 'repo', - 'issue': 3, 'created_by_role': 'impl', 'mode': 'detached'})); state.chmod(0o600) - wrong_head = copy.deepcopy(p); wrong_head['target']['head'] = 'b' * 40 - wrong_head['body'] = wrong_head['body'].replace(head, 'b' * 40) - with self.assertRaisesRegex(m.ValidationError, 'worktree HEAD mismatch'): lease.check(wrong_head) - fcntl.flock(fd, fcntl.LOCK_UN) - with self.assertRaises(m.ValidationError): lease.check(p) - self.assertNotIn('lock:', Path(f'/proc/self/fdinfo/{fd}').read_text()) - fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB) - from unittest.mock import patch - fakebin = root / 'fakebin'; fakebin.mkdir() - marker = root / 'path-git-executed' - fakegit = fakebin / 'git' - fakegit.write_text(f'#!/bin/sh\n/usr/bin/touch "{marker}"\nexit 1\n'); fakegit.chmod(0o700) - with patch.dict(os.environ, {'PATH': str(fakebin)}): lease.check(p) - self.assertFalse(marker.exists()) - lease.deadline_monotonic = time.monotonic() - 1 - with self.assertRaises(m.ValidationError): lease.check(p) - lease.deadline_monotonic = time.monotonic() + 30 - (wt / 'a.txt').write_text('dirty') - with self.assertRaises(m.ValidationError): lease.check(p) - # Every failed source validation must close all transient FDs. - before_fds = len(list((Path('/proc/self/fd')).iterdir())) - for _ in range(20): - with self.assertRaises(m.ValidationError): lease.check(p) - after_fds = len(list((Path('/proc/self/fd')).iterdir())) - self.assertLessEqual(after_fds, before_fds + 1) - (wt / 'a.txt').unlink() - for directory in [wt.parent.parent, wt.parent, wt]: - directory.chmod(0o770) - with self.subTest(directory=directory), self.assertRaises(m.ValidationError): - lease.check(p) - directory.chmod(0o700) - finally: - os.close(fd) - def test_cli_preflight_and_saved_wire_verification_are_read_only(self): - m = self.module(); p, before, after, exact = publication_fixture() - with tempfile.TemporaryDirectory() as td: - root = Path(td) - for name, value in [('plan', p), ('before', before), ('after', after), ('exact', exact)]: - (root / (name + '.json')).write_text(json.dumps(value, ensure_ascii=False)) - base = [sys.executable, str(SCRIPT)] - env = {'PATH': os.environ['PATH'], 'PYTHONDONTWRITEBYTECODE': '1'} - pre = subprocess.run(base + ['preflight', '--plan', str(root / 'plan.json'), - '--baseline', str(root / 'before.json')], capture_output=True, text=True, env=env) - self.assertEqual(pre.returncode, 0, pre.stderr) - self.assertIn('"ready": true', pre.stdout, 'read-only preflight CLI is missing') - verified = subprocess.run(base + ['verify', '--plan', str(root / 'plan.json'), - '--before', str(root / 'before.json'), '--after', str(root / 'after.json'), - '--exact', str(root / 'exact.json')], capture_output=True, text=True, env=env) - self.assertEqual(verified.returncode, 0, verified.stderr) - self.assertTrue(json.loads(verified.stdout)['read_only']) - self.assertEqual(json.loads(verified.stdout)['review_id'], 10) - unsupported = subprocess.run(base + ['publish'], capture_output=True, text=True, env=env) - self.assertNotEqual(unsupported.returncode, 0) - def test_spec_approval_may_reference_unchanged_surrounding_test_paths(self): - m = self.module(); p, before, after, exact = publication_fixture() - p['spec_paths'] = ['src/a.py', 'tests/existing.py'] - spec_result = support.parse_review(before['reviews'][1]['body'], {'src/a.py'}) - spec_result['criteria'][0]['test_paths'] = ['tests/existing.py'] - before['reviews'][1]['body'] = support.render_review(spec_result, set(p['spec_paths']), declaration={'mode': 'none'}) - try: - m.validate_snapshot(p, before) - except m.ValidationError: - self.fail('spec approval must use its declared scope, not only changed files') - def test_malformed_historical_actor_and_missing_exact_review_flags_fail_closed(self): - m = self.module(); p, before, after, exact = publication_fixture() - bad = copy.deepcopy(before); bad['reviews'][0]['user'] = {'id': True, 'login': 'x'} - with self.assertRaises(m.ValidationError): - m.validate_snapshot(p, bad) - p, before, after, exact = publication_fixture() - del exact['official']; del after['reviews'][-1]['official'] - with self.assertRaises(m.ValidationError): - m.verify_publication(p, before, after, exact) - def test_invalid_plan_bindings_are_not_normalized(self): - m = self.module() - mutations = [lambda p: p['target'].__setitem__('issue', True), - lambda p: p['target'].__setitem__('head', 'A' * 40), - lambda p: p['target'].__setitem__('owner', 'acme/other'), - lambda p: p.__setitem__('verdict', 'blocked'), - lambda p: p.__setitem__('body', p['body'] + '- Verdict: `approve`\n'), - lambda p: p['provider_contract'].__setitem__('evidence', []), - lambda p: p.__setitem__('spec_paths', ['../secret']), - lambda p: p.__setitem__('spec_paths', ['src/a.py', 'src/a.py']), - lambda p: p['reviewer'].__setitem__('id', p['spec_reviewer']['id'])] - for i, mutate in enumerate(mutations): - with self.subTest(case=i): - p = plan_fixture(); mutate(p) - with self.assertRaises(m.ValidationError): m.build_payload(p) - - def test_wrong_role_and_expired_lease_prevent_any_post(self): - m = self.module() - for kind in ['wrong_role', 'expired']: - with self.subTest(kind=kind), tempfile.TemporaryDirectory() as td: - p, before, after, exact = publication_fixture(); api = FakeAPI(p, before, after, exact); lease = FakeLease() - if kind == 'expired': lease.valid = False - else: api.identity = lambda: p['spec_reviewer'] - with self.assertRaises(m.ValidationError): m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'}) - self.assertEqual(api.posts, 0) - self.assertFalse((self.canonical(m, p) / 'intent.json').exists()) - - def test_fresh_head_contract_and_source_linkage_drift_prevent_post(self): - m = self.module() - changes = [lambda s: s['pr']['head'].__setitem__('sha', 'b' * 40), - lambda s: s['issue'].__setitem__('body', 'changed contract'), - lambda s: s['pr'].__setitem__('body', 'Closes #4'), - lambda s: s['pr']['base'].__setitem__('ref', 'other'), - lambda s: s['pr'].__setitem__('merged', True)] - for i, change in enumerate(changes): - with self.subTest(case=i), tempfile.TemporaryDirectory() as td: - p, before, after, exact = publication_fixture(); api = FakeAPI(p, copy.deepcopy(before), after, exact) - change(api.before) - with self.assertRaises(m.ValidationError): m.publish(p, before, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'}) - self.assertEqual(api.posts, 0) - - def test_pagination_total_duplicate_and_truncation_are_hard_errors(self): - m = self.module() - changes = [lambda s: s['pagination']['files'][0].__setitem__('declared_total', 2), - lambda s: s['pagination']['reviews'][0].__setitem__('link', '; rel="next"'), - lambda s: s['pagination']['commits'][0].__setitem__('count', True), - lambda s: s['files'].append(copy.deepcopy(s['files'][0])), - lambda s: s['review_comments'].pop('8')] - for i, change in enumerate(changes): - with self.subTest(case=i): - p, before, after, exact = publication_fixture(); change(before) - with self.assertRaises(m.ValidationError): m.validate_snapshot(p, before) - - def test_spec_approval_body_head_role_and_state_are_bound(self): - m = self.module() - changes = [lambda s: s['reviews'][1].__setitem__('state', 'REQUEST_CHANGES'), - lambda s: s['reviews'][1].__setitem__('commit_id', 'b' * 40), - lambda s: s['reviews'][1].__setitem__('dismissed', True), - lambda s: s['reviews'][1].__setitem__('body', s['reviews'][1]['body'].replace('충족', '임의 변경', 1)), - lambda s: s['reviews'][1]['user'].__setitem__('id', 23)] - for i, change in enumerate(changes): - with self.subTest(case=i): - p, before, after, exact = publication_fixture(); change(before) - with self.assertRaises(m.ValidationError): m.validate_snapshot(p, before) - - def test_normal_supersession_is_allowed_without_projection_opt_in(self): - for already_dismissed in (False, True): - with self.subTest(already_dismissed=already_dismissed): - m = self.module() - p, before, after, exact = publication_fixture() - p['provider_contract'].update(supersede_same_author=False, - retouch_dismissed=False, - unordered_commit_files=False, evidence=[]) - if already_dismissed: - before['reviews'][0].update(official=False, dismissed=True) - after['reviews'][0].update(official=False, dismissed=True, - updated_at=exact['submitted_at']) - saved_plan = copy.deepcopy(p) - saved_before = copy.deepcopy(before) - try: - result = m.verify_publication(p, before, after, exact) - except m.ValidationError as exc: - self.fail(f'normal same-author replacement was rejected: {exc}') - self.assertTrue(result['verified']) - self.assertEqual(result['projected_review_ids'], [8]) - self.assertEqual(p, saved_plan) - self.assertEqual(before, saved_before) - - def test_legacy_replacement_flags_do_not_require_external_evidence(self): - m = self.module() - p, before, after, exact = publication_fixture() - p['provider_contract'].update(unordered_commit_files=False, evidence=[]) - after['reviews'][0].update(official=False, dismissed=True, - updated_at=exact['submitted_at']) - try: - result = m.verify_publication(p, before, after, exact) - except m.ValidationError as exc: - self.fail(f'compatibility-only replacement flags required an opt-in: {exc}') - self.assertTrue(result['verified']) - self.assertEqual(m.build_payload(p)['body'], p['body']) - # Unordered file comparisons remain a separate, evidence-backed exception. - p['provider_contract']['unordered_commit_files'] = True - with self.assertRaisesRegex(m.ValidationError, 'provider projection requires evidence'): - m.validate_plan(p) - - def test_normal_replacement_does_not_allow_historical_content_mutation(self): - m = self.module() - for field, value in [('body', 'altered'), ('commit_id', 'e' * 40), - ('state', 'APPROVED'), ('user', {'id': 72, 'login': 'other'}), - ('submitted_at', '2026-10-04T00:03:00Z'), - ('official', True), ('dismissed', False), - ('updated_at', '2026-10-04T00:03:00Z')]: - with self.subTest(field=field): - p, before, after, exact = publication_fixture() - p['provider_contract'].update(supersede_same_author=False, - retouch_dismissed=False, - unordered_commit_files=False, evidence=[]) - after['reviews'][0].update(official=False, dismissed=True, - updated_at=exact['submitted_at']) - after['reviews'][0][field] = value - with self.assertRaises(m.ValidationError): - m.verify_publication(p, before, after, exact) - p, before, after, exact = publication_fixture() - p['provider_contract'].update(supersede_same_author=False, retouch_dismissed=False, - unordered_commit_files=False, evidence=[]) - after['reviews'][0].update(official=False, dismissed=True, updated_at=exact['submitted_at']) - for snap in (before, after): - snap['review_comments']['8'] = [{'id': 71, 'body': 'original'}] - snap['reviews'][0]['comments_count'] = 1 - snap['pagination']['review_comments_8'][0].update(count=1, declared_total=1) - after['review_comments']['8'][0]['body'] = 'altered' - with self.assertRaisesRegex(m.ValidationError, 'historical review comments changed'): - m.verify_publication(p, before, after, exact) - - def test_normal_replacement_reconciles_original_intent_without_second_post(self): - m = self.module() - p, before, after, exact = publication_fixture() - p['provider_contract'].update(supersede_same_author=False, retouch_dismissed=False, - unordered_commit_files=False, evidence=[]) - after['reviews'][0].update(official=False, dismissed=True, updated_at=exact['submitted_at']) - api = FakeAPI(p, before, after, exact) - original_read = api.read_review - api.read_review = lambda target, rid: (_ for _ in ()).throw(TimeoutError('unavailable')) - with self.assertRaises(m.ValidationError): - m.publish(p, before, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'}) - self.assertEqual(api.posts, 1) - journal = self.canonical(m, p) - intent = (journal / 'intent.json').read_bytes() - self.assertFalse((journal / 'verified.json').exists()) - api.read_review = original_read - result = m.publish(p, before, api, FakeLease(), journal, declaration={'mode': 'none'}) - self.assertTrue(result['verified']) - self.assertFalse(result['wrote']) - self.assertEqual(result['projected_review_ids'], [8]) - self.assertEqual(api.posts, 1) - self.assertEqual((journal / 'intent.json').read_bytes(), intent) - self.assertEqual(json.loads(intent)['fingerprint'], - __import__('hashlib').sha256(m.canonical(p)).hexdigest()) - self.assertTrue(json.loads((journal / 'verified.json').read_text())['verified']) - - def test_provider_projection_requires_explicit_bound_evidence(self): - m = self.module() - for kind in ['wrong_initial_flags', 'wrong_timestamp', 'wrong_author']: - with self.subTest(kind=kind): - p, before, after, exact = publication_fixture() - after['reviews'][0].update(official=False, dismissed=True, updated_at=exact['submitted_at']) - if kind == 'wrong_initial_flags': before['reviews'][0].update(official=True, dismissed=True) - elif kind == 'wrong_timestamp': after['reviews'][0]['updated_at'] = '2026-10-04T00:02:00Z' - else: - before['reviews'][0]['user'] = {'id': 72, 'login': 'other'} - after['reviews'][0]['user'] = {'id': 72, 'login': 'other'} - with self.assertRaises(m.ValidationError): m.verify_publication(p, before, after, exact) - - def test_order_normalization_does_not_mask_file_contents_or_parent_order(self): - m = self.module() - p, before, after, exact = publication_fixture() - after['commits'][0]['files'].reverse() - self.assertTrue(m.verify_publication(p, before, after, exact)['commit_file_order_normalized']) - p['provider_contract']['unordered_commit_files'] = False - with self.assertRaises(m.ValidationError): m.verify_publication(p, before, after, exact) - p, before, after, exact = publication_fixture() - after['commits'][0]['files'][0]['status'] = 'removed' - with self.assertRaises(m.ValidationError): m.verify_publication(p, before, after, exact) - p, before, after, exact = publication_fixture() - before['commits'][0]['parents'] = [{'sha': 'e' * 40}, {'sha': 'f' * 40}] - after['commits'][0]['parents'] = list(reversed(before['commits'][0]['parents'])) - with self.assertRaises(m.ValidationError): m.verify_publication(p, before, after, exact) - - def test_historical_head_state_body_and_comments_remain_immutable(self): - m = self.module() - changes = [lambda s: s['reviews'][0].__setitem__('body', 'altered'), - lambda s: s['reviews'][0].__setitem__('commit_id', 'e' * 40), - lambda s: s['reviews'][0].__setitem__('state', 'APPROVED')] - for i, change in enumerate(changes): - with self.subTest(case=i): - p, before, after, exact = publication_fixture(); change(after) - with self.assertRaises(m.ValidationError): m.verify_publication(p, before, after, exact) - p, before, after, exact = publication_fixture() - for snap in [before, after]: - snap['review_comments']['8'] = [{'id': 71, 'body': 'original'}] - snap['reviews'][0]['comments_count'] = 1 - snap['pagination']['review_comments_8'][0].update(count=1, declared_total=1) - after['review_comments']['8'][0]['body'] = 'altered' - with self.assertRaises(m.ValidationError): m.verify_publication(p, before, after, exact) - - def test_existing_canonical_review_is_write_free_no_op(self): - m = self.module(); p, before, after, exact = publication_fixture() - api = FakeAPI(p, before, after, exact); api.created = True - with tempfile.TemporaryDirectory() as td: - result = m.publish(p, after, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'}) - self.assertEqual(result['mode'], 'no_op'); self.assertFalse(result['wrote']) - self.assertEqual(api.posts, 0) - self.assertFalse((self.canonical(m, p) / 'intent.json').exists()) - - def test_duplicate_or_conflicting_canonical_review_is_not_reposted(self): - m = self.module() - for kind in ['body', 'duplicate']: - with self.subTest(kind=kind), tempfile.TemporaryDirectory() as td: - p, before, after, exact = publication_fixture() - if kind == 'body': after['reviews'][-1]['body'] += 'other' - else: - extra = copy.deepcopy(exact); extra['id'] = 11; after['reviews'].append(extra) - after['review_comments']['11'] = [] - after['pagination']['reviews'][0].update(count=4, declared_total=4) - after['pagination']['review_comments_11'] = [{'page': 1, 'count': 0, 'declared_total': 0, 'link': None}] - api = FakeAPI(p, before, after, exact); api.created = True - with self.assertRaises(m.ValidationError): m.publish(p, before, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'}) - self.assertEqual(api.posts, 0) - - def test_readback_failure_and_changed_journal_plan_never_retry_post(self): - m = self.module(); p, before, after, exact = publication_fixture(); api = FakeAPI(p, before, after, exact) - api.read_review = lambda target, rid: (_ for _ in ()).throw(TimeoutError('DO_NOT_ECHO_TRANSPORT_SECRET')) - with tempfile.TemporaryDirectory() as td: - for _ in range(2): - with self.assertRaises(Exception) as raised: m.publish(p, before, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'}) - self.assertNotIn('DO_NOT_ECHO_TRANSPORT_SECRET', str(raised.exception)) - self.assertEqual(api.posts, 1) - different = copy.deepcopy(p); different['body'] += 'different' - with self.assertRaises(m.ValidationError): m.publish(different, before, api, FakeLease(), td, declaration={'mode': 'none'}) - self.assertEqual(api.posts, 1) - - def test_journal_symlinks_hardlinks_and_changed_baseline_are_rejected(self): - m = self.module(); p, before, after, exact = publication_fixture() - with tempfile.TemporaryDirectory() as td: - root = Path(td); link = root / 'link'; link.symlink_to(root, target_is_directory=True) - with self.assertRaises(m.ValidationError): m.Journal(link) - api = FakeAPI(p, before, after, exact); api.timeout = True; api.apply = False - journal = self.canonical(m, p) - with self.assertRaises(m.UncertainPublication): m.publish(p, before, api, FakeLease(), journal, declaration={'mode': 'none'}) - saved = json.loads((journal / 'before.json').read_text()); saved['issue']['body'] = 'changed' - (journal / 'before.json').write_text(json.dumps(saved)) - with self.assertRaises(m.ValidationError): m.publish(p, before, api, FakeLease(), journal, declaration={'mode': 'none'}) - self.assertEqual(api.posts, 1) - with m.Journal(root / 'hardlinks') as j: - j.write('before.json', {'safe': True}); os.link(j.path / 'before.json', j.path / 'b.json') - with self.assertRaises(m.ValidationError): j.read('before.json') - - def test_transport_origin_credential_source_and_exception_text_are_guarded(self): - m = self.module() - with self.assertRaises(m.ValidationError): m.ApiAdapter(lambda *args: None, origin='https://other/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN') - with self.assertRaises(m.ValidationError): m.ApiAdapter(lambda *args: None, origin=m.ORIGIN + '/api/v1', credential_source='other-profile') - def bad(*args): raise TimeoutError('DO_NOT_ECHO_TRANSPORT_SECRET') - api = m.ApiAdapter(bad, origin=m.ORIGIN + '/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN') - with self.assertRaises(m.ValidationError) as raised: api.identity() - self.assertNotIn('DO_NOT_ECHO_TRANSPORT_SECRET', str(raised.exception)) - self.assertTrue(raised.exception.__suppress_context__) - def test_false_lease_result_does_not_authorize_publication(self): - m = self.module(); p, before, after, exact = publication_fixture(); api = FakeAPI(p, before, after, exact) - lease = FakeLease(); lease.check = lambda plan: False - with tempfile.TemporaryDirectory() as td: - with self.assertRaises(m.ValidationError): m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'}) - self.assertEqual(api.posts, 0) - def test_live_rest_bridge_over_loopback_publishes_once_then_no_ops(self): - import threading - from http.server import BaseHTTPRequestHandler, HTTPServer - from urllib.request import Request, urlopen - m = self.module(); p, before, after, exact = publication_fixture() - after['reviews'][0].update(official=False, dismissed=True, updated_at=exact['submitted_at']) - after['commits'][0]['files'].reverse() - state = {'posted': False, 'posts': 0} - class Handler(BaseHTTPRequestHandler): - def log_message(self, *args): pass - def respond(self, value, status=200, total=None): - body = json.dumps(value, ensure_ascii=False).encode() - self.send_response(status); self.send_header('Content-Type', 'application/json') - if total is not None: self.send_header('X-Total-Count', str(total)) - self.send_header('Content-Length', str(len(body))); self.end_headers(); self.wfile.write(body) - def do_GET(self): - path = self.path.split('?')[0]; s = after if state['posted'] else before - if path == '/user': return self.respond(p['reviewer']) - if path == '/users/gitea-spec-agent': return self.respond(p['spec_reviewer']) - prefix = '/repos/acme/repo'; pull = prefix + '/pulls/7' - singles = {prefix: s['repository'], prefix + '/issues/3': s['issue'], pull: s['pr']} - if path in singles: return self.respond(singles[path]) - if path == pull + '/reviews/10': return self.respond(exact) - arrays = {pull + '/files': s['files'], pull + '/commits': s['commits'], pull + '/reviews': s['reviews'], - prefix + '/issues/3/comments': s['issue_comments'], prefix + '/issues/7/comments': s['pr_comments'], - prefix + '/statuses/' + p['target']['head']: s['statuses']} - for rid, comments in s['review_comments'].items(): arrays[pull + '/reviews/' + rid + '/comments'] = comments - if path in arrays: return self.respond(arrays[path], total=len(arrays[path])) - self.respond({}, 404) - def do_POST(self): - self.server.received = json.loads(self.rfile.read(int(self.headers['Content-Length']))) - state['posts'] += 1; state['posted'] = True - self.respond(exact, 201) - server = HTTPServer(('127.0.0.1', 0), Handler) - thread = threading.Thread(target=server.serve_forever, daemon=True); thread.start() - def request(path, payload=None): - wire = None if payload is None else json.dumps(payload, ensure_ascii=False).encode() - req = Request('http://127.0.0.1:' + str(server.server_port) + path, data=wire, - headers={'Content-Type': 'application/json'}) - with urlopen(req, timeout=5) as response: - return json.loads(response.read()), dict(response.headers), response.status - try: - # Controlled local fixture transport, not a real role credential/origin assertion. - api = m.ApiAdapter(request, origin=m.ORIGIN + '/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN') - baseline = api.snapshot(p['target']) - with tempfile.TemporaryDirectory() as td: - result = m.publish(p, baseline, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'}) - self.assertEqual(result['mode'], 'created'); self.assertEqual(result['projected_review_ids'], [8]) - self.assertTrue(result['commit_file_order_normalized']) - second = m.publish(p, baseline, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'}) - self.assertEqual(second['mode'], 'no_op'); self.assertFalse(second['wrote']) - self.assertEqual(state['posts'], 1); self.assertEqual(server.received, m.build_payload(p)) - finally: - server.shutdown(); server.server_close(); thread.join(timeout=5) - self.assertFalse(thread.is_alive()) - def test_expired_lease_does_not_create_new_journal_directory(self): - m = self.module(); p, before, after, exact = publication_fixture() - lease = FakeLease(); lease.valid = False - with tempfile.TemporaryDirectory() as td: - directory = Path(td) / 'not-created' - with self.assertRaises(m.ValidationError): m.publish(p, before, FakeAPI(p, before, after, exact), lease, directory) - self.assertFalse(directory.exists()) - - def test_broker_issue_lease_accepts_authenticated_matching_status(self): - m = self.module(); p, before, after, exact = publication_fixture() - now = time.monotonic() - lease = m.BrokerIssueLease('/trusted/handle.json', '/trusted/policy.md', now + 30) - handle = {'owner': 'acme', 'repository': 'repo', 'issue': 3, 'role': 'gitea-review-agent', - 'pid': 1234, 'deadline': now + 20, 'token': 'not-returned', 'socket': '/trusted/broker.sock'} - status = {'status': 'held', 'owner': 'acme', 'repository': 'repo', 'issue': 3, - 'role': 'gitea-review-agent', 'pid': 1234, 'deadline': handle['deadline']} - from unittest.mock import patch - with patch.object(m._broker, 'read_handle_metadata', return_value={k: v for k, v in handle.items() if k != 'token'}), \ - patch.object(m._broker, 'request_handle', return_value=status), \ - patch.object(m, '_validate_registered_source') as source: - self.assertIsNone(lease.check(p)) - source.assert_called_once_with(p, '/trusted/policy.md', lease.deadline_monotonic) - - def test_broker_issue_lease_rejects_wrong_identity_status_or_deadline(self): - m = self.module(); p, before, after, exact = publication_fixture(); now = time.monotonic() - lease = m.BrokerIssueLease('/trusted/handle.json', '/trusted/policy.md', now + 30) - handle = {'pid': 1234, 'deadline': now + 20, 'token': 'hidden', 'socket': '/trusted/broker.sock'} - cases = [ - {'status': 'held', 'owner': 'acme', 'repository': 'repo', 'issue': 3, 'role': 'gitea-spec-agent', 'pid': 1234, 'deadline': handle['deadline']}, - {'status': 'held', 'owner': 'other', 'repository': 'repo', 'issue': 3, 'role': 'gitea-review-agent', 'pid': 1234, 'deadline': handle['deadline']}, - {'status': 'released', 'owner': 'acme', 'repository': 'repo', 'issue': 3, 'role': 'gitea-review-agent', 'pid': 1234, 'deadline': handle['deadline']}, - {'status': 'held', 'owner': 'acme', 'repository': 'repo', 'issue': 3, 'role': 'gitea-review-agent', 'pid': 9999, 'deadline': handle['deadline']}, - {'status': 'held', 'owner': 'acme', 'repository': 'repo', 'issue': 3, 'role': 'gitea-review-agent', 'pid': 1234, 'deadline': now + 40}, - ] - from unittest.mock import patch - for status in cases: - with self.subTest(status=status): - with patch.object(m._broker, 'read_handle_metadata', return_value={k: v for k, v in handle.items() if k != 'token'}), \ - patch.object(m._broker, 'request_handle', return_value=status), \ - patch.object(m, '_validate_registered_source') as source: - with self.assertRaises(m.ValidationError): lease.check(p) - source.assert_not_called() - - def test_journal_intent_fields_are_validated_on_resume(self): - for field in ['schema', 'head', 'event', 'body_sha256']: - m = self.module() - with self.subTest(field=field), tempfile.TemporaryDirectory() as td: - p, before, after, exact = publication_fixture(); api = FakeAPI(p, before, after, exact) - api.timeout = True; api.apply = False - with self.assertRaises(m.ValidationError): m.publish(p, before, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'}) - path = self.canonical(m, p) / 'intent.json'; intent = json.loads(path.read_text()); intent[field] = 'corrupt' - path.write_text(json.dumps(intent)); api.created = True - with self.assertRaises(m.ValidationError): m.publish(p, before, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'}) - self.assertEqual(api.posts, 1) - - def test_issue_repository_mismatch_is_not_accepted(self): - m = self.module(); p, before, after, exact = publication_fixture() - before['issue']['repository']['full_name'] = 'other/repo' - with self.assertRaises(m.ValidationError): m.validate_snapshot(p, before) - def test_fork_head_and_nonofficial_native_states_preserve_accepted_contract(self): - m = self.module(); p, before, after, exact = publication_fixture() - for snapshot in [before, after]: - snapshot['pr']['head']['repo']['full_name'] = 'contributor/repo' - snapshot['reviews'][1]['official'] = False - after['reviews'][-1]['official'] = exact['official'] = False - self.assertTrue(m.verify_publication(p, before, after, exact)['verified']) - after['pr']['head']['repo']['full_name'] = 'different/repo' - with self.assertRaises(m.ValidationError): m.verify_publication(p, before, after, exact) - - -if __name__ == '__main__': - unittest.main() diff --git a/harness/tests/review/test_publication_journal.py b/harness/tests/review/test_publication_journal.py deleted file mode 100644 index 3fbcbe1..0000000 --- a/harness/tests/review/test_publication_journal.py +++ /dev/null @@ -1,89 +0,0 @@ -import copy -import importlib.util -import os -import tempfile -import unittest -from pathlib import Path - -ROOT = Path(__file__).resolve().parents[3] -SCRIPT = ROOT / 'harness/agents/review/skills/gitea-code-review/scripts/native_review.py' -FIXTURES = ROOT / 'harness/tests/review/test_native_review.py' - -def load(path, name): - spec = importlib.util.spec_from_file_location(name, path) - module = importlib.util.module_from_spec(spec) - spec.loader.exec_module(module) - return module - - -class CanonicalJournalTests(unittest.TestCase): - def test_binding_path_is_stable_and_head_scoped(self): - m = load(SCRIPT, 'native_review_canonical_test') - fixtures = load(FIXTURES, 'native_review_fixture_test') - plan = fixtures.plan_fixture() - first = m.canonical_journal_path(plan) - again = m.canonical_journal_path(copy.deepcopy(plan)) - self.assertEqual(first, again) - changed = copy.deepcopy(plan) - changed['target']['head'] = 'b' * 40 - changed['body'] = changed['body'].replace('head=' + plan['target']['head'], 'head=' + changed['target']['head']) - self.assertNotEqual(first, m.canonical_journal_path(changed)) - changed = copy.deepcopy(plan) - changed['reviewer']['id'] = 43 - self.assertNotEqual(first, m.canonical_journal_path(changed)) - self.assertEqual(first.parts[-2], 'v1') - self.assertEqual(first.parts[-3], 'native-review-journals') - - def test_missing_entry_with_presence_anchor_fails_closed(self): - m = load(SCRIPT, 'native_review_anchor_test') - fixtures = load(FIXTURES, 'native_review_anchor_fixture_test') - plan = fixtures.plan_fixture() - with tempfile.TemporaryDirectory() as td: - root = Path(td) / 'native-review-journals' - old = m._journal.root_for - m._journal.root_for = lambda anchor: root - try: - path = m.canonical_journal_path(plan) - entry = m._journal.open_binding(path, plan, anchor_file=str(SCRIPT), - origin=m.ORIGIN, validate=m.validate_plan) - self.assertTrue((entry / 'binding.json').exists()) - import shutil - shutil.rmtree(entry) - with self.assertRaises(ValueError): - m._journal.open_binding(path, plan, anchor_file=str(SCRIPT), - origin=m.ORIGIN, validate=m.validate_plan) - finally: - m._journal.root_for = old - - def test_deleted_root_with_persistent_anchor_fails_closed(self): - m = load(SCRIPT, 'native_review_root_loss_test') - fixtures = load(FIXTURES, 'native_review_root_loss_fixture_test') - plan = fixtures.plan_fixture() - with tempfile.TemporaryDirectory() as td: - root = Path(td) / 'native-review-journals' - old = m._journal.root_for - m._journal.root_for = lambda anchor: root - try: - path = m.canonical_journal_path(plan) - m._journal.open_binding(path, plan, anchor_file=str(SCRIPT), - origin=m.ORIGIN, validate=m.validate_plan) - import shutil - shutil.rmtree(root) - with self.assertRaises(ValueError): - m._journal.open_binding(path, plan, anchor_file=str(SCRIPT), - origin=m.ORIGIN, validate=m.validate_plan) - finally: - m._journal.root_for = old - - def test_alternate_path_is_rejected_before_publication(self): - m = load(SCRIPT, 'native_review_alternate_test') - fixtures = load(FIXTURES, 'native_review_alternate_fixture_test') - plan, before, after, exact = fixtures.publication_fixture() - with tempfile.TemporaryDirectory() as td: - with self.assertRaises(m.ValidationError): - m.publish(plan, before, fixtures.FakeAPI(plan, before, after, exact), - fixtures.FakeLease(), td, declaration={'mode': 'none'}) - - -if __name__ == '__main__': - unittest.main() diff --git a/harness/tests/review/test_publication_journal_processes.py b/harness/tests/review/test_publication_journal_processes.py deleted file mode 100644 index b61e359..0000000 --- a/harness/tests/review/test_publication_journal_processes.py +++ /dev/null @@ -1,277 +0,0 @@ -import copy -import hashlib -import importlib.util -import json -import os -import stat -import subprocess -import sys -import tempfile -import textwrap -import time -import unittest -from pathlib import Path - -ROOT = Path(__file__).resolve().parents[3] -SCRIPT = ROOT / 'harness/agents/review/skills/gitea-code-review/scripts/native_review.py' -FIXTURES = ROOT / 'harness/tests/review/test_native_review.py' - - -def load(path, name): - spec = importlib.util.spec_from_file_location(name, path) - module = importlib.util.module_from_spec(spec) - spec.loader.exec_module(module) - return module - - -class PublicationJournalProcessTests(unittest.TestCase): - def setUp(self): - self.m = load(SCRIPT, 'native_review_process_' + str(id(self))) - self.fixtures = load(FIXTURES, 'native_review_process_fixture_' + str(id(self))) - self.tmp = tempfile.TemporaryDirectory() - self.addCleanup(self.tmp.cleanup) - self.root = Path(self.tmp.name) / 'native-review-journals' - self.m._journal.root_for = lambda anchor, root=self.root: root - - def plan_data(self): - p, before, after, exact = self.fixtures.publication_fixture() - return p, before, after, exact - - def declaration(self): - return {'mode': 'none'} - - def run_child(self, script, *args, timeout=8): - env = {'PATH': os.environ['PATH'], 'PYTHONPATH': str(ROOT), 'PYTHONDONTWRITEBYTECODE': '1'} - return subprocess.run([sys.executable, '-c', script, *map(str, args)], env=env, - capture_output=True, text=True, timeout=timeout) - - def child_prelude(self): - return textwrap.dedent(f''' - import copy, importlib.util, json, os, sys, time - from pathlib import Path - root = Path(sys.argv[1]); payload = json.loads(Path(sys.argv[2]).read_text()) - script = Path({str(SCRIPT)!r}); fixtures_path = Path({str(FIXTURES)!r}) - def load(path, name): - spec = importlib.util.spec_from_file_location(name, path) - m = importlib.util.module_from_spec(spec); spec.loader.exec_module(m); return m - m = load(script, 'native_child_' + str(os.getpid())) - f = load(fixtures_path, 'fixture_child_' + str(os.getpid())) - m._journal.root_for = lambda anchor: root - p, before, after, exact = f.publication_fixture() - class Lease: - def check(self, plan): return None - class API: - def __init__(self): self.created = Path(sys.argv[3]).exists() - def identity(self): return copy.deepcopy(p['reviewer']) - def snapshot(self, target): return copy.deepcopy(after if self.created else before) - def read_review(self, target, rid): return copy.deepcopy(exact) - def create_review(self, target, body): - counter = Path(sys.argv[3]); n = int(counter.read_text() or '0') if counter.exists() else 0 - counter.write_text(str(n + 1)) - with counter.open('r+') as fd: - fd.flush(); os.fsync(fd.fileno()) - self.created = True - return copy.deepcopy(exact) - api = API() - ''') - - def test_pending_intent_survives_process_crash_and_blocks_second_post(self): - p, before, after, exact = self.plan_data() - plan_file = Path(self.tmp.name) / 'plan.json'; plan_file.write_text(json.dumps(p, ensure_ascii=False)) - counter = Path(self.tmp.name) / 'posts' - crash = self.child_prelude() + textwrap.dedent(''' - class CrashAPI(API): - def create_review(self, target, body): - Path(sys.argv[4]).write_text('create-entered') - os._exit(17) - m.publish(p, before, CrashAPI(), Lease(), m.canonical_journal_path(p), declaration={'mode': 'none'}) - ''') - entered = Path(self.tmp.name) / 'entered' - crashed = self.run_child(crash, self.root, plan_file, counter, entered) - self.assertEqual(crashed.returncode, 17) - self.assertEqual(entered.read_text(), 'create-entered') - resume = self.child_prelude() + textwrap.dedent(''' - try: - m.publish(p, before, API(), Lease(), m.canonical_journal_path(p), declaration={'mode': 'none'}) - except Exception as exc: - print(type(exc).__name__) - ''') - resumed = self.run_child(resume, self.root, plan_file, counter) - self.assertEqual(resumed.returncode, 0, resumed.stderr) - self.assertIn('UncertainPublication', resumed.stdout) - self.assertFalse(counter.exists(), 'resume must not issue a second POST') - - def test_process_lock_contention_fails_fast_without_post(self): - p, before, after, exact = self.plan_data() - plan_file = Path(self.tmp.name) / 'plan.json'; plan_file.write_text(json.dumps(p)) - counter = Path(self.tmp.name) / 'posts' - ready = Path(self.tmp.name) / 'ready' - holder = self.child_prelude() + textwrap.dedent(''' - entry = m._journal.open_binding(m.canonical_journal_path(p), p, anchor_file=str(script), origin=m.ORIGIN, validate=m.validate_plan) - fd = m._journal.acquire_lock(entry); Path(sys.argv[4]).write_text('ready'); time.sleep(3); os.close(fd) - ''') - proc = subprocess.Popen([sys.executable, '-c', holder, str(self.root), str(plan_file), str(counter), str(ready)], - env={'PATH': os.environ['PATH'], 'PYTHONPATH': str(ROOT)}, text=True, - stdout=subprocess.PIPE, stderr=subprocess.PIPE) - try: - deadline = time.monotonic() + 3 - while not ready.exists() and time.monotonic() < deadline: time.sleep(.02) - self.assertTrue(ready.exists()) - contender = self.child_prelude() + textwrap.dedent(''' - try: - m.publish(p, before, API(), Lease(), m.canonical_journal_path(p), declaration={'mode': 'none'}) - except Exception as exc: - print(type(exc).__name__) - ''') - started = time.monotonic(); result = self.run_child(contender, self.root, plan_file, counter, timeout=2) - self.assertLess(time.monotonic() - started, 2) - self.assertEqual(result.returncode, 0, result.stderr) - self.assertIn('ValidationError', result.stdout) - self.assertFalse(counter.exists()) - finally: - proc.kill(); proc.communicate(timeout=3) - - def test_crash_after_transport_write_reconciles_without_second_write(self): - p, _, _, _ = self.plan_data() - plan_file = Path(self.tmp.name) / 'plan.json'; plan_file.write_text(json.dumps(p)) - counter = Path(self.tmp.name) / 'posts' - crash = self.child_prelude() + textwrap.dedent(""" - class CrashAfterWrite(API): - def create_review(self, target, body): - super().create_review(target, body) - os._exit(17) - m.publish(p, before, CrashAfterWrite(), Lease(), declaration={'mode': 'none'}) - """) - result = self.run_child(crash, self.root, plan_file, counter) - self.assertEqual(result.returncode, 17, result.stderr) - self.assertEqual(counter.read_text(), '1') - resume = self.child_prelude() + textwrap.dedent(""" - result = m.publish(p, before, API(), Lease(), declaration={'mode': 'none'}) - print(result['mode']) - """) - result = self.run_child(resume, self.root, plan_file, counter) - self.assertEqual(result.returncode, 0, result.stderr) - self.assertIn('no_op', result.stdout) - self.assertEqual(counter.read_text(), '1') - - def test_two_publishing_processes_create_only_once(self): - p, _, _, _ = self.plan_data() - plan_file = Path(self.tmp.name) / 'plan.json'; plan_file.write_text(json.dumps(p)) - counter = Path(self.tmp.name) / 'posts' - entered = Path(self.tmp.name) / 'create-entered'; release = Path(self.tmp.name) / 'release' - first = self.child_prelude() + textwrap.dedent(""" - class GatedAPI(API): - def create_review(self, target, body): - result = super().create_review(target, body) - Path(sys.argv[4]).write_text('entered') - deadline = time.monotonic() + 8 - while not Path(sys.argv[5]).exists(): - if time.monotonic() >= deadline: raise TimeoutError('test watchdog') - time.sleep(.02) - return result - result = m.publish(p, before, GatedAPI(), Lease(), declaration={'mode': 'none'}) - print(result['mode']) - """) - proc = subprocess.Popen([sys.executable, '-B', '-c', first, str(self.root), str(plan_file), str(counter), str(entered), str(release)], - env={'PATH': os.environ['PATH'], 'PYTHONPATH': str(ROOT)}, - text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) - try: - deadline = time.monotonic() + 4 - while not entered.exists() and time.monotonic() < deadline: time.sleep(.02) - self.assertTrue(entered.exists(), 'first publisher must reach transport') - self.assertEqual(counter.read_text(), '1') - contender = self.child_prelude() + textwrap.dedent(""" - try: m.publish(p, before, API(), Lease(), declaration={'mode': 'none'}) - except m.ValidationError: print('blocked') - """) - result = self.run_child(contender, self.root, plan_file, counter, timeout=3) - self.assertEqual(result.returncode, 0, result.stderr) - self.assertIn('blocked', result.stdout) - self.assertEqual(counter.read_text(), '1') - release.write_text('release') - stdout, stderr = proc.communicate(timeout=3) - self.assertEqual(proc.returncode, 0, stderr) - self.assertIn('created', stdout) - self.assertEqual(counter.read_text(), '1') - finally: - if proc.poll() is None: proc.kill() - proc.communicate(timeout=3) - - def _legacy_files(self, p, before, *, verified=True): - directory = Path(tempfile.mkdtemp(prefix='legacy-', dir=self.tmp.name)) - directory.chmod(0o700) - canonical = self.m.canonical - intent = {'schema': 'gitea-agent.native-review-intent/v1', - 'fingerprint': hashlib.sha256(self.m.canonical(p)).hexdigest(), - 'before_sha256': hashlib.sha256(self.m.canonical(before)).hexdigest(), - 'head': p['target']['head'], 'event': 'APPROVED', - 'body_sha256': hashlib.sha256(p['body'].encode()).hexdigest()} - with self.m.Journal(directory) as journal: - journal.write('before.json', before) - journal.write('intent.json', intent) - if verified: - journal.write('verified.json', {'verified': True}) - return directory - - def _legacy_digest(self, directory): - helper = getattr(self.m, 'legacy_journal_digest', None) - self.assertTrue(callable(helper), 'public legacy journal digest helper is required') - return helper(directory) - - def test_valid_legacy_registration_is_read_only_and_preserves_source_bytes(self): - p, before, after, exact = self.plan_data(); source = self._legacy_files(p, before) - original = {name: (source / name).read_bytes() for name in ['before.json', 'intent.json', 'verified.json']} - modes = {name: stat.S_IMODE((source / name).stat().st_mode) for name in original} - api = self.fixtures.FakeAPI(p, before, after, exact) - result = self.m.register_legacy_journal(p, source, expected_legacy_sha256=self._legacy_digest(source), - api=api, lease=self.fixtures.FakeLease(), declaration=self.declaration()) - self.assertTrue(result['read_only']); self.assertEqual(api.posts, 0) - for name, data in original.items(): - self.assertEqual((source / name).read_bytes(), data) - self.assertEqual(stat.S_IMODE((source / name).stat().st_mode), modes[name]) - target = self.m.canonical_journal_path(p) - self.assertEqual((target / 'before.json').read_bytes(), original['before.json']) - self.assertFalse((target / 'import.pending.json').exists()) - - def test_legacy_source_mutation_digest_mismatch_and_extra_intent_fields_reject(self): - p, before, after, exact = self.plan_data(); source = self._legacy_files(p, before) - digest = self._legacy_digest(source) - mutated = json.loads((source / 'before.json').read_text()); mutated['issue']['body'] = 'changed' - (source / 'before.json').write_text(json.dumps(mutated, separators=(',', ':'))) - with self.assertRaises(self.m.ValidationError): - self.m.register_legacy_journal(p, source, expected_legacy_sha256=digest, api=self.fixtures.FakeAPI(p, before, after, exact), - lease=self.fixtures.FakeLease(), declaration=self.declaration()) - source = self._legacy_files(p, before) - intent = json.loads((source / 'intent.json').read_text()); intent['unexpected'] = True - (source / 'intent.json').write_text(json.dumps(intent, separators=(',', ':'))) - with self.assertRaises(self.m.ValidationError): - self.m.register_legacy_journal(p, source, expected_legacy_sha256=self._legacy_digest(source), api=self.fixtures.FakeAPI(p, before, after, exact), - lease=self.fixtures.FakeLease(), declaration=self.declaration()) - - def test_legacy_symlink_hardlink_and_duplicate_json_are_rejected(self): - p, before, after, exact = self.plan_data() - source = self._legacy_files(p, before) - (source / 'before.json').unlink(); (source / 'before.json').symlink_to(Path(self.tmp.name) / 'elsewhere') - with self.assertRaises(self.m.ValidationError): - self.m.register_legacy_journal(p, source, expected_legacy_sha256='0' * 64, api=self.fixtures.FakeAPI(p, before, after, exact), - lease=self.fixtures.FakeLease(), declaration=self.declaration()) - source = self._legacy_files(p, before) - os.link(source / 'before.json', source / 'duplicate.json') - with self.assertRaises(self.m.ValidationError): - self.m.register_legacy_journal(p, source, expected_legacy_sha256=self._legacy_digest(source), api=self.fixtures.FakeAPI(p, before, after, exact), - lease=self.fixtures.FakeLease(), declaration=self.declaration()) - - def test_registration_rejects_existing_canonical_binding_without_post(self): - p, before, after, exact = self.plan_data(); source = self._legacy_files(p, before) - target = self.m.canonical_journal_path(p) - target.parent.mkdir(parents=True, exist_ok=True); target.mkdir(mode=0o700) - (target / 'intent.json').write_text('{}'); (target / 'intent.json').chmod(0o600) - api = self.fixtures.FakeAPI(p, before, after, exact) - with self.assertRaises(self.m.ValidationError): - self.m.register_legacy_journal(p, source, expected_legacy_sha256=self._legacy_digest(source), api=api, - lease=self.fixtures.FakeLease(), declaration=self.declaration()) - self.assertEqual(api.posts, 0) - - -if __name__ == '__main__': - unittest.main() diff --git a/harness/tests/review/test_review_convergence_contract_docs.py b/harness/tests/review/test_review_convergence_contract_docs.py deleted file mode 100644 index 52dc621..0000000 --- a/harness/tests/review/test_review_convergence_contract_docs.py +++ /dev/null @@ -1,26 +0,0 @@ -"""Static documentation contracts, not agent behavior or automatic loop detection.""" -from pathlib import Path -import unittest - -ROOT = Path(__file__).resolve().parents[3] -SKILL = ROOT / 'harness/agents/review/skills/gitea-code-review/SKILL.md' - - -class ReviewConvergenceDocumentationTests(unittest.TestCase): - def test_documentation_contract_requires_recurrence_escalation_and_bound_evidence(self): - text = SKILL.read_text() - heading = '### 재리뷰 수렴과 에스컬레이션' - self.assertIn(heading, text) - section = text.split(heading, 1)[1].split('\n## ', 1)[0] - for rule in ('수정 matrix', 'source delta', '영향 경로', - 'head와 실행 입력', '미변경 코드', '첫 리뷰 누락', - '재차 반복', '자동 rework handoff를 중단', '소유자', - '남은 findings', 'closure plan', '확인 불가', - '자동 승인', 'blocker를 면제', '현재 head의 spec·code pass', - '새 counter', '자동 반복 탐지기', '기존 journal'): - with self.subTest(rule=rule): - self.assertIn(rule, section) - - -if __name__ == '__main__': - unittest.main() diff --git a/harness/tests/review/test_review_observability_contract_docs.py b/harness/tests/review/test_review_observability_contract_docs.py deleted file mode 100644 index 384ab21..0000000 --- a/harness/tests/review/test_review_observability_contract_docs.py +++ /dev/null @@ -1,46 +0,0 @@ -"""Documentation contracts only: no policy discovery/log append/agent execution.""" -from pathlib import Path -import unittest - -ROOT = Path(__file__).resolve().parents[3] -SKILL = ROOT / 'harness/agents/review/skills/gitea-code-review/SKILL.md' -HEADING = '## 조건부 observability 정책 확인과 종료 기록' - - -class ReviewObservabilityDocumentationTests(unittest.TestCase): - def test_documentation_contract_requires_active_role_policy_and_conditional_scope(self): - text = SKILL.read_text() - self.assertIn(HEADING, text, 'conditional observability guidance is missing') - section = text.split(HEADING, 1)[1].split('\n## ', 1)[0] - for rule in ('현재 review role', '실제 설정', '적용 범위', 'not_applicable', - '추정하지', '다른 profile', 'spec 정책', '정책 확인 실패', - '참조된 정책이 누락', '비적용으로 처리하지 않는다', - '정책 참조가 없음을 실제 설정에서 확인', - '새 logger', '전역 설정', 'schema'): - with self.subTest(rule=rule): - self.assertIn(rule, section) - self.assertNotIn('/opt/policies/', section) - startup = text.split('## 사용 시점과 정본', 1)[1].split('\n## ', 1)[0] - self.assertIn('조건부 observability 정책 확인과 종료 기록', startup) - - - def test_documentation_contract_requires_separate_logging_status_and_final_report(self): - text = SKILL.read_text() - section = text.split(HEADING, 1)[1].split('\n## ', 1)[0] - for rule in ('최종 보고 전에', 'run 기록', '개선 후보', '일일 요약', - '새로 append한', 'readback', 'credential', '0건', - '완료/실패/미확인/비적용', 'review verdict', 'publication', 'cleanup', - '테스트나 native POST를 반복하지', '기록 경로', '실패·한계', - '출처·내용이 결합', '문서 계약', '실제 append', - '정책이 요구하는 보고 항목에 한정', - '후보 수를 요구하지 않으면', '임의로 0건'): - with self.subTest(rule=rule): - self.assertIn(rule, section) - final = text.split('## 주의 및 완료 보고', 1)[1].split('\n## ', 1)[0] - self.assertIn('조건부 observability', final) - self.assertIn('세부 항목은 적용 정책이 요구한 경우에만', final) - self.assertNotIn('기록 상태·경로와 적용 정책', final) - - -if __name__ == '__main__': - unittest.main() diff --git a/harness/tests/review/test_review_probe_contract_docs.py b/harness/tests/review/test_review_probe_contract_docs.py deleted file mode 100644 index 0b3eaec..0000000 --- a/harness/tests/review/test_review_probe_contract_docs.py +++ /dev/null @@ -1,53 +0,0 @@ -"""Static documentation contracts; no dependency resolver or probe parser is executed.""" -from pathlib import Path -import unittest - -ROOT = Path(__file__).resolve().parents[3] -SKILL = ROOT / 'harness/agents/review/skills/gitea-code-review/SKILL.md' - - -class ReviewProbeDocumentationTests(unittest.TestCase): - def test_documentation_contract_requires_dependency_provenance_and_failure_preservation(self): - text = SKILL.read_text() - heading = '### Dependency provenance와 probe JSON framing' - self.assertIn(heading, text) - section = text.split(heading, 1)[1].split('\n## ', 1)[0] - for rule in ('별도로 승인된', '실제 interpreter', 'resolved dependency 버전', - 'source revision', 'lockfile', '실패한 resolution·collection', - 'baseline', '제품 회귀', '새 venv', '설치·pinning 권한', - '기존 evidence'): - with self.subTest(rule=rule): - self.assertIn(rule, section) - - def test_documentation_contract_requires_exactly_one_framed_record_and_safe_errors(self): - section = SKILL.read_text().split('### Dependency provenance와 probe JSON framing', 1)[1].split('\n## ', 1)[0] - for rule in ('producer와 consumer', '고유 marker', '정확히 하나의 완전한 JSON record', - '누락·중복·malformed·truncation', '전체 stdout', 'brace', - 'warning', 'exit status', 'TDD RED', 'timeout', '출력 한도', - 'redaction', 'generic parser'): - with self.subTest(rule=rule): - self.assertIn(rule, section) - - def test_documentation_contract_requires_bound_content_and_control_response_handling(self): - text = SKILL.read_text() - heading = '### Programmatic read control response와 script 준비' - self.assertIn(heading, text) - section = text.split(heading, 1)[1].split('\n## ', 1)[0] - for rule in ('첫 content-bearing 응답', 'exact 경로', 'source revision', - 'hash', '읽은 범위', '응답 shape', 'control response', - 'unchanged/dedup', '보존된 내용', 'evidence_gap', - 'force-read', '전역 dedup', 'stale cache'): - with self.subTest(rule=rule): - self.assertIn(rule, section) - - def test_documentation_contract_requires_compile_before_execution_and_preserves_authority(self): - section = SKILL.read_text().split('### Programmatic read control response와 script 준비', 1)[1].split('\n## ', 1)[0] - for rule in ('typed JSON', '다중 quoted', 'compile', '실행·로그 append·외부 action 전에', - '문법 검사', '안전성·인증·실행 성공', 'credential', - 'commit/push', 'review role', '권한을 확장하지 않는다', 'TDD RED'): - with self.subTest(rule=rule): - self.assertIn(rule, section) - - -if __name__ == '__main__': - unittest.main() diff --git a/harness/tests/review/test_timeout_evidence_recipe_docs.py b/harness/tests/review/test_timeout_evidence_recipe_docs.py deleted file mode 100644 index 0537f34..0000000 --- a/harness/tests/review/test_timeout_evidence_recipe_docs.py +++ /dev/null @@ -1,22 +0,0 @@ -"""Reviewer timeout recipe documentation only; not a runner/runtime test.""" -from pathlib import Path -import unittest - -SKILL = Path(__file__).resolve().parents[3] / 'harness/agents/review/skills/gitea-code-review/SKILL.md' -HEADING = '### 장시간 suite의 timeout evidence recipe' - -class TimeoutEvidenceRecipeDocumentationTests(unittest.TestCase): - def test_conditional_recipe_preserves_evidence_without_automatic_retry(self): - text = SKILL.read_text() - self.assertIn(HEADING, text) - section = text.split(HEADING, 1)[1].split('\n### ', 1)[0].split('\n## ', 1)[0] - for rule in ('장시간 suite', '실행 시작부터', '승인된 artifact 경로', - 'source revision', 'run-id', '명령', 'deadline', 'redaction', - '출력 한도', 'exit status', 'timeout', '실제 종료', 'cleanup', - '새 run-id', '덮어쓰지 않는다', '자동 재실행하지 않는다', - '유한한 budget', '관측 시간', '여유 근거', '동일 입력', - 'JUnit', '이미 지원', '완성된 JUnit', 'pytest를 강제하지', - '새 runner', 'streaming framework', '미검증', '문서 계약'): - with self.subTest(rule=rule): - self.assertIn(rule, section) - self.assertIn('lease·head·source', section) diff --git a/harness/tests/spec/contracts/__init__.py b/harness/tests/spec/contracts/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/harness/tests/spec/contracts/test_actual_collector.py b/harness/tests/spec/contracts/test_actual_collector.py new file mode 100644 index 0000000..007c52e --- /dev/null +++ b/harness/tests/spec/contracts/test_actual_collector.py @@ -0,0 +1,30 @@ +"""Actual collector invocation coverage, separate from documentation contracts.""" +from pathlib import Path +import importlib.util +import unittest + +from harness.tests._support.paths import HARNESS + + +class ActualCollectorTests(unittest.TestCase): + def test_actual_collector_calls_synthetic_callback_with_one_page_argument(self): + import importlib.util + import inspect + path = HARNESS / 'agents/spec/skills/gitea-implementation-spec-review/scripts/review_support.py' + spec = importlib.util.spec_from_file_location('readback_invocation_support', path) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + calls = [] + + def pages(page): + calls.append(page) + return {'items': [{'id': page}], 'total_count': 2, 'has_more': page < 2} + + inspect.signature(pages).bind(1) + self.assertEqual(calls, []) + self.assertEqual(module.collect_pages(pages, 'id', page_size=1), [{'id': 1}, {'id': 2}]) + self.assertEqual(calls, [1, 2]) + + if __name__ == "__main__": + unittest.main() diff --git a/harness/tests/spec/contracts/test_priority_review_contracts.py b/harness/tests/spec/contracts/test_priority_review_contracts.py new file mode 100644 index 0000000..142cf52 --- /dev/null +++ b/harness/tests/spec/contracts/test_priority_review_contracts.py @@ -0,0 +1,67 @@ +"""Documentation contracts only; runtime collector coverage is separate.""" +from pathlib import Path +import re +import unittest + +from harness.tests._support.paths import HARNESS +DISCOVERY = HARNESS / 'agents/spec/skills/gitea-discovery-review/SKILL.md' +IMPLEMENTATION = HARNESS / 'agents/spec/skills/gitea-implementation-spec-review/SKILL.md' +README = HARNESS / 'docs/discovery/README.md' + + +class PriorityReviewContracts(unittest.TestCase): + + def test_issue_body_provider_revision_documentation_contract(self): + access = (HARNESS / 'operations/gitea-access.md').read_text() + contract = access.split('### Issue body update의 provider-derived readback', 1)[-1] + for required in ('content_version', 'updated_at'): + self.assertIn(required, contract) + authoring = (HARNESS / 'agents/spec/skills/gitea-discovery-authoring/SKILL.md').read_text() + for text in (DISCOVERY.read_text(), authoring): + self.assertIn('Issue body update의 provider-derived readback', text) + self.assertIn('content_version', text) + self.assertIn('post-comment snapshot을 body update 직전 baseline', authoring) + + + def test_rework_report_documentation_contract_requires_categories_and_counting_rules(self): + text = IMPLEMENTATION.read_text() + section = text.split('### Named test 근거와 재작업 보고', 1)[1].split('\n## ', 1)[0] + for rule in ('해소:', '잔여:', '신규:', '문서 보완:', '해당 없음'): + with self.subTest(rule=rule): + self.assertIn(rule, section) + + + def test_changed_readback_helper_documentation_contract_is_conditional_and_precreate(self): + text = IMPLEMENTATION.read_text() + heading = '### 비정본·변경된 readback helper 호출 계약' + self.assertIn(heading, text, 'conditional readback invocation guidance is missing') + section = text.split(heading, 1)[1].split('\n## ', 1)[0] + for rule in ('native create 이전', 'inspect.signature', '.bind('): + with self.subTest(rule=rule): + self.assertIn(rule, section) + step = text.split('17. 호출 전에', 1)[1].split('\n18.', 1)[0] + self.assertIn('비정본·변경된 readback helper 호출 계약', step) + self.assertLess(step.index('비정본·변경된 readback helper 호출 계약'), step.index('Native review create를')) + + def test_discovery_checker_documentation_contract_preserves_bytes_and_bounded_transformations(self): + text = DISCOVERY.read_text() + heading = '### 임시·변경된 Discovery checker 준비' + self.assertIn(heading, text, 'bounded Discovery checker preparation guidance is missing') + section = text.split(heading, 1)[1].split('\n## ', 1)[0] + for rule in ('LF/CRLF', 'splitlines', 'flags='): + with self.subTest(rule=rule): + self.assertIn(rule, section) + step = text.split('11. trusted workflow directory', 1)[1].split('\n12.', 1)[0] + self.assertIn('임시·변경된 Discovery checker 준비', step) + + def test_local_links_resolve_and_procedures_reference_new_rules(self): + for path in (DISCOVERY, IMPLEMENTATION, README): + text = path.read_text() + for link in re.findall('\\]\\(([^)]+)\\)', text): + if not link.startswith(('http:', 'https:', '#')): + self.assertTrue((path.parent / link.split('#')[0]).exists(), link) + + + +if __name__ == "__main__": + unittest.main() diff --git a/harness/tests/spec/projection/__init__.py b/harness/tests/spec/projection/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/harness/tests/spec/test_requested_reviewer_projection.py b/harness/tests/spec/projection/test_requested_reviewer.py similarity index 97% rename from harness/tests/spec/test_requested_reviewer_projection.py rename to harness/tests/spec/projection/test_requested_reviewer.py index 4e4c61c..f775c60 100644 --- a/harness/tests/spec/test_requested_reviewer_projection.py +++ b/harness/tests/spec/projection/test_requested_reviewer.py @@ -5,7 +5,7 @@ import sys import tempfile from pathlib import Path import unittest -from harness.tests.spec import test_implementation_spec_review_support as fixture +from harness.tests._support import spec as fixture class RequestedReviewerProjectionTests(unittest.TestCase): @@ -168,9 +168,9 @@ class RequestedReviewerProjectionTests(unittest.TestCase): def test_document_declares_version_default_and_narrow_collection_scope(self): skill = fixture.MODPATH.parent.parent / 'SKILL.md' text = skill.read_text() - for token in ('requested_reviewers_presentation_order', '1.27.3', - '생략하면 `false`', 'Raw before/after snapshot', 'POST 재시도 금지'): - with self.subTest(token=token): self.assertIn(token, text) + for token in ('requested_reviewers_presentation_order', '1.27.3'): + with self.subTest(token=token): + self.assertIn(token, text) if __name__ == '__main__': unittest.main() diff --git a/harness/tests/spec/test_source_issue_reference_projection.py b/harness/tests/spec/projection/test_source_issue_reference.py similarity index 99% rename from harness/tests/spec/test_source_issue_reference_projection.py rename to harness/tests/spec/projection/test_source_issue_reference.py index 92ee981..2087bf9 100644 --- a/harness/tests/spec/test_source_issue_reference_projection.py +++ b/harness/tests/spec/projection/test_source_issue_reference.py @@ -4,7 +4,7 @@ import subprocess import sys import tempfile import unittest -from harness.tests.spec import test_implementation_spec_review_support as fixture +from harness.tests._support import spec as fixture class SourceIssueReferenceProjectionTests(unittest.TestCase): diff --git a/harness/tests/spec/support/__init__.py b/harness/tests/spec/support/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/harness/tests/spec/support/test_ac_checker.py b/harness/tests/spec/support/test_ac_checker.py new file mode 100644 index 0000000..1073cfc --- /dev/null +++ b/harness/tests/spec/support/test_ac_checker.py @@ -0,0 +1,134 @@ +import copy +import json +import os +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path + +from harness.tests._support.paths import ROOT +from harness.tests._support.spec import MODPATH, mod, publication_fixture, result, verify_fixture + + +class ACClassificationTests(unittest.TestCase): + TEMPLATE = ROOT / 'harness/docs/requirements/TEMPLATE.md' + + def body(self): + return ('## 7. 인수 기준\n\n### REQ-001.01-AC01 — example\n\n' + '**판정 기준**\n\nexample\n\n**검증**\n\nexample\n\n' + '**검증 분류**\n\n- 수준: unit\n- 백엔드: 없음\n- 케이스: normal\n\n' + '**상태 불변조건**\n\n없음\n\n## 8. 오류 및 충돌 계약\n\n없음\n') + + def check(self, body): + import hashlib + self.assertTrue(callable(getattr(mod, 'check_ac_classification', None)), + 'trusted-template AC classification checker missing') + return mod.check_ac_classification(body, hashlib.sha256(self.TEMPLATE.read_bytes()).hexdigest()) + + def test_valid_canonical_ac_classification(self): + out = self.check(self.body()) + self.assertEqual(out['ac_ids'], ['REQ-001.01-AC01']) + self.assertEqual(out['checked_fields'], ['수준', '백엔드', '케이스']) + self.assertTrue(out['read_only']) + + + def test_missing_empty_duplicate_and_placeholder_fields_rejected_per_ac(self): + valid = self.body() + self.check(valid) + for changed in (valid.replace('- 수준: unit\n', ''), + valid.replace('- 백엔드: 없음', '- 백엔드: '), + valid.replace('- 케이스: normal', '- 케이스: ${verification_cases}'), + valid.replace('- 수준: unit', '- 수준: unit\n- 수준: unit'), + valid.replace('**검증 분류**', '**검증**'), + valid.replace('**검증 분류**', '**검증 분류**\n\n**검증 분류**')): + with self.subTest(changed=changed): + with self.assertRaises(mod.ValidationError): + self.check(changed) + + def test_other_ac_and_fenced_examples_cannot_supply_missing_field(self): + good = self.body().split('## 8.', 1)[0] + bad = good.replace('AC01', 'AC02').replace('- 수준: unit\n', '') + self.check(good + good.replace('AC01', 'AC02').split('\n\n', 1)[1] + '## 8. 오류 및 충돌 계약\n') + for extra in ('', '```markdown\n- 수준: unit\n```\n', '~~~\n- 수준: unit\n~~~\n'): + with self.subTest(extra=extra): + with self.assertRaises(mod.ValidationError): + self.check(good + bad.split('\n\n', 1)[1].replace('**상태 불변조건**', extra + '**상태 불변조건**') + '## 8. 오류 및 충돌 계약\n') + + + def test_ambiguous_hidden_sections_and_template_drift_fail_closed(self): + valid = self.body() + self.check(valid) + import hashlib + with self.assertRaises(mod.ValidationError): + mod.check_ac_classification(valid, '0' * 64) + for bad in ('', valid + '\n```unclosed', + valid.replace('### REQ', '#### REQ'), + valid.replace('## 8. 오류 및 충돌 계약', '## 7. 인수 기준'), + valid.replace('- 수준: unit', ' - 수준: unit')): + with self.subTest(bad=bad): + with self.assertRaises(mod.ValidationError): + self.check(bad) + crlf = valid.replace('\n', '\r\n') + self.assertEqual(self.check(crlf)['document_sha256'], hashlib.sha256(crlf.encode()).hexdigest()) + + def test_cli_read_only_exact_bytes_digest_and_sanitized_failure(self): + import hashlib + with tempfile.TemporaryDirectory() as tmp: + path = Path(tmp) / 'req.md' + raw = self.body().replace('\n', '\r\n').encode() + path.write_bytes(raw) + os.chmod(path, 0o600) + before = (path.read_bytes(), path.stat().st_mode, path.stat().st_mtime_ns, path.stat().st_ino) + argv = [sys.executable, '-B', str(MODPATH), 'check-ac-classification', + '--document', str(path), '--expected-document-sha256', hashlib.sha256(raw).hexdigest(), + '--expected-template-sha256', hashlib.sha256(self.TEMPLATE.read_bytes()).hexdigest()] + actual = subprocess.run(argv, capture_output=True, text=True, timeout=10) + self.assertEqual(actual.returncode, 0, actual.stderr) + self.assertEqual(json.loads(actual.stdout)['document_sha256'], hashlib.sha256(raw).hexdigest()) + self.assertEqual(before, (path.read_bytes(), path.stat().st_mode, path.stat().st_mtime_ns, path.stat().st_ino)) + argv[-3] = '0' * 64 + bad = subprocess.run(argv, capture_output=True, text=True, timeout=10) + self.assertEqual(bad.returncode, 2) + self.assertEqual(json.loads(bad.stderr), {'error': 'invalid input'}) + self.assertEqual(bad.stdout, '') + + + def test_supported_scope_rejects_hidden_html_and_noncanonical_ac_heading(self): + valid = self.body() + self.check(valid) + for bad in ('
\n' + valid + '
', + valid.replace('### REQ-001.01-AC01 — example', '### REQ-001.01-AC01'), + valid.replace('**검증 분류**', '**검증 분류**')): + with self.subTest(bad=bad): + with self.assertRaises(mod.ValidationError): + self.check(bad) + + def test_file_admission_rejects_symlink_fifo_invalid_utf8_and_oversize(self): + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + valid = root / 'valid.md' + valid.write_text(self.body()) + self.assertEqual(mod._classification_bytes(valid), valid.read_bytes()) + alias = root / 'alias.md' + alias.symlink_to(valid) + fifo = root / 'fifo' + os.mkfifo(fifo) + invalid = root / 'invalid.md' + invalid.write_bytes(b'\xff') + large = root / 'large.md' + large.write_bytes(b'x' * 1048577) + for path in (alias, fifo, invalid, large, root): + with self.subTest(path=path.name): + with self.assertRaises(mod.ValidationError): + mod._classification_bytes(path) + + def test_documentation_contract_scopes_checker_and_binds_digests(self): + text = (MODPATH.parent.parent / 'SKILL.md').read_text() + self.assertIn('### AC 검증 분류의 read-only 구조 검사', text) + section = text.split('### AC 검증 분류의 read-only 구조 검사', 1)[1].split('\n## ', 1)[0] + for rule in ('check-ac-classification', '--expected-document-sha256', '--expected-template-sha256'): + self.assertIn(rule, section) + +if __name__ == "__main__": + unittest.main() diff --git a/harness/tests/spec/support/test_cache.py b/harness/tests/spec/support/test_cache.py new file mode 100644 index 0000000..93f53df --- /dev/null +++ b/harness/tests/spec/support/test_cache.py @@ -0,0 +1,74 @@ +import copy +import json +import os +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path + +from harness.tests._support.paths import ROOT +from harness.tests._support.spec import MODPATH, mod, publication_fixture, result, verify_fixture + + +class CacheTests(unittest.TestCase): + + def test_cache_validation_and_rejects_symlink(self): + with tempfile.TemporaryDirectory(dir="/opt/data/cache/scratch") as td: + root = Path(td); ops = root / "harness" / "operations"; ops.mkdir(parents=True) + contract = ops / "repository-cache.md"; contract.write_text("cache", encoding="utf-8") + rs = root / "repositories"; owner = rs / "acme"; repo = owner / "repo" + (repo / "locks").mkdir(parents=True) + (repo / "repository.git").mkdir() + for p in (rs, owner, repo, repo / "locks"): os.chmod(p, 0o700) + info = mod.validate_cache_paths(contract, "acme", "repo", 3) + self.assertTrue(info["repository_git"].endswith("repository.git")) + (repo / "locks" / "issue-3.lock").symlink_to(contract) + with self.assertRaises(mod.ValidationError): + mod.validate_cache_paths(contract, "acme", "repo", 3) + + def test_cache_enforces_root_boundary_uid_and_dangling_locks(self): + with tempfile.TemporaryDirectory(dir="/opt/data/cache/scratch") as td: + root = Path(td); ops = root / "harness" / "operations"; ops.mkdir(parents=True) + contract = ops / "repository-cache.md"; contract.write_text("cache", encoding="utf-8") + rs = root / "repositories"; owner = rs / "acme"; repo = owner / "repo" + (repo / "locks").mkdir(parents=True); (repo / "repository.git").mkdir() + for path in (rs, owner, repo, repo / "locks"): + path.chmod(0o700) + os.chmod(root, 0o777) + with self.assertRaises(mod.ValidationError): + mod.validate_cache_paths(contract, "acme", "repo", 3) + os.chmod(root, 0o700) + with self.assertRaises(mod.ValidationError): + mod.validate_cache_paths(contract, "acme", "repo", 3, uid=os.getuid() + 10000) + (repo / "locks" / "issue-3.lock").symlink_to(root / "missing") + with self.assertRaises(mod.ValidationError): + mod.validate_cache_paths(contract, "acme", "repo", 3) + (repo / "locks" / "issue-3.lock").unlink() + alias = root / "alias"; alias.symlink_to(ops.parent, target_is_directory=True) + with self.assertRaises(mod.ValidationError): + mod.validate_cache_paths(alias / "operations" / "repository-cache.md", "acme", "repo", 3) + + def test_cache_group_membership_includes_other_primary_accounts(self): + from types import SimpleNamespace + from unittest.mock import patch + role = SimpleNamespace(pw_name="role", pw_uid=os.getuid(), pw_gid=111) + other = SimpleNamespace(pw_name="other", pw_uid=os.getuid() + 1, pw_gid=111) + group = SimpleNamespace(gr_gid=111, gr_mem=["role"]) + with tempfile.TemporaryDirectory(dir="/opt/data/cache/scratch") as td: + root = Path(td); ops = root / "harness" / "operations"; ops.mkdir(parents=True) + contract = ops / "repository-cache.md"; contract.write_text("cache") + rs = root / "repositories"; repo = rs / "acme" / "repo" + (repo / "locks").mkdir(parents=True); (repo / "repository.git").mkdir() + for path in (rs / "acme", repo, repo / "locks"): + path.chmod(0o700) + rs.chmod(0o770) + role.pw_gid = other.pw_gid = group.gr_gid = rs.stat().st_gid + with patch.object(mod.grp, "getgrgid", return_value=group), \ + patch.object(mod.pwd, "getpwuid", return_value=role), \ + patch.object(mod.pwd, "getpwall", return_value=[role, other]): + with self.assertRaises(mod.ValidationError): + mod.validate_cache_paths(contract, "acme", "repo", 3) + + if __name__ == "__main__": + unittest.main() diff --git a/harness/tests/spec/support/test_codec.py b/harness/tests/spec/support/test_codec.py new file mode 100644 index 0000000..2cad9be --- /dev/null +++ b/harness/tests/spec/support/test_codec.py @@ -0,0 +1,223 @@ +import copy +import json +import os +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path + +from harness.tests._support.paths import ROOT +from harness.tests._support.spec import MODPATH, mod, publication_fixture, result, verify_fixture + + +class CodecTests(unittest.TestCase): + + def test_validate_and_render_roundtrip(self): + r = result() + self.assertEqual(mod.validate_result(r, {"src/a.py", "tests/test_a.py"}), r) + body = mod.render_review(r, {"src/a.py", "tests/test_a.py"}, declaration={"mode": "none"}) + parsed = mod.parse_review(body, {"src/a.py", "tests/test_a.py"}) + self.assertEqual(parsed, r) + self.assertEqual(mod.render_review(parsed, {"src/a.py", "tests/test_a.py"}, declaration={"mode": "none"}), body) + + def test_validation_rejects_bool_path_traversal_bad_verdict(self): + r = result() + r["target"]["issue"] = True + with self.assertRaises(mod.ValidationError): + mod.validate_result(r, {"src/a.py"}) + r = result(); r["criteria"][0]["implementation_paths"] = ["../x"] + with self.assertRaises(mod.ValidationError): + mod.validate_result(r, {"../x"}) + r = result("request_changes") + with self.assertRaises(mod.ValidationError): + mod.validate_result(r, {"src/a.py", "tests/test_a.py"}) + + def test_collect_pages_requires_identity_and_reads_all_pages(self): + pages = [[{"id": 1}, {"id": 2}], [{"id": 3}]] + def fetch(n): + return {"items": pages[n - 1], "total_count": 3, "has_more": n == 1} + self.assertEqual(mod.collect_pages(fetch, "id", page_size=2), pages[0] + pages[1]) + def bad(n): + return {"items": [{"id": 1}], "total_count": 2, "has_more": True} + with self.assertRaises(mod.ValidationError): + mod.collect_pages(bad, "id", max_pages=1) + + def test_verify_unchanged(self): + self.assertTrue(mod.verify_unchanged({"a": [1]}, {"a": [1]})) + with self.assertRaises(mod.ValidationError): + mod.verify_unchanged({"a": 1}, {"a": 2}) + + def test_cli_check_pages_and_unchanged(self): + with tempfile.TemporaryDirectory(dir="/opt/data/cache/scratch") as td: + td = Path(td); env = dict(os.environ, PYTHONDONTWRITEBYTECODE="1") + pages = td / "pages.json"; pages.write_text(json.dumps([ + {"items": [{"id": 1}], "total_count": 1, "has_more": False}],), encoding="utf-8") + base = [sys.executable, str(MODPATH)] + p = subprocess.run(base + ["check-pages", "--pages", str(pages), "--key", "id"], capture_output=True, text=True, env=env) + self.assertEqual(p.returncode, 0, p.stderr) + b = td / "before.json"; a = td / "after.json" + b.write_text('{"x":1}', encoding="utf-8"); a.write_text('{"x":1}', encoding="utf-8") + p = subprocess.run(base + ["check-unchanged", "--before", str(b), "--after", str(a)], capture_output=True, text=True, env=env) + self.assertEqual(p.returncode, 0, p.stderr); self.assertIn('"unchanged": true', p.stdout) + + def test_cli_validate_render_parse(self): + with tempfile.TemporaryDirectory(dir="/opt/data/cache/scratch") as td: + td = Path(td); rf = td / "r.json"; af = td / "a.json"; bf = td / "b.md" + rf.write_text(json.dumps(result(), ensure_ascii=False), encoding="utf-8") + af.write_text(json.dumps(["src/a.py", "tests/test_a.py"]), encoding="utf-8") + env = dict(os.environ, PYTHONDONTWRITEBYTECODE="1") + base = [sys.executable, str(MODPATH)] + p = subprocess.run(base + ["validate-result", "--result", str(rf), "--allowed-paths", str(af)], + text=True, capture_output=True, env=env) + self.assertEqual(p.returncode, 0, p.stderr) + p = subprocess.run(base + ["render", "--no-delegated-evidence", "--result", str(rf), "--allowed-paths", str(af)], + text=True, capture_output=True, env=env) + self.assertEqual(p.returncode, 0, p.stderr); bf.write_text(p.stdout, encoding="utf-8") + p = subprocess.run(base + ["parse", "--body", str(bf), "--allowed-paths", str(af)], + text=True, capture_output=True, env=env) + self.assertEqual(p.returncode, 0, p.stderr) + self.assertEqual(json.loads(p.stdout), result()) + + def test_result_codec_preserves_existing_native_format(self): + r = result() + paths = {"src/a.py", "tests/test_a.py"} + body = mod.render_review(r, paths, declaration={"mode": "none"}) + self.assertIn(" - 구현 근거: src/a.py; 증거\n", body) + self.assertIn("모든 기준을 충족했다.\n\n', valid + '\n```unclosed', - valid.replace('### REQ', '#### REQ'), - valid.replace('## 8. 오류 및 충돌 계약', '## 7. 인수 기준'), - valid.replace('- 수준: unit', ' - 수준: unit')): - with self.subTest(bad=bad): - with self.assertRaises(mod.ValidationError): - self.check(bad) - crlf = valid.replace('\n', '\r\n') - self.assertEqual(self.check(crlf)['document_sha256'], hashlib.sha256(crlf.encode()).hexdigest()) - - def test_cli_read_only_exact_bytes_digest_and_sanitized_failure(self): - import hashlib - with tempfile.TemporaryDirectory() as tmp: - path = Path(tmp) / 'req.md' - raw = self.body().replace('\n', '\r\n').encode() - path.write_bytes(raw) - os.chmod(path, 0o600) - before = (path.read_bytes(), path.stat().st_mode, path.stat().st_mtime_ns, path.stat().st_ino) - argv = [sys.executable, '-B', str(MODPATH), 'check-ac-classification', - '--document', str(path), '--expected-document-sha256', hashlib.sha256(raw).hexdigest(), - '--expected-template-sha256', hashlib.sha256(self.TEMPLATE.read_bytes()).hexdigest()] - actual = subprocess.run(argv, capture_output=True, text=True, timeout=10) - self.assertEqual(actual.returncode, 0, actual.stderr) - self.assertEqual(json.loads(actual.stdout)['document_sha256'], hashlib.sha256(raw).hexdigest()) - self.assertEqual(before, (path.read_bytes(), path.stat().st_mode, path.stat().st_mtime_ns, path.stat().st_ino)) - argv[-3] = '0' * 64 - bad = subprocess.run(argv, capture_output=True, text=True, timeout=10) - self.assertEqual(bad.returncode, 2) - self.assertEqual(json.loads(bad.stderr), {'error': 'invalid input'}) - self.assertEqual(bad.stdout, '') - - - def test_supported_scope_rejects_hidden_html_and_noncanonical_ac_heading(self): - valid = self.body() - self.check(valid) - for bad in ('
\n' + valid + '
', - valid.replace('### REQ-001.01-AC01 — example', '### REQ-001.01-AC01'), - valid.replace('**검증 분류**', '**검증 분류**')): - with self.subTest(bad=bad): - with self.assertRaises(mod.ValidationError): - self.check(bad) - - def test_file_admission_rejects_symlink_fifo_invalid_utf8_and_oversize(self): - with tempfile.TemporaryDirectory() as tmp: - root = Path(tmp) - valid = root / 'valid.md' - valid.write_text(self.body()) - self.assertEqual(mod._classification_bytes(valid), valid.read_bytes()) - alias = root / 'alias.md' - alias.symlink_to(valid) - fifo = root / 'fifo' - os.mkfifo(fifo) - invalid = root / 'invalid.md' - invalid.write_bytes(b'\xff') - large = root / 'large.md' - large.write_bytes(b'x' * 1048577) - for path in (alias, fifo, invalid, large, root): - with self.subTest(path=path.name): - with self.assertRaises(mod.ValidationError): - mod._classification_bytes(path) - - def test_documentation_contract_scopes_checker_and_binds_digests(self): - text = (MODPATH.parent.parent / 'SKILL.md').read_text() - self.assertIn('### AC 검증 분류의 read-only 구조 검사', text) - section = text.split('### AC 검증 분류의 read-only 구조 검사', 1)[1].split('\n## ', 1)[0] - for rule in ('check-ac-classification', '--expected-document-sha256', - '--expected-template-sha256', '전체 문서 적합성', '의미', - '없음', '미확인', '코드 fence', '새 schema', '단일 행'): - self.assertIn(rule, section) - - -class SupportTests(unittest.TestCase): - def test_loads_json_rejects_duplicate_and_nonfinite(self): - with self.assertRaises(mod.ValidationError): - mod.loads_json('{"a":1,"a":2}') - with self.assertRaises(mod.ValidationError): - mod.loads_json('{"a":NaN}') - - def test_validate_and_render_roundtrip(self): - r = result() - self.assertEqual(mod.validate_result(r, {"src/a.py", "tests/test_a.py"}), r) - body = mod.render_review(r, {"src/a.py", "tests/test_a.py"}, declaration={"mode": "none"}) - parsed = mod.parse_review(body, {"src/a.py", "tests/test_a.py"}) - self.assertEqual(parsed, r) - self.assertEqual(mod.render_review(parsed, {"src/a.py", "tests/test_a.py"}, declaration={"mode": "none"}), body) - - def test_validation_rejects_bool_path_traversal_bad_verdict(self): - r = result() - r["target"]["issue"] = True - with self.assertRaises(mod.ValidationError): - mod.validate_result(r, {"src/a.py"}) - r = result(); r["criteria"][0]["implementation_paths"] = ["../x"] - with self.assertRaises(mod.ValidationError): - mod.validate_result(r, {"../x"}) - r = result("request_changes") - with self.assertRaises(mod.ValidationError): - mod.validate_result(r, {"src/a.py", "tests/test_a.py"}) - - def test_collect_pages_requires_identity_and_reads_all_pages(self): - pages = [[{"id": 1}, {"id": 2}], [{"id": 3}]] - def fetch(n): - return {"items": pages[n - 1], "total_count": 3, "has_more": n == 1} - self.assertEqual(mod.collect_pages(fetch, "id", page_size=2), pages[0] + pages[1]) - def bad(n): - return {"items": [{"id": 1}], "total_count": 2, "has_more": True} - with self.assertRaises(mod.ValidationError): - mod.collect_pages(bad, "id", max_pages=1) - - def test_verify_unchanged(self): - self.assertTrue(mod.verify_unchanged({"a": [1]}, {"a": [1]})) - with self.assertRaises(mod.ValidationError): - mod.verify_unchanged({"a": 1}, {"a": 2}) - - def test_cache_validation_and_rejects_symlink(self): - with tempfile.TemporaryDirectory(dir="/opt/data/cache/scratch") as td: - root = Path(td); ops = root / "harness" / "operations"; ops.mkdir(parents=True) - contract = ops / "repository-cache.md"; contract.write_text("cache", encoding="utf-8") - rs = root / "repositories"; owner = rs / "acme"; repo = owner / "repo" - (repo / "locks").mkdir(parents=True) - (repo / "repository.git").mkdir() - for p in (rs, owner, repo, repo / "locks"): os.chmod(p, 0o700) - info = mod.validate_cache_paths(contract, "acme", "repo", 3) - self.assertTrue(info["repository_git"].endswith("repository.git")) - (repo / "locks" / "issue-3.lock").symlink_to(contract) - with self.assertRaises(mod.ValidationError): - mod.validate_cache_paths(contract, "acme", "repo", 3) - - def test_publication_accepts_one_exact_review_and_rejects_competing_event(self): - before, after, review, r, paths = publication_fixture() - self.assertEqual(verify_fixture(before, after, review, r, paths)["review_id"], 11) - bad = copy.deepcopy(after) - bad["timeline"].append({"id": 102, "review_id": 12, "user": {"id": 42}, "body": review["body"]}) - with self.assertRaises(mod.ValidationError): - verify_fixture(before, bad, review, r, paths) - - def test_cli_check_pages_and_unchanged(self): - with tempfile.TemporaryDirectory(dir="/opt/data/cache/scratch") as td: - td = Path(td); env = dict(os.environ, PYTHONDONTWRITEBYTECODE="1") - pages = td / "pages.json"; pages.write_text(json.dumps([ - {"items": [{"id": 1}], "total_count": 1, "has_more": False}],), encoding="utf-8") - base = [sys.executable, str(MODPATH)] - p = subprocess.run(base + ["check-pages", "--pages", str(pages), "--key", "id"], capture_output=True, text=True, env=env) - self.assertEqual(p.returncode, 0, p.stderr) - b = td / "before.json"; a = td / "after.json" - b.write_text('{"x":1}', encoding="utf-8"); a.write_text('{"x":1}', encoding="utf-8") - p = subprocess.run(base + ["check-unchanged", "--before", str(b), "--after", str(a)], capture_output=True, text=True, env=env) - self.assertEqual(p.returncode, 0, p.stderr); self.assertIn('"unchanged": true', p.stdout) - - def test_cli_validate_render_parse(self): - with tempfile.TemporaryDirectory(dir="/opt/data/cache/scratch") as td: - td = Path(td); rf = td / "r.json"; af = td / "a.json"; bf = td / "b.md" - rf.write_text(json.dumps(result(), ensure_ascii=False), encoding="utf-8") - af.write_text(json.dumps(["src/a.py", "tests/test_a.py"]), encoding="utf-8") - env = dict(os.environ, PYTHONDONTWRITEBYTECODE="1") - base = [sys.executable, str(MODPATH)] - p = subprocess.run(base + ["validate-result", "--result", str(rf), "--allowed-paths", str(af)], - text=True, capture_output=True, env=env) - self.assertEqual(p.returncode, 0, p.stderr) - p = subprocess.run(base + ["render", "--no-delegated-evidence", "--result", str(rf), "--allowed-paths", str(af)], - text=True, capture_output=True, env=env) - self.assertEqual(p.returncode, 0, p.stderr); bf.write_text(p.stdout, encoding="utf-8") - p = subprocess.run(base + ["parse", "--body", str(bf), "--allowed-paths", str(af)], - text=True, capture_output=True, env=env) - self.assertEqual(p.returncode, 0, p.stderr) - self.assertEqual(json.loads(p.stdout), result()) - - - def test_result_codec_preserves_existing_native_format(self): - r = result() - paths = {"src/a.py", "tests/test_a.py"} - body = mod.render_review(r, paths, declaration={"mode": "none"}) - self.assertIn(" - 구현 근거: src/a.py; 증거\n", body) - self.assertIn("모든 기준을 충족했다.\n\n