[verified] Canonicalize native review publication journals

This commit is contained in:
2026-10-11 09:51:49 +00:00
parent 18222f2ca8
commit 17751f65fe
12 changed files with 1444 additions and 57 deletions
@@ -119,3 +119,12 @@ Finding마다 `F-001` 같은 ID, 파일·행, 발생 조건, 영향·심각도,
- PR의 성공 주장, delegate 요약, process exit 또는 native state만으로 완료를 주장하지 않는다. HTTP/framework test를 실제 browser 검증으로 표현하지 않는다.
- 보고에는 exact target·head·contract·spec review ID, 검토 범위, findings·verdict, 실제 실행과 재사용 근거, 미검증 범위, native review ID/state/readback 또는 미게시 이유, cleanup·partial 상태를 포함한다.
- clean review는 검토 범위에서 중요 결함을 발견하지 못했다는 의미이며 무결함의 증명은 아니다.
## Canonical native-review journal
Native publication의 정본 root는 `<harness-root>/harness/.runtime/native-review-journals`이며 authenticated origin/repository/Issue/PR/head/contract/reviewer binding으로 결정한다. `canonical_journal_path(plan)`은 표시용 계산일 뿐 인증 증명이 아니다. 임의 caller 경로·alias는 거절하고 같은 binding의 body/verdict 변경, 삭제된 기록, interrupted bootstrap/import는 fail-closed한다.
이전 durable intent가 있으면 read-only reconciliation만 수행하고 자동 POST 재시도는 금지한다. Legacy는 현재 role/lease/adoption과 사전 exact digest를 검증하는 명시적 `register_legacy_journal` 절차만 허용한다. 기존 journal을 자동 이동·삭제하지 않으며 등록도 외부 write를 하지 않는다. Same-install retained registry/cooperative filesystem 범위이며 cross-host 또는 malicious same-UID 보호를 주장하지 않는다.
Binding/permission/atomic-bootstrap/legacy pending receipt와 보장 한계의 정본 계약은 [native review reference](references/native-review.md#canonical-native-review-journal)를 따른다. `verified.json`은 publication 확인이며 최종 source 검증·lease release·cleanup 완료를 대체하지 않는다.
@@ -6,7 +6,7 @@ This adapter is a publication guard, not a substitute for the main reviewer's co
## Supported interface
- Python API: `build_payload(plan)`, `validate_snapshot(plan, snapshot)`, `verify_publication(plan, before, after, exact)`, and `publish(plan, baseline, api, lease, journal_directory)`.
- Python API: `build_payload(plan)`, `validate_snapshot(plan, snapshot)`, `verify_publication(plan, before, after, exact)`, and `publish(plan, baseline, api, lease, journal_directory=None)`, `canonical_journal_path(plan)`, and explicit `register_legacy_journal(...)`.
- REST bridge: `ApiAdapter(authenticated_request, origin=..., credential_source=...)`.
- Primary lease bridge: `BrokerIssueLease(broker_handle, absolute_cache_policy, deadline_monotonic)` backed by `managed_lock_broker.py`; `HeldIssueLease(issue_fd, absolute_cache_policy, deadline_monotonic)` remains compatibility-only for already-admitted legacy callers. Broker requests are limited to `status`, `renew`, `acquire_repository`, `release_repository` and `release`; renew before expiry and use canonical release/wait/reap/probe cleanup.
- Read-only CLI: `preflight --plan PLAN --baseline SNAPSHOT` and `verify --plan PLAN --before BEFORE --after AFTER --exact EXACT`.
@@ -40,11 +40,11 @@ Keep `gitea-access.md` and `repository-cache.md` authoritative. Before calling `
2. Authenticate **one** transport from the current role's `$HERMES_HOME/.env` `GITEA_TOKEN`, or an explicitly authorized role-scoped external secret source. No ambient-token fallback, cross-profile credentials, credential-bearing Git URLs or anonymous downgrade is allowed.
3. Validate the exact fixed origin `https://gitea.jsfamily.quest/api/v1` and `GET /user` account ID/login. The transport must verify TLS, reject redirects, bound network timeouts and **not retry mutation requests**. The bridge's provenance labels are assertions supplied by this already-validated caller, not a mechanism to load or certify an arbitrary client.
4. Use the common broker Issue lease throughout publication/readback. Pass a live monotonic deadline, not an old PID/`ready.json` flag. `BrokerIssueLease` checks broker status and canonical worktree invariants; renew before expiry. `HeldIssueLease` is compatibility-only and may be used only by an already-admitted legacy caller, never to create a second lock protocol. The caller must run canonical release/wait/reap/probe cleanup. A custom supervisor lease must implement the same invariants: `check(plan)` returns `None` only on success and otherwise raises; boolean/dict success flags are unsupported. Finalization follows the [review execution procedure](../SKILL.md#실행-검증과-기존-워크트리-사용): complete required final source verification under the held lease and preserve its evidence before releasing. `verified.json` proves publication verification, not workflow cleanup. Retain the same publication journal and existing result artifacts when cleanup is incomplete; report workflow partial and reconcile only unfinished cleanup checks, without replaying a full review, suite or POST. Keep the common Issue-resource `free=true` completion requirement; do not add a released-only terminal-handle gate or a new automatic retry/heartbeat loop.
5. Put the journal outside every source checkout in a trusted artifact parent. Use the **same journal for every retry/resume of this plan**, not a new directory after a timeout. The adapter creates mode 0700 directories and 0600 atomic/fsynced artifacts, rejects symlinks/hardlinks/foreign or unsafe modes, and holds a separate nonblocking journal mutex. Redact secrets before constructing plans/artifacts; do not log request headers, raw transport exceptions or token values. Journals contain complete source snapshots and must remain private.
5. Use the canonical binding journal at `<harness-root>/harness/.runtime/native-review-journals`; do not select a caller-owned run directory. The publisher enforces the same journal for every retry/resume of this binding. The adapter creates mode 0700 directories and 0600 atomic/fsynced artifacts, rejects symlinks/hardlinks/foreign or unsafe modes, and holds a separate nonblocking journal mutex. Redact secrets before constructing plans/artifacts; do not log request headers, raw transport exceptions or token values. Journals contain complete source snapshots and must remain private.
The current helper is strict about worktree child directories: group/world-writable checkout paths are rejected. Do not change a real worktree's permissions to force publication; stop and reconcile an admission/publisher compatibility conflict. The legacy compatibility bridge is Linux-only: it validates an already-admitted broker-held whole-file lease represented by an exclusive `FLOCK` owned by this process in `/proc/self/fdinfo`, and never acquires or reacquires that resource. New callers use `BrokerIssueLease` and the broker public request API. It uses a root-owned, non-group/world-writable `/usr/bin/git` with a fixed minimal environment, not the caller's `PATH`.
These guards assume a cooperative, trusted role-local filesystem and transport. They do not isolate a malicious process with the same UID or pin every filesystem ancestor across subprocess path resolution. At-most-once creation is scoped to the retained journal and held canonical Issue lease, **not** arbitrary alternate journal paths, unrelated publishers, or multiple hosts. Do not share one lease across simultaneous publication callers or change the journal after an uncertain attempt.
These guards assume a cooperative, trusted role-local filesystem and transport. They do not isolate a malicious process with the same UID or pin every filesystem ancestor across subprocess path resolution. At-most-once creation is scoped to the retained canonical binding registry and held canonical Issue lease; alternate caller-selected paths are rejected. Unrelated publishers, other installations and cross-host coordination remain outside this guarantee. Do not share one lease across simultaneous publication callers or change the journal after an uncertain attempt.
The request callback is `authenticated_request(relative_api_path, payload=None) -> (decoded_json, headers, integer_status)`. `None` means GET; a dict means exactly one POST. It always uses the fixed origin and original role credential, with no authorization fallback. It is provided by the existing role-local transport; do not regenerate the full gate/publisher per review run.
@@ -63,7 +63,7 @@ api = native_review.ApiAdapter(
lease = native_review.BrokerIssueLease(broker_handle, absolute_cache_policy, deadline_monotonic)
baseline = api.snapshot(plan["target"])
native_review.validate_snapshot(plan, baseline)
result = native_review.publish(plan, baseline, api, lease, journal_directory)
result = native_review.publish(plan, baseline, api, lease)
```
Preserve the original baseline until `publish` returns verified or the uncertainty is manually reconciled. The helper rechecks authoritative role/head/base/contract/default branch/source Issue linkage/spec approval and complete source snapshots immediately before creating. It explicitly sends `commit_id`, `body`, `event`, `comments` and performs at most one `create_review` call per durable attempt.
@@ -96,3 +96,35 @@ PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s harness/tests/impl -v
```
Run the test commands from the canonical harness repository root. The existing spec suite requires `/opt/data/cache/scratch`: prepare that directory only if absent after confirming a nonsymlink, role-owned, non-group/world-writable parent, and require mode 0700 for the scratch child. Preserve existing shared parent permissions; do not chmod/chown them to satisfy a test. Use isolated/local runner variables and properly quoted command arguments rather than reusing mutable interpreter globals across review steps. Keep local stub/loopback transport checks, real temporary Git/worktree/lease checks, and saved production-wire replay clearly distinguished from a live Gitea mutation test. Introducing this helper does not authorize changing any existing product PR verdict, source, Issue state, merge or deployment.
## Canonical native-review journal
Native code-review publication의 정본 journal root는 `<harness-root>/harness/.runtime/native-review-journals`이다. Canonical publisher의 물리적인 harness 위치에서 결정하며 caller의 run-id, 임의 journal 경로, 환경 변수 변경으로 다른 registry를 선택하지 않는다. `canonical_journal_path(plan)`은 검증된 plan의 표시용 경로 계산일 뿐 인증·게시 권한의 증명이 아니다. 실제 registry claim 전에 현재 role identity와 held Issue lease 및 source/delegation gate를 검증한다.
Versioned binding은 고정 provider origin, exact owner/repository, Issue, PR, head SHA, contract digest, reviewer의 인증된 stable ID를 묶는다. Body, verdict, run-id는 binding key에 넣지 않는다. 같은 binding의 full-plan fingerprint가 달라지면 충돌로 차단한다. `publish(plan, baseline, api, lease)`는 정본 entry를 사용하며 기존 positional journal 인자를 넘길 경우 exact canonical path만 허용한다. Alternate path, path alias, missing/corrupt identity, symlink/hardlink, owner/mode 불일치 및 중간 상태는 fail-closed하며 기존 directory를 자동 chmod하거나 repair하지 않는다.
Private root/entry는 mode0700, artifact와 mutex는 mode0600이다. 동일 binding의 nonblocking process mutex 아래 entry 확인 → durable intent 기록 → 단일 POST → authoritative readback을 수행한다. 이전 attempt가 있으면 timeout, crash, readback 실패, lease 만료 뒤에도 read-only reconciliation만 허용하며 자동 POST 재시도는 하지 않는다. Root/entry presence anchor로 알려진 기록의 삭제와 interrupted bootstrap을 새 게시로 오인하지 않는다. Pending registration 역시 게시를 차단한다. `verified.json`은 publication 확인이며 source 최종 검증·lease release·cleanup 완료를 대신하지 않는다.
Legacy 등록은 명시적으로 승인된 `register_legacy_journal` 경로만 사용한다. 기존 journal을 자동 이동·삭제하지 않는다. Source는 이미 존재하는 private journal이어야 하며 정확히 선택한 기록의 사전 expected digest, plan/intent/baseline binding, 현재 role 인증·lease/adoption을 확인한다. 완성된 durable intent가 없는 legacy, 기존 canonical entry와 충돌하는 등록은 거절한다. Import pending을 복사 전에 fsync하고 완료 receipt 전에는 publication을 금지한다. 승인된 known artifacts만 복사하고 원본 bytes·권한을 보존하며 등록 자체는 Gitea write를 하지 않는다. 중단된 import는 자동으로 fresh attempt로 복구하지 않는다. Registered intent의 재개도 read-only 확인이며 authoritative match가 없으면 uncertainty를 유지한다.
보장은 동일 설치의 retained role-local registry와 cooperative trusted filesystem/transport 범위다. Shared harness ancestry의 배포 trust는 기존 repository-cache 계약을 따른다. Malicious same-UID process, registry와 anchor를 모두 지우는 운영자, 임의 다른 publisher/install, cross-host global exactly-once는 보장하지 않는다. 다른 host·설치로 이전하려면 별도 승인된 공유 조정/기록 이전 절차가 필요하며 새 root를 만들어 retry하지 않는다. 기존 provider collection 및 snapshot 비교 규칙은 이 journal 변경으로 완화하지 않는다.
### Explicit legacy digest and registration
등록할 source path를 운영자가 명시적으로 승인해야 한다. `legacy_journal_digest(legacy_directory)`는 기존 `publication.lock`을 nonblocking으로 잡아 source를 변경하지 않고 exact bytes inventory의 SHA-256을 반환한다. Digest는 아래 순서의 존재하는 known file들에 대해 filename·raw byte length·raw SHA-256을 담은 `gitea-agent.legacy-native-journal-digest/v1` canonical JSON의 SHA-256이다: `before.json`, `intent.json`, optional `verified.json`. Source content를 출력하거나 새 lock을 만들지 않는다.
```python
approved_digest = native_review.legacy_journal_digest(approved_legacy_directory)
# Freeze/approve this exact digest independently before registration.
native_review.register_legacy_journal(
plan, approved_legacy_directory,
expected_legacy_sha256=approved_digest,
api=api, lease=lease, declaration=adoption_declaration,
)
```
Legacy source의 entry는 `before.json`, `intent.json`, optional `verified.json`, 기존 `publication.lock`만 허용한다. 추가 파일/디렉터리, symlink/hardlink/FIFO, 중복 JSON key 및 nonfinite/overflow numeric 값은 거절한다. 승인된 before/intent는 raw bytes 그대로 복사하며 legacy verified는 `legacy-verified.json`으로 별도 보존한다. Receipt의 source→target 목록은 완전히 일치해야 하며 각 복사 기록의 길이와 raw SHA-256을 확인한 뒤 완료 처리한다. 재개 때도 모든 imported artifact의 무결성을 확인한다. 이후 `verified.json`의 현재 readback 기록 갱신은 보존된 legacy archive를 덮어쓰지 않는다. 여기서 read-only reconciliation은 provider에 대한 read-only이며 로컬 journal의 확인 결과 갱신은 수행할 수 있다.
Canonical entry가 이미 존재하면 빈 entry라도 legacy 등록으로 덮어쓰지 않는다. Bootstrap mutex 아래 pending sentinel을 먼저 durable하게 만들고, 영구 `registration.json` pending receipt → known artifacts 복사/fsync → complete receipt/fsync → sentinel 제거/fsync 순서를 따른다. Pending receipt나 sentinel, 손실된 복사 기록은 모두 게시를 차단한다. 등록이 성공해도 `publish`의 read-only reconciliation과 authoritative readback이 필요하며 등록 자체는 provider 게시 성공의 증명이 아니다. 운영자가 원본을 삭제하거나 보존 기록/anchor를 초기화해 새 시도로 처리하면 이 보장의 범위를 벗어난다.
@@ -22,6 +22,10 @@ _BROKER_PATH = Path(__file__).resolve().parents[5] / 'lib/managed_lock_broker.py
_broker_spec = importlib.util.spec_from_file_location('gitea_managed_lock_broker', _BROKER_PATH)
_broker = importlib.util.module_from_spec(_broker_spec)
_broker_spec.loader.exec_module(_broker)
_JOURNAL_PATH = Path(__file__).resolve().with_name('publication_journal.py')
_journal_spec = importlib.util.spec_from_file_location('gitea_publication_journal', _JOURNAL_PATH)
_journal = importlib.util.module_from_spec(_journal_spec)
_journal_spec.loader.exec_module(_journal)
ValidationError = _support.ValidationError
loads_json = _support.loads_json
collect_pages = _support.collect_pages
@@ -29,6 +33,249 @@ ORIGIN = 'https://gitea.jsfamily.quest'
SCHEMA = 'gitea-agent.code-review-publication/v1'
def canonical_journal_path(plan):
"""Return the fixed, versioned path for this publication binding."""
return _journal.canonical_path(plan, anchor_file=__file__, origin=ORIGIN,
validate=validate_plan)
def _expected_intent(plan, before):
payload = build_payload(plan)
return {'schema': 'gitea-agent.native-review-intent/v1',
'fingerprint': hashlib.sha256(canonical(plan)).hexdigest(),
'before_sha256': hashlib.sha256(canonical(before)).hexdigest(),
'head': plan['target']['head'], 'event': payload['event'],
'body_sha256': hashlib.sha256(payload['body'].encode()).hexdigest()}
def _legacy_path(directory):
import os, stat
value = os.fspath(directory)
if not isinstance(value, str) or not value.startswith('/') or any(
part in ('', '.', '..') for part in value.split('/')[1:]):
fail('unsafe legacy journal path')
source = Path(value)
try:
_journal._check_ancestor_chain(source)
st = source.lstat()
except (OSError, ValueError):
fail('unsafe legacy journal')
if not stat.S_ISDIR(st.st_mode) or st.st_uid != os.getuid() or stat.S_IMODE(st.st_mode) != 0o700:
fail('unsafe legacy journal')
return source, (st.st_dev, st.st_ino)
from contextlib import contextmanager
@contextmanager
def _locked_legacy(directory):
import fcntl, os, stat
source, identity = _legacy_path(directory)
fd = None
try:
# Never create a source mutex: registration preserves the entire source.
fd = os.open(source / 'publication.lock', os.O_RDWR | os.O_NOFOLLOW | os.O_NONBLOCK)
st = os.fstat(fd)
if not stat.S_ISREG(st.st_mode) or st.st_uid != os.getuid() or stat.S_IMODE(st.st_mode) != 0o600 or st.st_nlink != 1:
fail('unsafe legacy journal lock')
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
def check():
_, current = _legacy_path(source)
live = (source / 'publication.lock').lstat()
if current != identity or (live.st_dev, live.st_ino) != (st.st_dev, st.st_ino):
fail('legacy journal identity changed')
check()
yield source, check
check()
except ValidationError:
raise
except (OSError, ValueError):
fail('legacy journal unavailable or unsafe')
finally:
if fd is not None:
os.close(fd)
def _capture_legacy(source):
import stat
records = {}; raw_records = {}; inventory = []
allowed = {'before.json', 'intent.json', 'verified.json', 'publication.lock'}
try:
entries = list(source.iterdir())
except OSError:
fail('legacy journal artifact unavailable or unsafe')
for entry in entries:
if entry.name not in allowed:
fail('unknown legacy journal artifact')
try:
st = entry.lstat()
except OSError:
fail('legacy journal artifact unavailable or unsafe')
if entry.name == 'publication.lock':
continue
if not stat.S_ISREG(st.st_mode) or st.st_nlink != 1:
fail('unsafe legacy journal artifact')
for name in ('before.json', 'intent.json', 'verified.json'):
try:
raw = _journal._secure_file(source / name, cap=16 * 1024 * 1024 if name == 'before.json' else 1024 * 1024)
except FileNotFoundError:
continue
except (OSError, ValueError):
fail('legacy journal artifact unavailable or unsafe')
try:
records[name] = loads_json(raw.decode('utf-8'))
raw_records[name] = raw
except (UnicodeError, ValueError):
fail('malformed legacy journal artifact')
inventory.append({'name': name, 'size': len(raw), 'sha256': hashlib.sha256(raw).hexdigest()})
framed = {'schema': 'gitea-agent.legacy-native-journal-digest/v1', 'files': inventory}
return records, raw_records, framed, hashlib.sha256(canonical(framed)).hexdigest()
def legacy_journal_digest(legacy_directory):
"""Read-only digest of exact, named source bytes under its existing mutex."""
with _locked_legacy(legacy_directory) as (source, check):
_, _, _, digest_value = _capture_legacy(source)
check()
return digest_value
def _assert_registry(entry, plan):
try:
_journal.assert_bound(entry, plan, anchor_file=__file__, origin=ORIGIN, validate=validate_plan)
except (OSError, ValueError):
fail('canonical journal binding unavailable or changed')
def _check_registration(journal, plan):
if journal.read('import.pending.json') is not None:
fail('interrupted legacy registration')
receipt = journal.read('registration.json')
if receipt is None:
return
keys = {'schema', 'state', 'legacy_sha256', 'source_files', 'imported_files', 'before_sha256', 'intent_sha256'}
if type(receipt) is not dict or set(receipt) != keys or receipt['schema'] != 'gitea-agent.native-review-import/v1' or receipt['state'] != 'complete':
fail('interrupted or malformed legacy registration')
files = receipt['source_files']
if type(files) is not list or len(files) not in (2, 3):
fail('malformed legacy registration file inventory')
if [f.get('name') if type(f) is dict else None for f in files] not in (
['before.json', 'intent.json'], ['before.json', 'intent.json', 'verified.json']):
fail('malformed legacy registration file inventory')
for f in files:
if set(f) != {'name', 'size', 'sha256'} or type(f['size']) is not int or not 0 <= f['size'] <= (16 * 1024 * 1024 if f['name'] == 'before.json' else 1024 * 1024):
fail('malformed legacy registration file inventory')
digest(f['sha256'], 64)
imported = receipt['imported_files']
if type(imported) is not list or [f.get('source') if type(f) is dict else None for f in imported] not in (
['before.json', 'intent.json'], ['before.json', 'intent.json', 'verified.json']):
fail('malformed imported legacy file inventory')
if [f['source'] for f in imported] != [f['name'] for f in files]:
fail('incomplete imported legacy file inventory')
for f in imported:
if (set(f) != {'source', 'target', 'size', 'sha256'}
or f['target'] != ('legacy-verified.json' if f['source'] == 'verified.json' else f['source'])
or type(f['size']) is not int or f['size'] < 0):
fail('malformed imported legacy file inventory')
digest(f['sha256'], 64)
source = next((row for row in files if row['name'] == f['source']), None)
if source is None or source['size'] != f['size'] or source['sha256'] != f['sha256']:
fail('legacy source/import receipt mismatch')
for field in ('legacy_sha256', 'before_sha256', 'intent_sha256'):
digest(receipt[field], 64)
framed = {'schema': 'gitea-agent.legacy-native-journal-digest/v1', 'files': files}
if hashlib.sha256(canonical(framed)).hexdigest() != receipt['legacy_sha256']:
fail('legacy registration receipt mismatch')
before = journal.read('before.json'); intent = journal.read('intent.json')
if before is None or intent is None or hashlib.sha256(canonical(before)).hexdigest() != receipt['before_sha256'] or hashlib.sha256(canonical(intent)).hexdigest() != receipt['intent_sha256']:
fail('legacy registration artifact mismatch')
if canonical(intent) != canonical(_expected_intent(plan, before)):
fail('legacy registration intent mismatch')
for f in imported:
try:
raw = _journal._secure_file(Path(journal.path) / f['target'],
cap=16 * 1024 * 1024 if f['source'] == 'before.json' else 1024 * 1024)
except (OSError, ValueError):
fail('legacy imported artifact unavailable or unsafe')
if len(raw) != f['size'] or hashlib.sha256(raw).hexdigest() != f['sha256']:
fail('legacy imported artifact integrity mismatch')
try:
loads_json(raw.decode('utf-8'))
except (UnicodeError, ValueError):
fail('malformed legacy imported artifact')
def register_legacy_journal(plan, legacy_directory, *, expected_legacy_sha256,
api, lease, declaration=None):
"""Explicit, local-only import; preserve source and never retry native POST."""
validate_plan(plan)
if type(expected_legacy_sha256) is not str or not re.fullmatch(r'[0-9a-f]{64}', expected_legacy_sha256):
fail('invalid legacy journal digest')
require_adoption(declaration, plan['target']['head'])
check_lease(lease, plan)
role_identity(read_api(api, 'identity'), plan['reviewer'])
validate_snapshot(plan, read_api(api, 'snapshot', plan['target']))
with _locked_legacy(legacy_directory) as (source, check):
records, raw_records, framed, digest_value = _capture_legacy(source)
if digest_value != expected_legacy_sha256:
fail('legacy journal digest mismatch')
before = records.get('before.json'); intent = records.get('intent.json')
if type(before) is not dict or type(intent) is not dict:
fail('legacy journal lacks durable intent')
validate_snapshot(plan, before)
if canonical(intent) != canonical(_expected_intent(plan, before)):
fail('legacy journal intent mismatch')
if 'verified.json' in records and type(records['verified.json']) is not dict:
fail('invalid legacy verification artifact')
check(); check_lease(lease, plan)
target = canonical_journal_path(plan)
try:
entry = _journal.open_binding(target, plan, anchor_file=__file__, origin=ORIGIN,
validate=validate_plan, require_new=True)
except (OSError, ValueError):
fail('canonical registration target unavailable or already exists')
with Journal(entry) as journal:
_assert_registry(entry, plan)
if journal.read('import.pending.json') != {'schema': 'gitea-agent.native-review-import/v1', 'state': 'pending'}:
fail('legacy registration bootstrap incomplete')
if any(journal.read(name) is not None for name in ('before.json', 'intent.json', 'verified.json', 'registration.json')):
fail('canonical journal already populated')
receipt = {'schema': 'gitea-agent.native-review-import/v1', 'state': 'pending',
'legacy_sha256': digest_value, 'source_files': framed['files'],
'before_sha256': hashlib.sha256(canonical(before)).hexdigest(),
'intent_sha256': hashlib.sha256(canonical(intent)).hexdigest()}
# Durable pending receipt precedes copying any former attempt.
journal.write('registration.json', receipt)
imported_files = []
for name in ('before.json', 'intent.json', 'verified.json'):
if name in raw_records:
target_name = 'legacy-verified.json' if name == 'verified.json' else name
try:
_journal._atomic_bytes(entry, target_name, raw_records[name])
persisted = _journal._secure_file(entry / target_name,
cap=16 * 1024 * 1024 if name == 'before.json' else 1024 * 1024)
except (OSError, ValueError):
fail('legacy artifact import failed')
if persisted != raw_records[name]:
fail('legacy artifact integrity mismatch')
imported_files.append({'source': name, 'target': target_name,
'size': len(raw_records[name]),
'sha256': hashlib.sha256(raw_records[name]).hexdigest()})
receipt['imported_files'] = imported_files
journal.write('registration.json', receipt)
check()
_assert_registry(entry, plan)
receipt['state'] = 'complete'
journal.write('registration.json', receipt)
# The final receipt is durable before removing bootstrap's sentinel.
import os
journal.check_identity()
os.unlink(entry / 'import.pending.json')
_journal._fsync_dir(entry)
return {'registered': True, 'journal_directory': str(entry), 'read_only': True}
def fail(message):
raise ValidationError(message)
@@ -396,6 +643,21 @@ class Journal:
if not stat.S_ISDIR(st.st_mode) or st.st_uid != os.getuid() or stat.S_IMODE(st.st_mode) != 0o700:
fail('unsafe journal directory')
self.fd = None
self.identity = (st.st_dev, st.st_ino)
def check_identity(self):
import os, stat
try:
_journal._check_ancestor_chain(self.path)
st = self.path.lstat()
if (st.st_dev, st.st_ino) != self.identity or st.st_uid != os.getuid() or stat.S_IMODE(st.st_mode) != 0o700:
fail('journal directory identity changed')
if self.fd is not None:
held = os.fstat(self.fd); named = (self.path / 'publication.lock').lstat()
if (held.st_dev, held.st_ino) != (named.st_dev, named.st_ino) or not stat.S_ISREG(named.st_mode) or named.st_nlink != 1 or named.st_uid != os.getuid() or stat.S_IMODE(named.st_mode) != 0o600:
fail('journal mutex identity changed')
except (OSError, ValueError):
fail('journal identity unavailable or changed')
def __enter__(self):
import fcntl, os, stat
@@ -405,6 +667,10 @@ class Journal:
if not stat.S_ISREG(st.st_mode) or st.st_uid != os.getuid() or stat.S_IMODE(st.st_mode) != 0o600 or st.st_nlink != 1:
fail('unsafe journal lock')
fcntl.flock(self.fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
self.check_identity()
except OSError:
os.close(self.fd); self.fd = None
fail('journal mutex unavailable or busy')
except BaseException:
os.close(self.fd); self.fd = None
raise
@@ -413,29 +679,38 @@ class Journal:
def __exit__(self, *args):
import os
os.close(self.fd)
self.fd = None
def read(self, name):
import os, stat
p = self.path / name
self.check_identity()
if name not in ('before.json', 'intent.json', 'verified.json', 'import.pending.json', 'registration.json'):
fail('unsupported journal artifact')
try:
fd = os.open(p, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK)
cap = 16 * 1024 * 1024 if name == 'before.json' else 1024 * 1024
data = _journal._secure_file(self.path / name, cap=cap)
except FileNotFoundError:
return None
except OSError:
raise ValidationError('journal artifact unavailable') from None
with os.fdopen(fd, 'r', encoding='utf-8') as artifact:
st = os.fstat(artifact.fileno())
if not stat.S_ISREG(st.st_mode) or st.st_uid != os.getuid() or stat.S_IMODE(st.st_mode) != 0o600 or st.st_nlink != 1:
fail('unsafe journal artifact')
return loads_json(artifact.read())
except (OSError, ValueError):
fail('journal artifact unavailable or unsafe')
try:
value = loads_json(data.decode('utf-8'))
except (UnicodeError, ValueError):
fail('malformed journal artifact')
self.check_identity()
return value
def write(self, name, value):
import os, tempfile
self.read(name) # Reject existing symlink/hardlink/foreign artifacts.
data = canonical(value)
cap = 16 * 1024 * 1024 if name == 'before.json' else 1024 * 1024
if len(data) > cap:
fail('journal artifact exceeds bounds')
fd, tmp = tempfile.mkstemp(dir=self.path, prefix='.native-review-')
try:
with os.fdopen(fd, 'wb') as out:
out.write(canonical(value)); out.flush(); os.fsync(out.fileno())
out.write(data); out.flush(); os.fsync(out.fileno())
self.check_identity()
os.replace(tmp, self.path / name)
directory = os.open(self.path, os.O_RDONLY | os.O_DIRECTORY)
try:
@@ -474,8 +749,8 @@ def check_lease(lease, plan):
fail('unsupported lease check result')
def publish(plan, baseline, api, lease, journal_directory, *, declaration=None):
"""Use one trusted role transport and an already-held Issue lease; never retry POST."""
def publish(plan, baseline, api, lease, journal_directory=None, *, declaration=None):
"""Publish through the fixed canonical journal binding; never retry POST."""
import copy
validate_snapshot(plan, baseline)
require_adoption(declaration, plan['target']['head'])
@@ -486,8 +761,24 @@ def publish(plan, baseline, api, lease, journal_directory, *, declaration=None):
fail('missing held Issue lease')
fingerprint = hashlib.sha256(canonical(plan)).hexdigest()
check_lease(lease, plan)
with Journal(journal_directory) as journal:
expected_directory = canonical_journal_path(plan)
if journal_directory is not None:
import os
supplied = os.fspath(journal_directory)
if type(supplied) is not str or supplied != str(expected_directory):
fail('non-canonical journal path')
# Authenticate the reviewer before claiming a registry binding.
role_identity(read_api(api, 'identity'), plan['reviewer'])
try:
bound_directory = _journal.open_binding(expected_directory, plan, anchor_file=__file__, origin=ORIGIN, validate=validate_plan)
except (OSError, ValueError):
fail('canonical journal unavailable or conflicting binding')
with Journal(bound_directory) as journal:
_assert_registry(bound_directory, plan)
_check_registration(journal, plan)
intent = journal.read('intent.json')
if intent is None and any(journal.read(name) is not None for name in ('before.json', 'verified.json')):
raise UncertainPublication('incomplete canonical journal; do not retry POST')
before = baseline
if intent is not None:
if type(intent) is not dict or intent.get('fingerprint') != fingerprint:
@@ -540,6 +831,9 @@ def publish(plan, baseline, api, lease, journal_directory, *, declaration=None):
'body_sha256': hashlib.sha256(payload['body'].encode()).hexdigest()})
check_lease(lease, plan)
require_adoption(declaration, plan['target']['head'])
journal.check_identity()
_assert_registry(bound_directory, plan)
_check_registration(journal, plan)
uncertain = False
response_id = None
try:
@@ -0,0 +1,380 @@
"""Canonical native-review publication journal registry."""
from __future__ import annotations
import errno
import fcntl
import hashlib
import json
import math
import os
import stat
import tempfile
from pathlib import Path
SCHEMA = "gitea-agent.native-review-binding/v1"
INTENT_SCHEMA = "gitea-agent.native-review-intent/v1"
ROOT_NAME = "native-review-journals"
_ROOT_PRESENCE_SCHEMA = "gitea-agent.native-review-root/v1"
_ENTRY_PRESENCE_SCHEMA = "gitea-agent.native-review-entry/v1"
_MAX_JSON = 16 * 1024 * 1024
def _fail(message):
raise ValueError(message)
def _canonical(value):
try:
return json.dumps(value, sort_keys=True, ensure_ascii=False,
allow_nan=False, separators=(",", ":"))
except (TypeError, ValueError, OverflowError):
_fail("non-canonical JSON value")
def _strict_load(data):
def pairs(pairs):
out = {}
for key, value in pairs:
if key in out:
_fail("duplicate JSON key")
out[key] = value
return out
try:
return json.loads(data.decode("utf-8"), object_pairs_hook=pairs,
parse_constant=lambda value: _fail("non-finite JSON"),
parse_float=lambda value: float(value) if math.isfinite(float(value)) else _fail("non-finite JSON"))
except ValueError:
raise
except Exception:
_fail("malformed journal artifact")
def binding_for(plan, *, origin):
t = plan["target"]
return {"schema": SCHEMA, "origin": origin, "owner": t["owner"],
"repository": t["repository"], "issue": t["issue"],
"pull": t["pull"], "head": t["head"],
"contract_sha256": plan["contract_sha256"],
"reviewer_id": plan["reviewer"]["id"]}
def plan_fingerprint(plan):
return hashlib.sha256(_canonical(plan).encode("utf-8")).hexdigest()
def root_for(anchor_file):
harness = Path(anchor_file).resolve().parents[5]
return harness / ".runtime" / ROOT_NAME
def canonical_path(plan, *, anchor_file, origin, validate):
validate(plan)
binding = binding_for(plan, origin=origin)
key = hashlib.sha256(_canonical(binding).encode("utf-8")).hexdigest()
return root_for(anchor_file) / "v1" / key
def _raw_path_safe(value):
if isinstance(value, str):
if not value.startswith("/"):
_fail("unsafe journal path")
parts = value.split("/")
if any(part in ("", ".", "..") for part in parts[1:]):
_fail("unsafe journal path")
def _check_ancestor_chain(path, *, allow_missing_leaf=False):
path = Path(path)
if not path.is_absolute() or any(part in ("", ".", "..") for part in path.parts):
_fail("unsafe journal path")
current = Path(path.anchor)
for index, part in enumerate(path.parts[1:]):
current /= part
try:
st = current.lstat()
except FileNotFoundError:
if allow_missing_leaf and index == len(path.parts) - 2:
return
_fail("missing trusted journal ancestor")
if stat.S_ISLNK(st.st_mode) or not stat.S_ISDIR(st.st_mode):
_fail("unsafe journal ancestor")
def _fsync_dir(path):
fd = os.open(path, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW | os.O_CLOEXEC)
try:
os.fsync(fd)
finally:
os.close(fd)
def _check_private_file(fd, *, what):
st = os.fstat(fd)
if (not stat.S_ISREG(st.st_mode) or st.st_uid != os.getuid()
or stat.S_IMODE(st.st_mode) != 0o600 or st.st_nlink != 1):
_fail("unsafe journal " + what)
return st
def _open_bootstrap(parent):
path = Path(parent) / (ROOT_NAME + ".bootstrap.lock")
try:
fd = os.open(path, os.O_RDWR | os.O_CREAT | os.O_NOFOLLOW | os.O_CLOEXEC, 0o600)
except OSError:
_fail("unsafe journal bootstrap lock")
try:
_check_private_file(fd, what="bootstrap lock")
try:
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
except (BlockingIOError, OSError) as exc:
if getattr(exc, "errno", None) in (errno.EACCES, errno.EAGAIN):
_fail("journal bootstrap busy")
raise
return fd
except Exception:
os.close(fd)
raise
def _presence_read(path, cap=4096):
try:
return _strict_load(_secure_file(path, cap=cap))
except FileNotFoundError:
return None
def _atomic_bytes(directory, name, data, *, replace=False):
directory = Path(directory)
_check_ancestor_chain(directory)
target = directory / name
try:
st = target.lstat()
if stat.S_ISLNK(st.st_mode) or not stat.S_ISREG(st.st_mode) or st.st_nlink != 1:
_fail("unsafe journal artifact")
if not replace:
_fail("journal artifact already exists")
except FileNotFoundError:
pass
fd, temp = tempfile.mkstemp(dir=directory, prefix=".native-review-")
try:
os.fchmod(fd, 0o600)
with os.fdopen(fd, "wb") as out:
out.write(data); out.flush(); os.fsync(out.fileno())
if replace:
os.replace(temp, target)
else:
try:
os.link(temp, target)
except FileExistsError:
_fail("journal artifact already exists")
os.unlink(temp)
_fsync_dir(directory)
finally:
try:
os.unlink(temp)
except FileNotFoundError:
pass
def atomic_json(directory, name, value):
_atomic_bytes(directory, name, _canonical(value).encode("utf-8"))
def ensure_root(root, *, _lock_fd=None):
root = Path(root)
parent = root.parent
_check_ancestor_chain(parent)
lock_fd = _lock_fd if _lock_fd is not None else _open_bootstrap(parent)
try:
root_presence = parent / (root.name + ".presence")
rp = _presence_read(root_presence)
root_exists = False
try:
rst = root.lstat(); root_exists = True
except FileNotFoundError:
rst = None
if rp is None:
if root_exists:
_fail("journal root presence anchor missing")
root.mkdir(mode=0o700)
rst = root.lstat()
atomic_json(parent, root_presence.name, {"schema": _ROOT_PRESENCE_SCHEMA,
"root": root.name + "/v1", "devino": [rst.st_dev, rst.st_ino]})
else:
if set(rp) != {"schema", "root", "devino"} or rp["schema"] != _ROOT_PRESENCE_SCHEMA:
_fail("malformed journal root presence")
if not root_exists or stat.S_ISLNK(rst.st_mode) or not stat.S_ISDIR(rst.st_mode):
_fail("journal root was deleted")
if [rst.st_dev, rst.st_ino] != rp["devino"]:
_fail("journal root was replaced")
rst = root.lstat()
if (stat.S_ISLNK(rst.st_mode) or not stat.S_ISDIR(rst.st_mode)
or rst.st_uid != os.getuid() or stat.S_IMODE(rst.st_mode) != 0o700):
_fail("unsafe journal root")
version = root / "v1"
try: vst = version.lstat()
except FileNotFoundError:
version.mkdir(mode=0o700); _fsync_dir(root); vst = version.lstat()
if (stat.S_ISLNK(vst.st_mode) or not stat.S_ISDIR(vst.st_mode)
or vst.st_uid != os.getuid() or stat.S_IMODE(vst.st_mode) != 0o700):
_fail("unsafe journal registry")
return root
finally:
if _lock_fd is None:
os.close(lock_fd)
def _secure_file(path, *, cap=_MAX_JSON):
_raw_path_safe(os.fspath(path))
fd = os.open(path, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK | os.O_CLOEXEC)
try:
st = _check_private_file(fd, what="artifact")
if st.st_size > cap:
_fail("oversize journal artifact")
chunks = []; total = 0
while True:
chunk = os.read(fd, min(65536, cap + 1 - total))
if not chunk: break
chunks.append(chunk); total += len(chunk)
if total > cap: _fail("oversize journal artifact")
return b"".join(chunks)
finally:
os.close(fd)
def _entry_check(entry):
st = Path(entry).lstat()
if (stat.S_ISLNK(st.st_mode) or not stat.S_ISDIR(st.st_mode)
or st.st_uid != os.getuid() or stat.S_IMODE(st.st_mode) != 0o700):
_fail("unsafe journal binding")
return st
def _presence_path(entry):
entry = Path(entry)
return entry.parents[1].parent / (entry.name + ".presence")
def open_binding(path, plan, *, anchor_file, origin, validate, require_new=False):
_raw_path_safe(os.fspath(path))
validate(plan)
expected = canonical_path(plan, anchor_file=anchor_file, origin=origin, validate=validate)
if Path(path) != expected or (isinstance(path, str) and path != str(expected)):
_fail("non-canonical journal path")
root = expected.parents[1]
_check_ancestor_chain(root.parent, allow_missing_leaf=True)
try:
root.parent.mkdir(mode=0o700)
_fsync_dir(root.parent.parent)
except FileExistsError:
pass
_entry_check(root.parent)
bootstrap = _open_bootstrap(root.parent)
try:
ensure_root(root, _lock_fd=bootstrap)
entry = expected
presence = _presence_path(entry)
ep = _presence_read(presence)
exists = entry.exists()
if require_new and (exists or ep is not None):
_fail("canonical journal entry already exists")
if ep is not None:
if set(ep) != {"schema", "entry", "devino", "fingerprint", "state"} or ep["schema"] != _ENTRY_PRESENCE_SCHEMA:
_fail("malformed journal entry presence")
if ep["state"] != "complete" or not exists:
_fail("interrupted or deleted journal entry")
st = _entry_check(entry)
if [st.st_dev, st.st_ino] != ep["devino"]:
_fail("journal entry was replaced")
elif exists:
_fail("journal entry presence anchor missing")
else:
entry.mkdir(mode=0o700); _fsync_dir(root)
if require_new:
atomic_json(entry, "import.pending.json", {
"schema": "gitea-agent.native-review-import/v1",
"state": "pending"})
st = _entry_check(entry)
pending = entry / "import.pending.json"
if not require_new and pending.exists():
_fail("interrupted legacy registration")
binding = binding_for(plan, origin=origin)
expected_record = {"schema": SCHEMA, "binding": binding, "fingerprint": plan_fingerprint(plan)}
record_path = entry / "binding.json"
try:
existing = _strict_load(_secure_file(record_path, cap=1024 * 1024))
except FileNotFoundError:
existing = None
if existing is not None and _canonical(existing) != _canonical(expected_record):
_fail("binding record mismatch")
if existing is None:
if ep is not None:
_fail("known journal binding record missing")
if require_new is False and record_path.exists() and not record_path.is_file():
_fail("unsafe journal binding record")
atomic_json(entry, "binding.json", expected_record)
if ep is None:
_atomic_bytes(presence.parent, presence.name, _canonical({
"schema": _ENTRY_PRESENCE_SCHEMA, "entry": entry.name,
"devino": [st.st_dev, st.st_ino], "fingerprint": expected_record["fingerprint"],
"state": "complete"}).encode())
else:
if ep["fingerprint"] != expected_record["fingerprint"]:
_fail("entry binding fingerprint mismatch")
return entry
finally:
os.close(bootstrap)
def assert_bound(entry, plan, *, anchor_file, origin, validate):
validate(plan)
entry = Path(entry)
expected = canonical_path(plan, anchor_file=anchor_file, origin=origin, validate=validate)
if entry != expected:
_fail("non-canonical journal path")
root = expected.parents[1]
_check_ancestor_chain(entry)
_entry_check(root.parent); rst = _entry_check(root)
_entry_check(root / 'v1'); st = _entry_check(entry)
rp = _presence_read(root.parent / (root.name + ".presence"))
expected_rp = {"schema": _ROOT_PRESENCE_SCHEMA, "root": root.name + "/v1",
"devino": [rst.st_dev, rst.st_ino]}
if _canonical(rp) != _canonical(expected_rp):
_fail("journal root presence mismatch")
expected_record = {"schema": SCHEMA, "binding": binding_for(plan, origin=origin), "fingerprint": plan_fingerprint(plan)}
ep = _presence_read(_presence_path(entry))
expected_ep = {"schema": _ENTRY_PRESENCE_SCHEMA, "entry": entry.name,
"devino": [st.st_dev, st.st_ino], "fingerprint": expected_record["fingerprint"],
"state": "complete"}
if _canonical(ep) != _canonical(expected_ep):
_fail("journal entry presence mismatch")
record = _strict_load(_secure_file(entry / "binding.json", cap=1024 * 1024))
if _canonical(record) != _canonical(expected_record):
_fail("binding record mismatch")
return entry
def acquire_lock(entry):
path = Path(entry) / "publication.lock"
fd = None
try:
fd = os.open(path, os.O_RDWR | os.O_CREAT | os.O_NOFOLLOW | os.O_NONBLOCK | os.O_CLOEXEC, 0o600)
_check_private_file(fd, what="lock")
try:
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
except OSError as exc:
if exc.errno in (errno.EACCES, errno.EAGAIN):
_fail("journal lock busy")
raise
return fd
except BaseException:
if fd is not None:
os.close(fd)
raise
def read_json(entry, name, cap=_MAX_JSON):
try:
return _strict_load(_secure_file(Path(entry) / name, cap=cap))
except FileNotFoundError:
return None
+9
View File
@@ -569,3 +569,12 @@ Command deadline 계약: 새 plan의 각 command는 `timeout_seconds`를 명시
`evidence-manifest/v3`는 실제 timeout, `execution_status`, `termination`의 `term_sent`·`kill_sent`·`direct_child_reaped`·`group_cleanup_completed` boolean 기록과 안전한 bounded 부분 출력을 결합한다. 새 claim pointer는 기존 6개에 timeout·실행 상태·종료 기록을 추가한 9개다. Linux `/proc`로 같은 group의 non-zombie 프로세스를 확인하며 확인·정리에 실패하면 `cleanup_failed`로 차단한다. TERM grace는 0.25 초, KILL 후 group 종료·direct-child 회수 예산은 0.5 초이며 오류 fallback wait도 0.1 초로 제한한다. 새 command의 `max_output_bytes`는 0–1048576 범위의 non-boolean 정수만 허용한다. `execution_status`는 `completed|timeout|output_limit|cleanup_failed|descendant_cleanup`이다. 정상 종료 leader 뒤의 살아 있는 descendant를 신호로 정리한 경우도 `descendant_cleanup` 실패이며, `completed`에는 TERM/KILL 신호가 있을 수 없다. Timeout·출력 한도 초과·정리 실패는 expected exit code와 같아도 성공이 아니며 **TDD RED** 또는 제품 assertion 실패로 분류하지 않는다. 해당 실행은 실패로 보존하고 뒤의 command를 실행하지 않으며, 실행된 prefix의 claim만 기록한다. Partial output에도 기존 secret 탐지를 적용하고 인증정보 유출은 fail-closed한다. 자동 재실행은 하지 않으며 새로 승인된 실행에는 새 run-id를 사용한다.
기존 `evidence-manifest/v2`는 명시적 `validate-archived-v2 --contract-path <canonical-contract> --repo-root <checkout> --artifact-root <canonical-run> --manifest <run-relative-manifest>`로 read-only 호환 검증한다. Canonical identity, exact current HEAD, source/plan/output binding을 유지하지만 과거 기록에 없던 timeout이나 종료 보장을 추론하지 않는다. 새 `run`/`validate`에는 v2 fallback이 없다. v1은 기존 `validate-legacy`만 사용한다. 두 archive 경로의 성공은 새 실행·TDD RED·fresh publication proof가 아니며 기존 artifact를 수정하거나 새 schema로 자동 변환하지 않는다.
## Canonical native-review journal
Native publication의 retained root는 `<harness-root>/harness/.runtime/native-review-journals`이며 repository cache/worktree 및 caller의 run directory와 별개의 binding registry다. `canonical_journal_path(plan)` 계산은 권한 증명이 아니다. Authenticated origin/repository/Issue/PR/head/contract/reviewer binding, full-plan 충돌, durable presence anchors, mode0700/0600 및 nonblocking mutex 정책을 적용한다. Alternate path와 missing/interrupted 상태를 자동 repair하거나 새 POST의 근거로 사용하지 않는다.
기존 intent의 재개는 read-only reconciliation뿐이며 legacy는 사전 digest·현재 role/lease/adoption 검증 후 명시적 `register_legacy_journal`로만 등록한다. Import pending receipt를 복사 전에 fsync하고 완료 receipt 전 게시를 차단한다. 원본 journal을 자동 이동·삭제하지 않는다. 외부 write와 canonical Issue lease의 acquire/renew/final verification/release/cleanup 계약은 그대로 유지한다. Same-install cooperative filesystem만 보호하며 cross-host global exactly-once를 주장하지 않는다.
구체적인 registry/legacy/retention 계약은 [native review reference](../agents/review/skills/gitea-code-review/references/native-review.md#canonical-native-review-journal)를 정본으로 따른다. `.gitignore`의 `/harness/.runtime/` 보호를 유지하며 journal을 Git commit·source evidence bundle에 포함하지 않는다.
@@ -93,6 +93,11 @@ class IntegrationTests(unittest.TestCase):
SCRATCH.mkdir(parents=True, exist_ok=True)
self.tmp = tempfile.TemporaryDirectory(dir=SCRATCH)
self.ev = RealEvidence(Path(self.tmp.name))
from unittest.mock import patch as mock_patch
self.registry_patch = mock_patch.object(native._journal, 'root_for',
return_value=Path(self.tmp.name) / 'native-review-journals')
self.registry_patch.start()
self.addCleanup(self.registry_patch.stop)
def tearDown(self): self.tmp.cleanup()
@@ -160,7 +165,7 @@ class IntegrationTests(unittest.TestCase):
self.ev.set_stage('code-review')
fixture, plan, before, after, exact = self.bound_publication_fixture()
api = fixture.FakeAPI(plan, before, after, exact)
with tempfile.TemporaryDirectory(dir=self.tmp.name) as journal:
journal = native.canonical_journal_path(plan)
first = native.publish(plan, before, api, fixture.FakeLease(), journal, declaration=self.ev.declaration())
second = native.publish(plan, before, api, fixture.FakeLease(), journal, declaration=self.ev.declaration())
self.assertEqual(first['mode'], 'created')
@@ -180,7 +185,7 @@ class IntegrationTests(unittest.TestCase):
self.ev.bundle.joinpath('src/a.py').write_text('VALUE = 9\n')
return out
api.snapshot = mutating_snapshot
with tempfile.TemporaryDirectory(dir=self.tmp.name) as journal:
journal = native.canonical_journal_path(plan)
with self.assertRaisesRegex(native.ValidationError, 'delegation adoption'):
native.publish(plan, before, api, fixture.FakeLease(), journal, declaration=self.ev.declaration())
self.assertEqual(calls['n'], 2, 'must reach both provider reads and final adoption check')
@@ -241,7 +246,7 @@ class IntegrationTests(unittest.TestCase):
self.ev.set_stage('code-review')
fixture, plan, before, after, exact = self.bound_publication_fixture()
api = fixture.FakeAPI(plan, before, after, exact)
journal = Path(self.tmp.name) / ('denied-journal-' + mode)
journal = native.canonical_journal_path(plan)
with self.subTest(mode=mode, consumer='review'):
with self.assertRaises(native.ValidationError):
native.publish(plan, before, api, fixture.FakeLease(), journal, declaration=declaration)
@@ -0,0 +1,126 @@
import hashlib
import importlib.util
import json
import stat
import tempfile
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[3]
SCRIPT = ROOT / 'harness/agents/review/skills/gitea-code-review/scripts/native_review.py'
FIXTURE = ROOT / 'harness/tests/review/test_native_review.py'
def load(path, name):
spec = importlib.util.spec_from_file_location(name, path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
class LegacyJournalContractTests(unittest.TestCase):
def test_legacy_intent_extra_fields_are_not_registered(self):
m = load(SCRIPT, 'legacy_contract_publisher')
f = load(FIXTURE, 'legacy_contract_fixture')
plan, before, after, exact = f.publication_fixture()
with tempfile.TemporaryDirectory(dir='/opt/data/cache/scratch') as td:
m._journal.root_for = lambda anchor: Path(td) / 'native-review-journals'
source = Path(td) / 'legacy'
payload = m.build_payload(plan)
intent = {'schema': 'gitea-agent.native-review-intent/v1',
'fingerprint': hashlib.sha256(m.canonical(plan)).hexdigest(),
'before_sha256': hashlib.sha256(m.canonical(before)).hexdigest(),
'head': plan['target']['head'], 'event': payload['event'],
'body_sha256': hashlib.sha256(payload['body'].encode()).hexdigest(),
'unexpected': 'cannot silently import this'}
with m.Journal(source) as journal:
journal.write('before.json', before)
journal.write('intent.json', intent)
digest = m.legacy_journal_digest(source)
api = f.FakeAPI(plan, before, after, exact)
with self.assertRaises(m.ValidationError):
m.register_legacy_journal(plan, source, expected_legacy_sha256=digest,
api=api, lease=f.FakeLease(), declaration={'mode': 'none'})
self.assertEqual(api.posts, 0)
self.assertFalse((m.canonical_journal_path(plan) / 'intent.json').exists())
def test_legacy_inventory_rejects_unknown_artifact_before_import(self):
m = load(SCRIPT, 'legacy_inventory_contract')
f = load(FIXTURE, 'legacy_inventory_fixture')
plan, before, after, exact = f.publication_fixture()
with tempfile.TemporaryDirectory(dir='/opt/data/cache/scratch') as td:
m._journal.root_for = lambda anchor: Path(td) / 'native-review-journals'
source = Path(td) / 'legacy'; source.mkdir(mode=0o700)
with m.Journal(source) as journal:
journal.write('before.json', before)
journal.write('intent.json', m._expected_intent(plan, before))
extra = source / 'unknown.json'; extra.write_bytes(b'{"unknown":true}')
extra.chmod(0o600)
with self.assertRaises(m.ValidationError):
m.legacy_journal_digest(source)
def test_import_preserves_raw_bytes_and_archives_verified_separately(self):
m = load(SCRIPT, 'legacy_raw_import_contract')
f = load(FIXTURE, 'legacy_raw_import_fixture')
plan, before, after, exact = f.publication_fixture()
with tempfile.TemporaryDirectory(dir='/opt/data/cache/scratch') as td:
m._journal.root_for = lambda anchor: Path(td) / 'native-review-journals'
source = Path(td) / 'legacy'; source.mkdir(mode=0o700)
intent = m._expected_intent(plan, before)
before_raw = (json.dumps(before, sort_keys=True, indent=1) + '\n').encode()
intent_raw = (json.dumps(intent, sort_keys=True, indent=1) + '\n').encode()
verified_raw = b'{\n "verified": true,\n "legacy": "exact"\n}\n'
for name, raw in [('before.json', before_raw), ('intent.json', intent_raw),
('verified.json', verified_raw)]:
path = source / name; path.write_bytes(raw); path.chmod(0o600)
with m.Journal(source):
pass
original = {name: (source / name).read_bytes() for name in
['before.json', 'intent.json', 'verified.json']}
digest = m.legacy_journal_digest(source)
result = m.register_legacy_journal(
plan, source, expected_legacy_sha256=digest,
api=f.FakeAPI(plan, before, after, exact), lease=f.FakeLease(),
declaration={'mode': 'none'})
self.assertTrue(result['read_only'])
target = m.canonical_journal_path(plan)
self.assertEqual((target / 'before.json').read_bytes(), original['before.json'])
self.assertEqual((target / 'intent.json').read_bytes(), original['intent.json'])
self.assertEqual((target / 'legacy-verified.json').read_bytes(), original['verified.json'])
self.assertFalse((target / 'verified.json').exists())
self.assertEqual(stat.S_IMODE((target / 'legacy-verified.json').stat().st_mode), 0o600)
api = f.FakeAPI(plan, before, after, exact); api.created = True
for _ in range(2):
result = m.publish(plan, before, api, f.FakeLease(), declaration={'mode': 'none'})
self.assertFalse(result['wrote'])
self.assertEqual(api.posts, 0)
self.assertTrue((target / 'verified.json').exists())
self.assertEqual((target / 'legacy-verified.json').read_bytes(), verified_raw)
(target / 'legacy-verified.json').write_bytes(b'{"verified": false}')
with self.assertRaisesRegex(m.ValidationError, 'integrity mismatch'):
m.publish(plan, before, api, f.FakeLease(), declaration={'mode': 'none'})
self.assertEqual(api.posts, 0)
def test_receipt_cannot_omit_imported_verified_archive(self):
m = load(SCRIPT, 'legacy_receipt_contract'); f = load(FIXTURE, 'legacy_receipt_fixture')
plan, before, after, exact = f.publication_fixture()
with tempfile.TemporaryDirectory(dir='/opt/data/cache/scratch') as td:
m._journal.root_for = lambda anchor: Path(td) / 'native-review-journals'
source = Path(td) / 'legacy'
with m.Journal(source) as journal:
journal.write('before.json', before)
journal.write('intent.json', m._expected_intent(plan, before))
journal.write('verified.json', {'verified': True})
m.register_legacy_journal(plan, source, expected_legacy_sha256=m.legacy_journal_digest(source),
api=f.FakeAPI(plan, before, after, exact), lease=f.FakeLease(), declaration={'mode': 'none'})
target = m.canonical_journal_path(plan)
receipt = json.loads((target / 'registration.json').read_text())
receipt['imported_files'].pop()
with m.Journal(target) as journal:
journal.write('registration.json', receipt)
with self.assertRaises(m.ValidationError): m._check_registration(journal, plan)
if __name__ == '__main__':
unittest.main()
@@ -0,0 +1,29 @@
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[3]
REFERENCE = ROOT / 'harness/agents/review/skills/gitea-code-review/references/native-review.md'
SKILL = ROOT / 'harness/agents/review/skills/gitea-code-review/SKILL.md'
CACHE = ROOT / 'harness/operations/repository-cache.md'
class NativeJournalDocumentationTests(unittest.TestCase):
def test_canonical_registry_and_legacy_policy_are_explicit(self):
for path in (REFERENCE, SKILL, CACHE):
text = path.read_text()
for token in ('harness/.runtime/native-review-journals',
'canonical_journal_path', 'register_legacy_journal',
'binding', 'read-only', 'cross-host'):
with self.subTest(path=str(path), token=token):
self.assertIn(token, text)
def test_no_caller_selected_journal_in_live_example(self):
text = REFERENCE.read_text()
self.assertNotIn('result = native_review.publish(plan, baseline, api, lease, journal_directory)', text)
self.assertIn('result = native_review.publish(plan, baseline, api, lease)', text)
self.assertIn('기존 journal을 자동 이동·삭제', text)
self.assertIn('pending', text)
if __name__ == '__main__':
unittest.main()
+36 -24
View File
@@ -121,7 +121,18 @@ class FakeLease:
class NativeReviewTests(unittest.TestCase):
def module(self):
self.assertTrue(SCRIPT.is_file(), 'shared native review adapter is missing')
return load(SCRIPT, 'native_review_test')
m = load(SCRIPT, 'native_review_test_' + str(id(self)))
# Every test gets a private registry root; production has no test-only
# root override, so isolate through the journal module seam.
private_root = tempfile.TemporaryDirectory()
self.addCleanup(private_root.cleanup)
root = Path(private_root.name) / 'native-review-journals'
m._journal.root_for = lambda anchor, root=root: root
return m
@staticmethod
def canonical(m, plan):
return m.canonical_journal_path(plan)
def test_payload_is_exact_bound_native_create(self):
m = self.module()
@@ -159,22 +170,23 @@ class NativeReviewTests(unittest.TestCase):
p, before, after, exact = publication_fixture()
api = FakeAPI(p, before, after, exact); lease = FakeLease()
with tempfile.TemporaryDirectory() as td:
result = m.publish(p, before, api, lease, td, declaration={'mode': 'none'})
result = m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'})
self.assertEqual(result['mode'], 'created')
self.assertEqual(api.posts, 1)
self.assertEqual(m.publish(p, before, api, lease, td, declaration={'mode': 'none'})['mode'], 'no_op')
self.assertEqual(m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'})['mode'], 'no_op')
self.assertEqual(api.posts, 1)
self.assertNotIn('body', json.loads((Path(td) / 'intent.json').read_text()))
self.assertNotIn('body', json.loads((self.canonical(m, p) / 'intent.json').read_text()))
for applies in [True, False]:
m = self.module(); p, before, after, exact = publication_fixture()
api = FakeAPI(p, before, after, exact); api.timeout = True; api.apply = applies
with self.subTest(applies=applies), tempfile.TemporaryDirectory() as td:
if applies:
self.assertEqual(m.publish(p, before, api, lease, td, declaration={'mode': 'none'})['mode'], 'reconciled')
self.assertEqual(m.publish(p, before, api, lease, td, declaration={'mode': 'none'})['mode'], 'no_op')
self.assertEqual(m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'})['mode'], 'reconciled')
self.assertEqual(m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'})['mode'], 'no_op')
else:
for _ in range(2):
with self.assertRaises(m.UncertainPublication) as error:
m.publish(p, before, api, lease, td, declaration={'mode': 'none'})
m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'})
self.assertNotIn('secret transport detail', str(error.exception))
self.assertEqual(api.posts, 1)
def test_api_adapter_checks_identity_first_and_collects_all_pages(self):
@@ -333,9 +345,9 @@ class NativeReviewTests(unittest.TestCase):
p, before, after, exact = publication_fixture(); api = FakeAPI(p, before, after, exact); lease = FakeLease()
if kind == 'expired': lease.valid = False
else: api.identity = lambda: p['spec_reviewer']
with self.assertRaises(m.ValidationError): m.publish(p, before, api, lease, td, declaration={'mode': 'none'})
with self.assertRaises(m.ValidationError): m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'})
self.assertEqual(api.posts, 0)
self.assertFalse((Path(td) / 'intent.json').exists())
self.assertFalse((self.canonical(m, p) / 'intent.json').exists())
def test_fresh_head_contract_and_source_linkage_drift_prevent_post(self):
m = self.module()
@@ -348,7 +360,7 @@ class NativeReviewTests(unittest.TestCase):
with self.subTest(case=i), tempfile.TemporaryDirectory() as td:
p, before, after, exact = publication_fixture(); api = FakeAPI(p, copy.deepcopy(before), after, exact)
change(api.before)
with self.assertRaises(m.ValidationError): m.publish(p, before, api, FakeLease(), td, declaration={'mode': 'none'})
with self.assertRaises(m.ValidationError): m.publish(p, before, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'})
self.assertEqual(api.posts, 0)
def test_pagination_total_duplicate_and_truncation_are_hard_errors(self):
@@ -424,10 +436,10 @@ class NativeReviewTests(unittest.TestCase):
m = self.module(); p, before, after, exact = publication_fixture()
api = FakeAPI(p, before, after, exact); api.created = True
with tempfile.TemporaryDirectory() as td:
result = m.publish(p, after, api, FakeLease(), td, declaration={'mode': 'none'})
result = m.publish(p, after, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'})
self.assertEqual(result['mode'], 'no_op'); self.assertFalse(result['wrote'])
self.assertEqual(api.posts, 0)
self.assertFalse((Path(td) / 'intent.json').exists())
self.assertFalse((self.canonical(m, p) / 'intent.json').exists())
def test_duplicate_or_conflicting_canonical_review_is_not_reposted(self):
m = self.module()
@@ -441,7 +453,7 @@ class NativeReviewTests(unittest.TestCase):
after['pagination']['reviews'][0].update(count=4, declared_total=4)
after['pagination']['review_comments_11'] = [{'page': 1, 'count': 0, 'declared_total': 0, 'link': None}]
api = FakeAPI(p, before, after, exact); api.created = True
with self.assertRaises(m.ValidationError): m.publish(p, before, api, FakeLease(), td, declaration={'mode': 'none'})
with self.assertRaises(m.ValidationError): m.publish(p, before, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'})
self.assertEqual(api.posts, 0)
def test_readback_failure_and_changed_journal_plan_never_retry_post(self):
@@ -449,7 +461,7 @@ class NativeReviewTests(unittest.TestCase):
api.read_review = lambda target, rid: (_ for _ in ()).throw(TimeoutError('DO_NOT_ECHO_TRANSPORT_SECRET'))
with tempfile.TemporaryDirectory() as td:
for _ in range(2):
with self.assertRaises(Exception) as raised: m.publish(p, before, api, FakeLease(), td, declaration={'mode': 'none'})
with self.assertRaises(Exception) as raised: m.publish(p, before, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'})
self.assertNotIn('DO_NOT_ECHO_TRANSPORT_SECRET', str(raised.exception))
self.assertEqual(api.posts, 1)
different = copy.deepcopy(p); different['body'] += 'different'
@@ -462,15 +474,15 @@ class NativeReviewTests(unittest.TestCase):
root = Path(td); link = root / 'link'; link.symlink_to(root, target_is_directory=True)
with self.assertRaises(m.ValidationError): m.Journal(link)
api = FakeAPI(p, before, after, exact); api.timeout = True; api.apply = False
journal = root / 'journal'
journal = self.canonical(m, p)
with self.assertRaises(m.UncertainPublication): m.publish(p, before, api, FakeLease(), journal, declaration={'mode': 'none'})
saved = json.loads((journal / 'before.json').read_text()); saved['issue']['body'] = 'changed'
(journal / 'before.json').write_text(json.dumps(saved))
with self.assertRaises(m.ValidationError): m.publish(p, before, api, FakeLease(), journal, declaration={'mode': 'none'})
self.assertEqual(api.posts, 1)
with m.Journal(root / 'hardlinks') as j:
j.write('a.json', {'safe': True}); os.link(j.path / 'a.json', j.path / 'b.json')
with self.assertRaises(m.ValidationError): j.read('a.json')
j.write('before.json', {'safe': True}); os.link(j.path / 'before.json', j.path / 'b.json')
with self.assertRaises(m.ValidationError): j.read('before.json')
def test_transport_origin_credential_source_and_exception_text_are_guarded(self):
m = self.module()
@@ -485,7 +497,7 @@ class NativeReviewTests(unittest.TestCase):
m = self.module(); p, before, after, exact = publication_fixture(); api = FakeAPI(p, before, after, exact)
lease = FakeLease(); lease.check = lambda plan: False
with tempfile.TemporaryDirectory() as td:
with self.assertRaises(m.ValidationError): m.publish(p, before, api, lease, td, declaration={'mode': 'none'})
with self.assertRaises(m.ValidationError): m.publish(p, before, api, lease, self.canonical(m, p), declaration={'mode': 'none'})
self.assertEqual(api.posts, 0)
def test_live_rest_bridge_over_loopback_publishes_once_then_no_ops(self):
import threading
@@ -533,10 +545,10 @@ class NativeReviewTests(unittest.TestCase):
api = m.ApiAdapter(request, origin=m.ORIGIN + '/api/v1', credential_source='$HERMES_HOME/.env:GITEA_TOKEN')
baseline = api.snapshot(p['target'])
with tempfile.TemporaryDirectory() as td:
result = m.publish(p, baseline, api, FakeLease(), td, declaration={'mode': 'none'})
result = m.publish(p, baseline, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'})
self.assertEqual(result['mode'], 'created'); self.assertEqual(result['projected_review_ids'], [8])
self.assertTrue(result['commit_file_order_normalized'])
second = m.publish(p, baseline, api, FakeLease(), td, declaration={'mode': 'none'})
second = m.publish(p, baseline, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'})
self.assertEqual(second['mode'], 'no_op'); self.assertFalse(second['wrote'])
self.assertEqual(state['posts'], 1); self.assertEqual(server.received, m.build_payload(p))
finally:
@@ -586,15 +598,15 @@ class NativeReviewTests(unittest.TestCase):
source.assert_not_called()
def test_journal_intent_fields_are_validated_on_resume(self):
m = self.module()
for field in ['schema', 'head', 'event', 'body_sha256']:
m = self.module()
with self.subTest(field=field), tempfile.TemporaryDirectory() as td:
p, before, after, exact = publication_fixture(); api = FakeAPI(p, before, after, exact)
api.timeout = True; api.apply = False
with self.assertRaises(m.UncertainPublication): m.publish(p, before, api, FakeLease(), td, declaration={'mode': 'none'})
path = Path(td) / 'intent.json'; intent = json.loads(path.read_text()); intent[field] = 'corrupt'
with self.assertRaises(m.ValidationError): m.publish(p, before, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'})
path = self.canonical(m, p) / 'intent.json'; intent = json.loads(path.read_text()); intent[field] = 'corrupt'
path.write_text(json.dumps(intent)); api.created = True
with self.assertRaises(m.ValidationError): m.publish(p, before, api, FakeLease(), td, declaration={'mode': 'none'})
with self.assertRaises(m.ValidationError): m.publish(p, before, api, FakeLease(), self.canonical(m, p), declaration={'mode': 'none'})
self.assertEqual(api.posts, 1)
def test_issue_repository_mismatch_is_not_accepted(self):
@@ -0,0 +1,89 @@
import copy
import importlib.util
import os
import tempfile
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[3]
SCRIPT = ROOT / 'harness/agents/review/skills/gitea-code-review/scripts/native_review.py'
FIXTURES = ROOT / 'harness/tests/review/test_native_review.py'
def load(path, name):
spec = importlib.util.spec_from_file_location(name, path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
class CanonicalJournalTests(unittest.TestCase):
def test_binding_path_is_stable_and_head_scoped(self):
m = load(SCRIPT, 'native_review_canonical_test')
fixtures = load(FIXTURES, 'native_review_fixture_test')
plan = fixtures.plan_fixture()
first = m.canonical_journal_path(plan)
again = m.canonical_journal_path(copy.deepcopy(plan))
self.assertEqual(first, again)
changed = copy.deepcopy(plan)
changed['target']['head'] = 'b' * 40
changed['body'] = changed['body'].replace('head=' + plan['target']['head'], 'head=' + changed['target']['head'])
self.assertNotEqual(first, m.canonical_journal_path(changed))
changed = copy.deepcopy(plan)
changed['reviewer']['id'] = 43
self.assertNotEqual(first, m.canonical_journal_path(changed))
self.assertEqual(first.parts[-2], 'v1')
self.assertEqual(first.parts[-3], 'native-review-journals')
def test_missing_entry_with_presence_anchor_fails_closed(self):
m = load(SCRIPT, 'native_review_anchor_test')
fixtures = load(FIXTURES, 'native_review_anchor_fixture_test')
plan = fixtures.plan_fixture()
with tempfile.TemporaryDirectory() as td:
root = Path(td) / 'native-review-journals'
old = m._journal.root_for
m._journal.root_for = lambda anchor: root
try:
path = m.canonical_journal_path(plan)
entry = m._journal.open_binding(path, plan, anchor_file=str(SCRIPT),
origin=m.ORIGIN, validate=m.validate_plan)
self.assertTrue((entry / 'binding.json').exists())
import shutil
shutil.rmtree(entry)
with self.assertRaises(ValueError):
m._journal.open_binding(path, plan, anchor_file=str(SCRIPT),
origin=m.ORIGIN, validate=m.validate_plan)
finally:
m._journal.root_for = old
def test_deleted_root_with_persistent_anchor_fails_closed(self):
m = load(SCRIPT, 'native_review_root_loss_test')
fixtures = load(FIXTURES, 'native_review_root_loss_fixture_test')
plan = fixtures.plan_fixture()
with tempfile.TemporaryDirectory() as td:
root = Path(td) / 'native-review-journals'
old = m._journal.root_for
m._journal.root_for = lambda anchor: root
try:
path = m.canonical_journal_path(plan)
m._journal.open_binding(path, plan, anchor_file=str(SCRIPT),
origin=m.ORIGIN, validate=m.validate_plan)
import shutil
shutil.rmtree(root)
with self.assertRaises(ValueError):
m._journal.open_binding(path, plan, anchor_file=str(SCRIPT),
origin=m.ORIGIN, validate=m.validate_plan)
finally:
m._journal.root_for = old
def test_alternate_path_is_rejected_before_publication(self):
m = load(SCRIPT, 'native_review_alternate_test')
fixtures = load(FIXTURES, 'native_review_alternate_fixture_test')
plan, before, after, exact = fixtures.publication_fixture()
with tempfile.TemporaryDirectory() as td:
with self.assertRaises(m.ValidationError):
m.publish(plan, before, fixtures.FakeAPI(plan, before, after, exact),
fixtures.FakeLease(), td, declaration={'mode': 'none'})
if __name__ == '__main__':
unittest.main()
@@ -0,0 +1,277 @@
import copy
import hashlib
import importlib.util
import json
import os
import stat
import subprocess
import sys
import tempfile
import textwrap
import time
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[3]
SCRIPT = ROOT / 'harness/agents/review/skills/gitea-code-review/scripts/native_review.py'
FIXTURES = ROOT / 'harness/tests/review/test_native_review.py'
def load(path, name):
spec = importlib.util.spec_from_file_location(name, path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
class PublicationJournalProcessTests(unittest.TestCase):
def setUp(self):
self.m = load(SCRIPT, 'native_review_process_' + str(id(self)))
self.fixtures = load(FIXTURES, 'native_review_process_fixture_' + str(id(self)))
self.tmp = tempfile.TemporaryDirectory()
self.addCleanup(self.tmp.cleanup)
self.root = Path(self.tmp.name) / 'native-review-journals'
self.m._journal.root_for = lambda anchor, root=self.root: root
def plan_data(self):
p, before, after, exact = self.fixtures.publication_fixture()
return p, before, after, exact
def declaration(self):
return {'mode': 'none'}
def run_child(self, script, *args, timeout=8):
env = {'PATH': os.environ['PATH'], 'PYTHONPATH': str(ROOT), 'PYTHONDONTWRITEBYTECODE': '1'}
return subprocess.run([sys.executable, '-c', script, *map(str, args)], env=env,
capture_output=True, text=True, timeout=timeout)
def child_prelude(self):
return textwrap.dedent(f'''
import copy, importlib.util, json, os, sys, time
from pathlib import Path
root = Path(sys.argv[1]); payload = json.loads(Path(sys.argv[2]).read_text())
script = Path({str(SCRIPT)!r}); fixtures_path = Path({str(FIXTURES)!r})
def load(path, name):
spec = importlib.util.spec_from_file_location(name, path)
m = importlib.util.module_from_spec(spec); spec.loader.exec_module(m); return m
m = load(script, 'native_child_' + str(os.getpid()))
f = load(fixtures_path, 'fixture_child_' + str(os.getpid()))
m._journal.root_for = lambda anchor: root
p, before, after, exact = f.publication_fixture()
class Lease:
def check(self, plan): return None
class API:
def __init__(self): self.created = Path(sys.argv[3]).exists()
def identity(self): return copy.deepcopy(p['reviewer'])
def snapshot(self, target): return copy.deepcopy(after if self.created else before)
def read_review(self, target, rid): return copy.deepcopy(exact)
def create_review(self, target, body):
counter = Path(sys.argv[3]); n = int(counter.read_text() or '0') if counter.exists() else 0
counter.write_text(str(n + 1))
with counter.open('r+') as fd:
fd.flush(); os.fsync(fd.fileno())
self.created = True
return copy.deepcopy(exact)
api = API()
''')
def test_pending_intent_survives_process_crash_and_blocks_second_post(self):
p, before, after, exact = self.plan_data()
plan_file = Path(self.tmp.name) / 'plan.json'; plan_file.write_text(json.dumps(p, ensure_ascii=False))
counter = Path(self.tmp.name) / 'posts'
crash = self.child_prelude() + textwrap.dedent('''
class CrashAPI(API):
def create_review(self, target, body):
Path(sys.argv[4]).write_text('create-entered')
os._exit(17)
m.publish(p, before, CrashAPI(), Lease(), m.canonical_journal_path(p), declaration={'mode': 'none'})
''')
entered = Path(self.tmp.name) / 'entered'
crashed = self.run_child(crash, self.root, plan_file, counter, entered)
self.assertEqual(crashed.returncode, 17)
self.assertEqual(entered.read_text(), 'create-entered')
resume = self.child_prelude() + textwrap.dedent('''
try:
m.publish(p, before, API(), Lease(), m.canonical_journal_path(p), declaration={'mode': 'none'})
except Exception as exc:
print(type(exc).__name__)
''')
resumed = self.run_child(resume, self.root, plan_file, counter)
self.assertEqual(resumed.returncode, 0, resumed.stderr)
self.assertIn('UncertainPublication', resumed.stdout)
self.assertFalse(counter.exists(), 'resume must not issue a second POST')
def test_process_lock_contention_fails_fast_without_post(self):
p, before, after, exact = self.plan_data()
plan_file = Path(self.tmp.name) / 'plan.json'; plan_file.write_text(json.dumps(p))
counter = Path(self.tmp.name) / 'posts'
ready = Path(self.tmp.name) / 'ready'
holder = self.child_prelude() + textwrap.dedent('''
entry = m._journal.open_binding(m.canonical_journal_path(p), p, anchor_file=str(script), origin=m.ORIGIN, validate=m.validate_plan)
fd = m._journal.acquire_lock(entry); Path(sys.argv[4]).write_text('ready'); time.sleep(3); os.close(fd)
''')
proc = subprocess.Popen([sys.executable, '-c', holder, str(self.root), str(plan_file), str(counter), str(ready)],
env={'PATH': os.environ['PATH'], 'PYTHONPATH': str(ROOT)}, text=True,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
try:
deadline = time.monotonic() + 3
while not ready.exists() and time.monotonic() < deadline: time.sleep(.02)
self.assertTrue(ready.exists())
contender = self.child_prelude() + textwrap.dedent('''
try:
m.publish(p, before, API(), Lease(), m.canonical_journal_path(p), declaration={'mode': 'none'})
except Exception as exc:
print(type(exc).__name__)
''')
started = time.monotonic(); result = self.run_child(contender, self.root, plan_file, counter, timeout=2)
self.assertLess(time.monotonic() - started, 2)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertIn('ValidationError', result.stdout)
self.assertFalse(counter.exists())
finally:
proc.kill(); proc.communicate(timeout=3)
def test_crash_after_transport_write_reconciles_without_second_write(self):
p, _, _, _ = self.plan_data()
plan_file = Path(self.tmp.name) / 'plan.json'; plan_file.write_text(json.dumps(p))
counter = Path(self.tmp.name) / 'posts'
crash = self.child_prelude() + textwrap.dedent("""
class CrashAfterWrite(API):
def create_review(self, target, body):
super().create_review(target, body)
os._exit(17)
m.publish(p, before, CrashAfterWrite(), Lease(), declaration={'mode': 'none'})
""")
result = self.run_child(crash, self.root, plan_file, counter)
self.assertEqual(result.returncode, 17, result.stderr)
self.assertEqual(counter.read_text(), '1')
resume = self.child_prelude() + textwrap.dedent("""
result = m.publish(p, before, API(), Lease(), declaration={'mode': 'none'})
print(result['mode'])
""")
result = self.run_child(resume, self.root, plan_file, counter)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertIn('no_op', result.stdout)
self.assertEqual(counter.read_text(), '1')
def test_two_publishing_processes_create_only_once(self):
p, _, _, _ = self.plan_data()
plan_file = Path(self.tmp.name) / 'plan.json'; plan_file.write_text(json.dumps(p))
counter = Path(self.tmp.name) / 'posts'
entered = Path(self.tmp.name) / 'create-entered'; release = Path(self.tmp.name) / 'release'
first = self.child_prelude() + textwrap.dedent("""
class GatedAPI(API):
def create_review(self, target, body):
result = super().create_review(target, body)
Path(sys.argv[4]).write_text('entered')
deadline = time.monotonic() + 8
while not Path(sys.argv[5]).exists():
if time.monotonic() >= deadline: raise TimeoutError('test watchdog')
time.sleep(.02)
return result
result = m.publish(p, before, GatedAPI(), Lease(), declaration={'mode': 'none'})
print(result['mode'])
""")
proc = subprocess.Popen([sys.executable, '-B', '-c', first, str(self.root), str(plan_file), str(counter), str(entered), str(release)],
env={'PATH': os.environ['PATH'], 'PYTHONPATH': str(ROOT)},
text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
try:
deadline = time.monotonic() + 4
while not entered.exists() and time.monotonic() < deadline: time.sleep(.02)
self.assertTrue(entered.exists(), 'first publisher must reach transport')
self.assertEqual(counter.read_text(), '1')
contender = self.child_prelude() + textwrap.dedent("""
try: m.publish(p, before, API(), Lease(), declaration={'mode': 'none'})
except m.ValidationError: print('blocked')
""")
result = self.run_child(contender, self.root, plan_file, counter, timeout=3)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertIn('blocked', result.stdout)
self.assertEqual(counter.read_text(), '1')
release.write_text('release')
stdout, stderr = proc.communicate(timeout=3)
self.assertEqual(proc.returncode, 0, stderr)
self.assertIn('created', stdout)
self.assertEqual(counter.read_text(), '1')
finally:
if proc.poll() is None: proc.kill()
proc.communicate(timeout=3)
def _legacy_files(self, p, before, *, verified=True):
directory = Path(tempfile.mkdtemp(prefix='legacy-', dir=self.tmp.name))
directory.chmod(0o700)
canonical = self.m.canonical
intent = {'schema': 'gitea-agent.native-review-intent/v1',
'fingerprint': hashlib.sha256(self.m.canonical(p)).hexdigest(),
'before_sha256': hashlib.sha256(self.m.canonical(before)).hexdigest(),
'head': p['target']['head'], 'event': 'APPROVED',
'body_sha256': hashlib.sha256(p['body'].encode()).hexdigest()}
with self.m.Journal(directory) as journal:
journal.write('before.json', before)
journal.write('intent.json', intent)
if verified:
journal.write('verified.json', {'verified': True})
return directory
def _legacy_digest(self, directory):
helper = getattr(self.m, 'legacy_journal_digest', None)
self.assertTrue(callable(helper), 'public legacy journal digest helper is required')
return helper(directory)
def test_valid_legacy_registration_is_read_only_and_preserves_source_bytes(self):
p, before, after, exact = self.plan_data(); source = self._legacy_files(p, before)
original = {name: (source / name).read_bytes() for name in ['before.json', 'intent.json', 'verified.json']}
modes = {name: stat.S_IMODE((source / name).stat().st_mode) for name in original}
api = self.fixtures.FakeAPI(p, before, after, exact)
result = self.m.register_legacy_journal(p, source, expected_legacy_sha256=self._legacy_digest(source),
api=api, lease=self.fixtures.FakeLease(), declaration=self.declaration())
self.assertTrue(result['read_only']); self.assertEqual(api.posts, 0)
for name, data in original.items():
self.assertEqual((source / name).read_bytes(), data)
self.assertEqual(stat.S_IMODE((source / name).stat().st_mode), modes[name])
target = self.m.canonical_journal_path(p)
self.assertEqual((target / 'before.json').read_bytes(), original['before.json'])
self.assertFalse((target / 'import.pending.json').exists())
def test_legacy_source_mutation_digest_mismatch_and_extra_intent_fields_reject(self):
p, before, after, exact = self.plan_data(); source = self._legacy_files(p, before)
digest = self._legacy_digest(source)
mutated = json.loads((source / 'before.json').read_text()); mutated['issue']['body'] = 'changed'
(source / 'before.json').write_text(json.dumps(mutated, separators=(',', ':')))
with self.assertRaises(self.m.ValidationError):
self.m.register_legacy_journal(p, source, expected_legacy_sha256=digest, api=self.fixtures.FakeAPI(p, before, after, exact),
lease=self.fixtures.FakeLease(), declaration=self.declaration())
source = self._legacy_files(p, before)
intent = json.loads((source / 'intent.json').read_text()); intent['unexpected'] = True
(source / 'intent.json').write_text(json.dumps(intent, separators=(',', ':')))
with self.assertRaises(self.m.ValidationError):
self.m.register_legacy_journal(p, source, expected_legacy_sha256=self._legacy_digest(source), api=self.fixtures.FakeAPI(p, before, after, exact),
lease=self.fixtures.FakeLease(), declaration=self.declaration())
def test_legacy_symlink_hardlink_and_duplicate_json_are_rejected(self):
p, before, after, exact = self.plan_data()
source = self._legacy_files(p, before)
(source / 'before.json').unlink(); (source / 'before.json').symlink_to(Path(self.tmp.name) / 'elsewhere')
with self.assertRaises(self.m.ValidationError):
self.m.register_legacy_journal(p, source, expected_legacy_sha256='0' * 64, api=self.fixtures.FakeAPI(p, before, after, exact),
lease=self.fixtures.FakeLease(), declaration=self.declaration())
source = self._legacy_files(p, before)
os.link(source / 'before.json', source / 'duplicate.json')
with self.assertRaises(self.m.ValidationError):
self.m.register_legacy_journal(p, source, expected_legacy_sha256=self._legacy_digest(source), api=self.fixtures.FakeAPI(p, before, after, exact),
lease=self.fixtures.FakeLease(), declaration=self.declaration())
def test_registration_rejects_existing_canonical_binding_without_post(self):
p, before, after, exact = self.plan_data(); source = self._legacy_files(p, before)
target = self.m.canonical_journal_path(p)
target.parent.mkdir(parents=True, exist_ok=True); target.mkdir(mode=0o700)
(target / 'intent.json').write_text('{}'); (target / 'intent.json').chmod(0o600)
api = self.fixtures.FakeAPI(p, before, after, exact)
with self.assertRaises(self.m.ValidationError):
self.m.register_legacy_journal(p, source, expected_legacy_sha256=self._legacy_digest(source), api=api,
lease=self.fixtures.FakeLease(), declaration=self.declaration())
self.assertEqual(api.posts, 0)
if __name__ == '__main__':
unittest.main()
@@ -0,0 +1,125 @@
import importlib.util
import json
import multiprocessing
import os
import stat
import tempfile
import unittest
from pathlib import Path
SCRIPT = Path(__file__).resolve().parents[2] / 'agents/review/skills/gitea-code-review/scripts/publication_journal.py'
def load():
spec = importlib.util.spec_from_file_location('publication_journal_safety', SCRIPT)
m = importlib.util.module_from_spec(spec); spec.loader.exec_module(m); return m
def plan(n='a'):
return {'target': {'owner':'o','repository':'r','issue':1,'pull':2,'head':n*40},
'contract_sha256':'b'*64, 'reviewer': {'id':3}}
def validate(p): return p
class SafetyTests(unittest.TestCase):
def test_binding_record_contains_plan_fingerprint_and_strict_schema(self):
m=load()
with tempfile.TemporaryDirectory() as td:
root=Path(td)/'root'; m.root_for=lambda _: root
p=plan(); e=m.open_binding(m.canonical_path(p,anchor_file=__file__,origin='x',validate=validate),p,anchor_file=__file__,origin='x',validate=validate)
record=json.loads((e/'binding.json').read_text())
self.assertEqual(set(record), {'schema','binding','fingerprint'})
self.assertEqual(record['fingerprint'], m.plan_fingerprint(p))
self.assertEqual(m.assert_bound(e,p,anchor_file=__file__,origin='x',validate=validate), e)
def test_different_plan_same_binding_conflicts_without_overwrite(self):
m=load()
with tempfile.TemporaryDirectory() as td:
root=Path(td)/'root'; m.root_for=lambda _: root
p=plan(); path=m.canonical_path(p,anchor_file=__file__,origin='x',validate=validate); m.open_binding(path,p,anchor_file=__file__,origin='x',validate=validate)
q=dict(p); q['extra']='different'
with self.assertRaises(ValueError): m.assert_bound(path,q,anchor_file=__file__,origin='x',validate=validate)
def test_strict_json_rejects_duplicate_and_nonfinite(self):
m=load()
with tempfile.TemporaryDirectory() as td:
p=Path(td)/'x'; p.write_bytes(b'{"a":1,"a":2}') ; os.chmod(p,0o600)
with self.assertRaises(ValueError): m._strict_load(m._secure_file(p))
p.write_bytes(b'{"a":NaN}')
with self.assertRaises(ValueError): m._strict_load(m._secure_file(p))
p.write_bytes(b'{"a":1e999}')
with self.assertRaises(ValueError): m._strict_load(m._secure_file(p))
def test_require_new_seeds_exact_pending_registration_sentinel(self):
m=load()
with tempfile.TemporaryDirectory() as td:
root=Path(td)/'root'; m.root_for=lambda _: root
p=plan(); path=m.canonical_path(p,anchor_file=__file__,origin='x',validate=validate)
entry=m.open_binding(path,p,anchor_file=__file__,origin='x',validate=validate,require_new=True)
self.assertEqual(json.loads((entry/'import.pending.json').read_text()), {'schema':'gitea-agent.native-review-import/v1','state':'pending'})
with self.assertRaises(ValueError):
m.open_binding(path,p,anchor_file=__file__,origin='x',validate=validate)
def test_fifo_does_not_block_and_symlink_hardlink_rejected(self):
m=load()
with tempfile.TemporaryDirectory() as td:
d=Path(td); fifo=d/'fifo'; os.mkfifo(fifo,0o600)
with self.assertRaises(ValueError): m._secure_file(fifo,cap=10)
real=d/'real'; real.write_bytes(b'{}'); os.chmod(real,0o600)
os.symlink(real,d/'link')
with self.assertRaises(Exception): m._secure_file(d/'link')
os.link(real,d/'hard')
with self.assertRaises(ValueError): m._secure_file(real)
def test_deleted_binding_record_is_not_reconstructed(self):
m = load()
with tempfile.TemporaryDirectory() as td:
m.root_for = lambda _: Path(td) / 'root'
p = plan(); path = m.canonical_path(p, anchor_file=__file__, origin='x', validate=validate)
m.open_binding(path, p, anchor_file=__file__, origin='x', validate=validate)
(path / 'binding.json').unlink()
with self.assertRaises(ValueError):
m.open_binding(path, p, anchor_file=__file__, origin='x', validate=validate)
self.assertFalse((path / 'binding.json').exists())
def test_first_use_bootstraps_private_runtime_parent(self):
m = load()
with tempfile.TemporaryDirectory() as td:
runtime = Path(td) / '.runtime'
m.root_for = lambda _: runtime / 'root'
p = plan(); path = m.canonical_path(p, anchor_file=__file__, origin='x', validate=validate)
self.assertEqual(m.open_binding(path, p, anchor_file=__file__, origin='x', validate=validate), path)
self.assertEqual(stat.S_IMODE(runtime.stat().st_mode), 0o700)
def test_prewrite_presence_validation_rejects_schema_corruption(self):
m = load()
for kind in ('root', 'entry'):
with self.subTest(kind=kind), tempfile.TemporaryDirectory() as td:
m.root_for = lambda _: Path(td) / 'root'
p = plan(); path = m.canonical_path(p, anchor_file=__file__, origin='x', validate=validate)
m.open_binding(path, p, anchor_file=__file__, origin='x', validate=validate)
marker = Path(td) / ('root.presence' if kind == 'root' else path.name + '.presence')
record = json.loads(marker.read_text()); record['schema'] = 'corrupt'
marker.write_text(json.dumps(record))
with self.assertRaises(ValueError):
m.assert_bound(path, p, anchor_file=__file__, origin='x', validate=validate)
def test_unsafe_mutex_failure_does_not_leak_descriptor(self):
m = load()
with tempfile.TemporaryDirectory() as td:
path = Path(td) / 'publication.lock'; path.write_text(''); path.chmod(0o644)
def live_fds():
found = set()
for name in os.listdir('/proc/self/fd'):
try: os.readlink('/proc/self/fd/' + name)
except FileNotFoundError: continue
found.add(name)
return found
before = live_fds()
with self.assertRaises(ValueError): m.acquire_lock(Path(td))
after = live_fds()
leaked = after - before
for value in leaked:
try: os.close(int(value))
except OSError: pass
self.assertEqual(after, before)
if __name__ == '__main__': unittest.main()