current state / answer policy / measurement
SEAM — Where We Are
The semantic conversation adapter is implemented and verified. It expands SEAM's improvement loop beyond retrieval tuning into cross-turn evidence completion and bounded inference. The implementation is green; its benchmark effect remains deliberately unclaimed until a new measurement run.
HISTORY#382 · PR #142 · implementation and review complete · free survivor measurement next
How the current state was reached
integrity
contract
adjudication
implemented
measurement
Product-correct cat1 and cat3 improvement
Raise both categories to at least 0.80 through behavior that improves the product, not by teaching the answerer to guess benchmark labels.
Raw benchmark scores remain visible. Adjudicated results are reported separately and may never conceal a raw regression.
What is complete right now
Last measured judge/2 values vs target
│ target = 0.800
PR #142 made no benchmark generation or provider call. These values predate the implementation and remain the latest measured truth.
Offline adjudication
Opportunity ceilings describe the reviewed failure set. They are not newly measured runtime scores.
29 reviewed non-correct cases
14 reviewed inference targets
Safe inference alone cannot honestly reach the raw target. Eight expected answers were judged defective or underspecified; six were defensible high-confidence inference targets.
What landed in the semantic conversation slice
conversation/1
Projects retrieved turns into a readable evidence view with stable rows while preserving the default path when disabled.
set-completion
Scans across turns, resolves aliases and pronouns, deduplicates facts, and validates requested counts before synthesis.
inference/high-confidence/1
Allows bounded world-knowledge inference only when one interpretation is well supported; ambiguity still requires abstention.
raw + adjudicated views
Emits both views from one scorer execution. Raw regressions remain promotion blockers instead of being hidden by correction.
fair comparator policy
SEAM, Mem0, and Zep can receive the same opt-in answer policy so a comparison still isolates memory-system behavior.
category-floor progress
The improvement loop can select changes that move cat1 or cat3 toward 0.80 even when an aggregate delta is small.
Review caught three boundary defects
FIXED · policy coupling
Inference-only candidates no longer accidentally enable cat1 set-completion behavior.
FIXED · overlay integrity
Adjudication overlays now fail closed when they name a case absent from the raw report.
FIXED · floor validation
CLI category floors must be numeric values inside the inclusive [0,1] range.
Final committed-diff review: zero findings · thread audit: zero unresolved threads.
Semantic conversation answer policy
agent/cat13-semantic-conversation-adapter
DRAFT · CLEANHead f69bf60 · implementation, review fixes, HISTORY#382, handoff, derived streams, and snapshot closeout.
Canonical verification
Two established expected failures remain explicitly marked as xfail.
External pgvector
Validated against the existing healthy service; no service lifecycle change was made.
Repo state
The next evidence-producing steps
What this page may and may not expose
PUBLIC-SAFE
- Aggregate scores and counts
- Architecture summaries
- PR status and test totals
- Public next actions
NEVER EMIT
- Private case text or tables
- Local paths or credentials
- Provider responses or hidden reasoning
- Private session links or raw history