RE-call benchmark results.

This is RE-call's frozen 21 August 2026 run, accepted on 23 August. The official retrieval evaluator rescored that same run on 4 October: Recall@10 is 90.0995%. The earlier value labelled Recall@10 measured whether any gold item was found, a different metric.

Voyage-4-large embeddings, Voyage rerank-2.5 and Qwen image captions; DeepSeek V4 Pro answers. The official QS judge model and prompt ran through OpenRouter, disclosed and accepted. QS is not answer-model-matched to the Qwen baselines. Memexa v2 currently leads the listed systems on both Recall@10 and QS. Oracle rows are excluded from the rank.

The Memexa v2 submission was merged on 4 October 2026. The board compares these submitted configurations from different dates; it is not a simultaneous head-to-head test or a new test of current RE-call.

GPT-4o answers; a homogeneous GPT-5.4 medium judge pass, without correction or citation stripping. Voyage-4-large, lexical plus SPLADE retrieval, Voyage reranking and a bounded context. Completeness is 53.23%.

The separate default judge pass reported 65.60% correctness but mixed reasoning settings. This page uses the homogeneous pass. A locally computed score does not receive a hypothetical place among accepted submissions.

GPT-4o generation with the official prompt, scored by the benchmark's GPT-4o-mini judge. All baselines were recomputed through the same corrected, answerability-conditioned scorer. Their instance set differs from the paper's, so these positions cannot be quoted as the published leaderboard.

The recomputed GPT-4o baseline scores 0.5591. Holding the prompt fixed, replacing ELSER contexts with RE-call contexts changes 0.5516 to 0.5527. Correct refusals are tied with Llama-3.1-70B; that metric depends on the generator prompt as well as retrieval.

Retrieval Task A, a separate 777-query evaluation, reaches nDCG@5 0.4342 and Recall@100 0.7668 with SPLADE and Voyage reranking. The free hybrid SPLADE arm scores 0.3573 and 0.7377 respectively. Task B supplies gold contexts and is a harness control, not a RE-call retrieval result.

GPT-4o reader and judge, token-matched retrieval budgets. RE-call uses the local bge embedder. The direction depends on the reader: a post-hoc Sonnet 4.5 subset reverses it. The paired artifact records the raw-run hashes; raw generated answers are not committed.

Separate retrieval-only tests use 1,536 answerable questions: turn-level hit@5 is 0.671 without reranking and 0.777 with the shipped cross-encoder. These are not session-level competitor scores or judged answer accuracy. Default retrieval abstained on none of the 446 adversarial questions; that failure is retained in the record.

The per-question haystacks contain roughly 49 sessions; the merged index contains 19,195. The intermediate Oracle scope scores 0.719. Knowledge-update retrieval is 36/36 on the comparable arm, while false abstention is 0.481.

This run predates the sparse-leg fix and is effectively dense-only. The indexes and original run output were not retained. The published commands provide a reproduction method, not a retained artifact or independent verification. A separate Bench'd harness run is recorded below.

The harness follows upstream answerer and judge prompts and compares against historical Mem0 Platform answers. The published temporal probe identifies wrong date selection. The source record does not retain a complete aggregate artifact, so this page does not assign an overall score or rank.

BEAM includes deliberately difficult unanswerable questions. Follow-up entailment and threshold probes did not establish a usable general abstention rule. The committed Voyage k-sweep measures nugget-coverage proxy, not judged answer quality.

Hybrid retrieval with bge-small-en-v1.5 versus Voyage-3. This measures the embedder change within RE-call, not RE-call versus other retrieval systems. The corpus samples and configuration are part of the study; these figures are not official full-corpus BEIR nDCG scores.

The study and analysis summaries are public; the complete set of per-corpus run artifacts is not retained in this tree.

Corpora: NFCorpus, SciFact, SCIDOCS, FiQA, ArguAna, and CQADupStack Android, English, Gaming, GIS, Mathematica, Physics, Programmers, Stats, TeX, Unix, Webmasters and WordPress.

Voyage-finance-2, dense retrieval plus reranking over 72,151 chunks, with the dense search-width cap corrected. This is a retrieval-pool experiment, not the benchmark's end-to-end financial QA score.

Claude Code with DeepSeek V4 Flash over a bulk-ingested corpus. No positive arm's confidence interval excludes zero against the designated CLAUDE.md baseline. RE-call ties the bare arm; placebo scores highest. This does not demonstrate an overall coding benefit.

The write lifecycle is not measured. The run disclosed late preregistration, no advance announcement, and source and condition names visible to agents. The current board joins later vendor runs with different admitted cell counts, so its descriptive order is not a controlled pairwise ranking. Earlier pilots and protocol runs remain in the linked history.

The retained documentation records earlier isolation and temporal runs. The original board's artifacts declared a version without confirmation; a declared version and deterministic scoring do not establish an independently verified run. The adapters are public, but the former board is not currently accessible.