nlparxiv

The Quine Test: How Much of a Model's Self Survives Its Own Description?

Frontier language models express coherent preferences, but behavioral evidence alone cannot say whether these belong to the model or to a character it portrays. Following Hofstadter's view of the…

Bibin Babu / Ivan P. Yamshchikov / Aleksei Shpilman

用途
原典で確認
難易度の目安
Hard
実装コストの目安
Low

概要

Frontier language models express coherent preferences, but behavioral evidence alone cannot say whether these belong to the model or to a character it portrays. Following Hofstadter's view of the self as a self-description, we ask whether a model's description of itself works as a portable specification of its preference profile, a quine test: can the text a model writes about itself regenerate its behavior elsewhere? Ten models from nine labs write self-descriptions; we place each one in the prompt of other models (receivers) and of fresh instances of the author, and measure how closely their choices on a 110-item preference battery match the author's. First, the tested native self-descriptions (native: the profile measured under a generic assistant prompt) show no average reconstruction gain across variations in length, framing, wording and prompt placement: on a 0 to 1 fidelity scale where a model's fidelity to a fresh instance of itself is 0.979 and to unrelated models about 0.89, the one-sided 95% upper bound on the pooled gain is below +0.01 in every tested system-prompt condition, and descriptions based on the author's observed choices do no better. Second, the channel itself is strong: explicitly listing the author's answers lets receivers reproduce its profile at fidelity 0.955 to 0.960 across three model grids, within 0.03 of same-model repeatability. Third, self-descriptions written under an explicit preference policy transfer the assigned preferences almost as well as the policy text itself. Fourth, reading its own self-description shifts even the author away from its baseline, modestly more than unrelated prose of the same length. For the preferences measured here, native self-descriptions influence behavior without reliably specifying how to reproduce it in another model. We release the pipeline, battery, raw call logs, and 198 model-written descriptions.

参考リンク

リンク先のライセンス・再現性・データ条件をご確認ください。自動分類や実装の目安は保証ではありません。

実装を始めるプロンプト

参考リンクを確認したうえで、次の内容をコピーして利用できます。

# 論文・記事
The Quine Test: How Much of a Model's Self Survives Its Own Description?

# 概要
Frontier language models express coherent preferences, but behavioral evidence alone cannot say whether these belong to the model or to a character it portrays. Following Hofstadter's view of the…

# 原典
https://arxiv.org/abs/2610.38726

上記の手法を実装してください。実行前に依存関係とライセンスを確認してください。

関連する記事