A Conditional Mutual Information Analysis of Response Variety in Language Models

概要

Large language models provide a variety of responses at inference (generation) that conventionally depend on a temperature parameter. Using a conditional mutual information approach, a quantitative, entropy based analysis of response variety is given. Three limiting behaviors are defined: arbiter, independent (conditionally) and neutral (deterministic) that span no to maximal mutual information. We give examples of each behavior and suggest their application to: temperature analysis, language analysis, prompt compression, and knowledge graphs.

参考リンク

• Paper ↗

リンク先のライセンス・再現性・データ条件をご確認ください。自動分類や実装の目安は保証ではありません。

実装を始めるプロンプト

参考リンクを確認したうえで、次の内容をコピーして利用できます。

# 論文・記事
A Conditional Mutual Information Analysis of Response Variety in Language Models

# 概要
Large language models provide a variety of responses at inference (generation) that conventionally depend on a temperature parameter. Using a conditional mutual information approach, a quantitative,…

# 原典
https://arxiv.org/abs/2611.03836

上記の手法を実装してください。実行前に依存関係とライセンスを確認してください。