Your Benchmark Is an Empirical Measure Over Difficulty
A benchmark is not merely a collection of test questions; it naturally induces an empirical measure over difficulty. In this work, we develop this measure-based view of benchmark evaluation.
nlp
MLINFO COLLECTION
技術文書・実験ノート検索
過去の実験ノート、技術報告書、特許、論文を探しやすくするための製造業向けRAG導線です。
関連する論文・実装
24 / 25Expandable Modality Experts for Medical Vision-Language Learning
Learning across medical imaging modalities requires making use of images without paired text and accommodating new domains while retaining existing representations. We propose a medical…
deep-learning
GeoMem-VLA: Internalizing 3D Scene Understanding into Temporal Memory for Vision-Language-Action Models
Geometric structure is essential for robotic manipulation and understanding the 3D world. Most VLA models rely on 2D representations of current observations, while geometry-aware approaches often…
nlp
Robot Generalization Beyond Volume: The Diversity Path via Zero-Shot Capabilities
General-purpose robots must handle tasks, objects and environments they were never trained on, yet zero-shot generalization remains one of the least solved problems in physical AI. In digital AI,…
nlp
UniRAP: Towards Unified Part-level Physical Affordance Reasoning and Actionable Perception
Vision-language perception has achieved impressive progress in aligning natural language with visual observations, yet grounding high-level semantics into part-level physical interaction remains…
nlp
AirPlayground: Benchmarking Aerial Navigation across Diverse Reconstructed Environments
Large-scale aerial resources provide a foundation for learning navigation across diverse environments. However, existing resources remain limited in providing (1) densely sampled flight…
nlp
Effects of Model Scale and Quantization on Social Bias and Abstention in Question Answering
The deployment of small and quantized language models in resource-constrained settings raises an important fairness question: when a compressed model produces fewer biased responses, does this…
deep-learning
LLMs as survey respondents: Comparison to simple semantic baselines
Large language models (LLMs) have been explored as surrogates for human survey respondents. A more promising application is data augmentation, where LLMs predict new responses from partial human data.
nlp
Unlocking Spatial Grounding in Flow-Matching Models via SNR Tuning
Despite the success of foundation models in image synthesis, achieving fine-grained spatial control remains a significant challenge, typically requiring auxiliary structural adapters or complex…
nlp
Rethinking Underwater Image Enhancement: Learning through Text-Guided Degradation Curriculum
Existing paired clean-degraded datasets have advanced learning-based underwater image enhancement, but they remain either limited in scale or difficult to acquire, and still fail to adequately cover…
nlp
Hyperspherical Simplex Representations for Backward-Compatible Retrieval
Training modern AI models has become increasingly expensive, and updating deployed models can alter their internal feature representations, affecting applications built upon them. In retrieval…
nlp
pplx-embed-v2-late Technical Report
In this report, we introduce pplx-embed-v2-late, a pair of 0.6B and 9B multilingual and multimodal late-interaction models. Both models achieve state-of-the-art results within their respective size…
deep-learning
AutoPhyX: Automatic Text-Condition Physics Property Generation
High-fidelity 4D physical simulation requires accurate physical parameter assignments that align with diverse material properties. Previous methods struggle with critical bottlenecks: they either…
nlp
Make it Walk: Computational Design of LEGO Walkers
LEGO walkers are walking automata constructed from modular LEGO Technic bricks. They exhibit intriguing locomotion and are also highly reconfigurable through the modular Technic bricks, making them…
nlp
Beyond Shortcut Learning: Hierarchical Spatial Prototype Regularization for Face Forgery Detection
Recent advances in generative models have enabled highly realistic face forgeries, yet generalizing deepfake detectors to unseen manipulation pipelines remains a major challenge. One critical yet…
nlp
Look Beyond the Question: Preserve-Focus-Fuse for Whole-Slide Image Question Answering
A question about a whole-slide image rarely names every morphological finding needed to answer it. Question-guided patch selection can therefore overlook supporting morphology, while slide-level…
nlp
A KAN-based Non-linear Factorization for Embedding Layer Compression
Among existing embedding compression approaches, low-rank factorization provides a simple and mathematically principled solution by approximating the embedding matrix with a reduced-rank…
deep-learning
Don't Stop Long-Context Continual Pre-Training Too Early: An Empirical Scaling Law
Long-context continual pre-training (LCCP) is widely used to improve long-context capability, but how its benefits scale with the number of tokens used for LCCP remains unclear. We investigate this…
nlp
OACap: Caption-Guided Active Observation for Detailed Video Captioning
Unlike question answering, where a query specifies a perceptual target, open-ended detailed captioning must determine what evidence to acquire and how to integrate it into a coherent global…
nlp
Signals and Queries: Token-Efficient Agentic Context Discovery
User-facing LLM agents must identify which behavioral policies apply at each turn, but evaluating every policy with an LLM is costly. We introduce Signals and Queries (S&Q), a lightweight first-stage…
deep-learning
Compressing Onboard Language Models for Space Robots under a Behavioral-Fidelity Requirement
Language models are being explored as the interface through which space robots interpret operator commands and select among verified skills. Running them onboard calls for low-bit compression, and…
deep-learning
Hierarchical Masked Diffusion Language Models
Masked diffusion language models have emerged as an alternative to autoregressive models, enabling parallel token generation. However, standard masked diffusion language models use a shared noise…
nlp
MOSAIC: An Omni-Modal Benchmark for Long-Term Memory and Cross-Modal Reasoning in Conversational Agents
Omni-modal assistants must remember not only what users say, but also what they show and convey. Existing long-term memory benchmarks are mostly text-and-image or report a single end-to-end score, so…
computer-vision
DepState-JEPA: Learning Depth-Grounded State Representation for Visual Perception and Manipulation
Predictive representations for robot manipulation should connect scene geometry with the state changes induced by actions. Our intuition is that understanding what an action will change starts with…
nlp