AI academic research

Cheating LLM Benchmarks is a research project aimed at exploring cheating in automated language model (LLM) benchmarks by building so-called "null models". The project experimentally found that even simple null models can achieve high winning rates on these benchmarks, challenging the validity and reliability of existing benchmarks. This research is important for understanding the limitations of current language models and improving benchmarking methods.

The ICSFSurvey is a survey on internal consistency and self-feedback of large language models. It provides a unified perspective on the self-evaluation and self-renewal mechanism of LLMs, including theoretical framework, systematic classification, evaluation methods, future research directions, etc.

Platonic Representation Hypothesis is a theory about how different AI systems learn and represent the real world. The theory is that although different AI systems may learn in different ways (such as images, text, etc.), their internal representations will eventually converge. This perspective is based on the intuition that all data (images, text, sounds, etc.) are projections of some underlying reality. The theory also explores how representation consistency is measured, and factors that lead to consistency, such as task and data pressure, and increases in model capacity. Additionally, possible implications and limitations of this consistency are discussed.