Tsinghua Proposes Jev-LDE: Editing ICL Demonstrations Just Once Can Sharply Boost 1-shot Average Accuracy

Large language models performing in-context learning (ICL) typically retrieve several demonstrations from an example pool and place them in the prompt with the current query. However, the retrieved Top-K set is not always suitable for the target model: too many examples increase context and inference cost, while irrelevant or redundant ones can hurt the model's judgment. Searching the full combinatorial space of k-example subsets grows quickly, and some model-aware methods require repeatedly querying the target LLM. Researchers at the Department of Automation, Tsinghua University, propose Local Demonstration Editing (LDE) and train a small editing model, Jev-LDE. It does not answer the question itself; instead, before the target LLM runs inference, it makes one edit to the already-retrieved demonstration set, restricted to Keep, Delete, or Replace. In experiments, Jev-LDE outperformed the Semantic TopK initialization in 46 of 48 classification configurations and achieved the best or tied-best result in 44. Across 3 classification datasets and 4 target LLMs, average one-shot accuracy rose from 81.2% to 88.1%. The method is placed after the retriever and performs a single membership edit on the initial demonstration set using replacement candidates. Training uses GRPO-based reinforcement learning, with the reward given by whether the frozen target LLM answers correctly after the edit.