Skip to content
The Chat Archive
deepseek-llm-67b-chat· DeepSeek Arena Human Preference 55K

summarize: Ranking has always been one of the top concerns in information retrieval…

  1. User

    summarize: Ranking has always been one of the top concerns in information retrieval researches. For decades, the lexical matching signal has dominated the ad-hoc retrieval process, but solely using this signal in retrieval may cause the vocabulary mismatch problem. In recent years, with the development of representation learning techniques, many researchers turn to Dense Retrieval (DR) models for better ranking performance. Although several existing DR models have already obtained promising results, their performance improvement heavily relies on the sampling of training examples. Many effective sampling strategies are not efficient enough for practical usage, and for most of them, there still lacks theoretical analysis in how and why performance improvement happens. To shed light on these research questions, we theoretically investigate different training strategies for DR models and try to explain why hard negative sampling performs better than random sampling. Through the analysis, we also find that there are many potential risks in static hard negative sampling, which is employed by many existing training methods. Therefore, we propose two training strategies named a Stable Training Algorithm for dense Retrieval (STAR) and a query-side training Algorithm for Directly Optimizing Ranking pErformance (ADORE), respectively. STAR improves the stability of DR training process by introducing random negatives. ADORE replaces the widely-adopted static hard negative sampling method with a dynamic one to directly optimize the ranking performance. Experimental results on two publicly available retrieval benchmark datasets show that either strategy gains significant improvements over existing competitive baselines and a combination of them leads to the best performance

  2. Assistant

    In information retrieval research, ranking has been a major concern. Traditional ad-hoc retrieval methods rely heavily on lexical matching signals, which can lead to the vocabulary mismatch problem. To address this, researchers have turned to Dense Retrieval (DR) models, which use representation learning techniques. While some DR models have shown promising results, their performance improvement heavily depends on the sampling of training examples. Many effective sampling strategies are not efficient for practical use, and there is a lack of theoretical analysis of how and why performance improvement occurs. To address these research questions, the authors investigate different training strategies for DR models and explain why hard negative sampling performs better than random sampling. They also identify potential risks in static hard negative sampling, which is commonly used in existing training methods. To address these issues, the authors propose two training strategies, STAR and ADORE, which improve the stability of the DR training process and optimize ranking performance. Experimental results show that either strategy outperforms existing competitive baselines, and combining them leads to the best performance.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.