About This Document
- sl:arxiv_author :
- sl:arxiv_firstAuthor : Omar Khattab
- sl:arxiv_num : 2004.12832
- sl:arxiv_published : 2020-04-27T14:21:03Z
- sl:arxiv_summary : Recent progress in Natural Language Understanding (NLU) is driving fast-paced
advances in Information Retrieval (IR), largely owed to fine-tuning deep
language models (LMs) for document ranking. While remarkably effective, the
ranking models based on these LMs increase computational cost by orders of
magnitude over prior approaches, particularly as they must feed each
query-document pair through a massive neural network to compute a single
relevance score. To tackle this, we present ColBERT, a novel ranking model that
adapts deep LMs (in particular, BERT) for efficient retrieval. ColBERT
introduces a late interaction architecture that independently encodes the query
and the document using BERT and then employs a cheap yet powerful interaction
step that models their fine-grained similarity. By delaying and yet retaining
this fine-granular interaction, ColBERT can leverage the expressiveness of deep
LMs while simultaneously gaining the ability to pre-compute document
representations offline, considerably speeding up query processing. Beyond
reducing the cost of re-ranking the documents retrieved by a traditional model,
ColBERT's pruning-friendly interaction mechanism enables leveraging
vector-similarity indexes for end-to-end retrieval directly from a large
document collection. We extensively evaluate ColBERT using two recent passage
search datasets. Results show that ColBERT's effectiveness is competitive with
existing BERT-based models (and outperforms every non-BERT baseline), while
executing two orders-of-magnitude faster and requiring four orders-of-magnitude
fewer FLOPs per query.@en
- sl:arxiv_title : ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT@en
- sl:arxiv_updated : 2020-06-04T05:28:21Z
- sl:bookmarkOf : https://arxiv.org/abs/2004.12832
- sl:creationDate : 2022-01-12
- sl:creationTime : 2022-01-12T00:15:40Z
Documents with similar tags (experimental)
Preferences...