arxiv.org/abs/2108.10341
Preview meta tags from the arxiv.org website.
Linked Hostnames
25- 21 links toarxiv.org
- 12 links toinfo.arxiv.org
- 4 links todblp.uni-trier.de
- 2 links todoi.org
- 2 links tosubscribe.sorryapp.com
- 1 link toapi.semanticscholar.org
- 1 link tocore.ac.uk
- 1 link todagshub.com
Thumbnail
Search Engine Appearance
Query Embedding Pruning for Dense Retrieval
Recent advances in dense retrieval techniques have offered the promise of being able not just to re-rank documents using contextualised language models such as BERT, but also to use such models to identify documents from the collection in the first place. However, when using dense retrieval approaches that use multiple embedded representations for each query, a large number of documents can be retrieved for each query, hindering the efficiency of the method. Hence, this work is the first to consider efficiency improvements in the context of a dense retrieval approach (namely ColBERT), by pruning query term embeddings that are estimated not to be useful for retrieving relevant documents. Our proposed query embeddings pruning reduces the cost of the dense retrieval operation, as well as reducing the number of documents that are retrieved and hence require to be fully scored. Experiments conducted on the MSMARCO passage ranking corpus demonstrate that, when reducing the number of query embeddings used from 32 to 3 based on the collection frequency of the corresponding tokens, query embedding pruning results in no statistically significant differences in effectiveness, while reducing the number of documents retrieved by 70%. In terms of mean response time for the end-to-end to end system, this results in a 2.65x speedup.
Bing
Query Embedding Pruning for Dense Retrieval
Recent advances in dense retrieval techniques have offered the promise of being able not just to re-rank documents using contextualised language models such as BERT, but also to use such models to identify documents from the collection in the first place. However, when using dense retrieval approaches that use multiple embedded representations for each query, a large number of documents can be retrieved for each query, hindering the efficiency of the method. Hence, this work is the first to consider efficiency improvements in the context of a dense retrieval approach (namely ColBERT), by pruning query term embeddings that are estimated not to be useful for retrieving relevant documents. Our proposed query embeddings pruning reduces the cost of the dense retrieval operation, as well as reducing the number of documents that are retrieved and hence require to be fully scored. Experiments conducted on the MSMARCO passage ranking corpus demonstrate that, when reducing the number of query embeddings used from 32 to 3 based on the collection frequency of the corresponding tokens, query embedding pruning results in no statistically significant differences in effectiveness, while reducing the number of documents retrieved by 70%. In terms of mean response time for the end-to-end to end system, this results in a 2.65x speedup.
DuckDuckGo
Query Embedding Pruning for Dense Retrieval
Recent advances in dense retrieval techniques have offered the promise of being able not just to re-rank documents using contextualised language models such as BERT, but also to use such models to identify documents from the collection in the first place. However, when using dense retrieval approaches that use multiple embedded representations for each query, a large number of documents can be retrieved for each query, hindering the efficiency of the method. Hence, this work is the first to consider efficiency improvements in the context of a dense retrieval approach (namely ColBERT), by pruning query term embeddings that are estimated not to be useful for retrieving relevant documents. Our proposed query embeddings pruning reduces the cost of the dense retrieval operation, as well as reducing the number of documents that are retrieved and hence require to be fully scored. Experiments conducted on the MSMARCO passage ranking corpus demonstrate that, when reducing the number of query embeddings used from 32 to 3 based on the collection frequency of the corresponding tokens, query embedding pruning results in no statistically significant differences in effectiveness, while reducing the number of documents retrieved by 70%. In terms of mean response time for the end-to-end to end system, this results in a 2.65x speedup.
General Meta Tags
17- title[2108.10341] Query Embedding Pruning for Dense Retrieval
- titleopen search
- titleopen navigation menu
- titlecontact arXiv
- titlesubscribe to arXiv mailings
Open Graph Meta Tags
10- og:typewebsite
- og:site_namearXiv.org
- og:titleQuery Embedding Pruning for Dense Retrieval
- og:urlhttps://arxiv.org/abs/2108.10341v1
- og:image/static/browse/0.3.4/images/arxiv-logo-fb.png
Twitter Meta Tags
6- twitter:site@arxiv
- twitter:cardsummary
- twitter:titleQuery Embedding Pruning for Dense Retrieval
- twitter:descriptionRecent advances in dense retrieval techniques have offered the promise of being able not just to re-rank documents using contextualised language models such as BERT, but also to use such models to...
- twitter:imagehttps://static.arxiv.org/icons/twitter/arxiv-logo-twitter-square.png
Link Tags
12- apple-touch-icon/static/browse/0.3.4/images/icons/apple-touch-icon.png
- canonical/abs/2108.10341
- icon/static/browse/0.3.4/images/icons/favicon-32x32.png
- icon/static/browse/0.3.4/images/icons/favicon-16x16.png
- manifest/static/browse/0.3.4/images/icons/site.webmanifest
Links
61- http://arxiv.org/licenses/nonexclusive-distrib/1.0
- http://gotit.pub/faq
- http://www.bibsonomy.org/BibtexHandler?requTask=upload&url=https://arxiv.org/abs/2108.10341&description=Query Embedding Pruning for Dense Retrieval
- https://api.semanticscholar.org/arXiv:2108.10341
- https://arxiv.org