About This Document
- sl:arxiv_author : Niklas Muennighoff
- sl:arxiv_firstAuthor : Niklas Muennighoff
- sl:arxiv_num : 2202.08904
- sl:arxiv_published : 2022-02-17T21:35:56Z
- sl:arxiv_summary : Decoder transformers have continued increasing in scale reaching hundreds of
billions of parameters. Due to their scale the same decoder sets
state-of-the-art results on various language tasks via prompting or
fine-tuning. Yet, these large foundation models remain unusable for the related
fields of semantic search and sentence embeddings. This prevents possibly new
state-of-the-art results and forces organizations to train and maintain
separate models. To this end, we propose SGPT to use decoders for sentence
embeddings and semantic search via prompting or fine-tuning. At 5.8 billion
parameters SGPT improves on the previously best sentence embeddings by a margin
of 7% and outperforms a concurrent method with 175 billion parameters as
measured on the BEIR search benchmark. Code, models and result files are freely
available at https://github.com/Muennighoff/sgpt.@en
- sl:arxiv_title : SGPT: GPT Sentence Embeddings for Semantic Search@en
- sl:arxiv_updated : 2022-08-05T09:33:10Z
- sl:bookmarkOf : https://arxiv.org/abs/2202.08904
- sl:creationDate : 2023-04-25
- sl:creationTime : 2023-04-25T00:02:46Z
- sl:relatedDoc : http://www.semanlink.net/doc/2022/09/2106_10199_bitfit_simple_par
Documents with similar tags (experimental)
Preferences...