About This Document
- sl:arxiv_author :
- sl:arxiv_firstAuthor : Hongjin Su
- sl:arxiv_num : 2212.09741
- sl:arxiv_published : 2022-12-19T18:57:05Z
- sl:arxiv_summary : We introduce INSTRUCTOR, a new method for computing text embeddings given
task instructions: every text input is embedded together with instructions
explaining the use case (e.g., task and domain descriptions). Unlike encoders
from prior work that are more specialized, INSTRUCTOR is a single embedder that
can generate text embeddings tailored to different downstream tasks and
domains, without any further training. We first annotate instructions for 330
diverse tasks and train INSTRUCTOR on this multitask mixture with a contrastive
loss. We evaluate INSTRUCTOR on 70 embedding evaluation tasks (66 of which are
unseen during training), ranging from classification and information retrieval
to semantic textual similarity and text generation evaluation. INSTRUCTOR,
while having an order of magnitude fewer parameters than the previous best
model, achieves state-of-the-art performance, with an average improvement of
3.4% compared to the previous best results on the 70 diverse datasets. Our
analysis suggests that INSTRUCTOR is robust to changes in instructions, and
that instruction finetuning mitigates the challenge of training a single model
on diverse datasets. Our model, code, and data are available at
https://instructor-embedding.github.io.@en
- sl:arxiv_title : One Embedder, Any Task: Instruction-Finetuned Text Embeddings@en
- sl:arxiv_updated : 2022-12-20T05:11:06Z
- sl:bookmarkOf : https://arxiv.org/abs/2212.09741
- sl:creationDate : 2023-02-17
- sl:creationTime : 2023-02-17T18:12:26Z
- sl:relatedDoc : http://www.semanlink.net/doc/2023/02/hkunlp_instructor_xl_·_hugging_
Documents with similar tags (experimental)
Preferences...