Au sujet de ce document
- sl:arxiv_author :
- sl:arxiv_firstAuthor : Kawin Ethayarajh
- sl:arxiv_num : 1810.04882
- sl:arxiv_published : 2018-10-11T08:08:40Z
- sl:arxiv_summary : A surprising property of word vectors is that word analogies can often be
solved with vector arithmetic. However, it is unclear why arithmetic operators
correspond to non-linear embedding models such as skip-gram with negative
sampling (SGNS). We provide a formal explanation of this phenomenon without
making the strong assumptions that past theories have made about the vector
space and word distribution. Our theory has several implications. Past work has
conjectured that linear substructures exist in vector spaces because relations
can be represented as ratios; we prove that this holds for SGNS. We provide
novel justification for the addition of SGNS word vectors by showing that it
automatically down-weights the more frequent word, as weighting schemes do ad
hoc. Lastly, we offer an information theoretic interpretation of Euclidean
distance in vector spaces, justifying its use in capturing word dissimilarity.@en
- sl:arxiv_title : Towards Understanding Linear Word Analogies@en
- sl:arxiv_updated : 2019-08-12T04:04:15Z
- sl:bookmarkOf : https://arxiv.org/abs/1810.04882
- sl:creationDate : 2019-06-24
- sl:creationTime : 2019-06-24T08:33:44Z
Documents with similar tags (experimental)
Préférences...