Semanlink - Favoris ; Machine learning AND Zero shot

> In most knowledge graphs there are usually concise descriptions for entities, which cannot be well utilized by existing methods... Experimental results on real-world datasets show that, our method outperforms other baselines on the knowledge graph completion and entity classification tasks, especially under the zero-shot setting, which indicates that **our method is capable of building representations for novel entities according to their descriptions**. [Source code on github](https://github.com/xrb92/DKRL).

For fact triples: TransE. Meanwhile, given an entity we will also learn to maximize
the likelihood of predicting its description (using either CBOW or CNN encoder) (head + relation = tail, also in "text space")

Two types of representations for entities: structure-based
representations and description-based representations. They are learned simultaneously into the same
vector space but not forced to be unified **so that novel
entities with only descriptions can be represented**.

2020-10-02 About

Hugging Face sur Twitter : "No labeled data? No problem. The 🤗 Transformers master branch now includes a built-in pipeline for zero-shot text classification...

Tags:

2020-08-12 About

[2006.15020] Pre-training via Paraphrasing

Tags:

2020-06-30 About

[1911.03814] Scalable Zero-shot Entity Linking with Dense Entity Retrieval

Tags:

2020-05-02 About

[1703.07464] No Fuss Distance Metric Learning using Proxies

Tags:

> We address the problem of distance metric learning (DML), defined as learning a distance consistent with a notion of semantic similarity...
> Traditionnaly, supervision is expressed in the form of sets of points that follow
an ordinal relationship – an anchor point x is similar to
a set of positive points Y , and dissimilar to a set of negative
points Z, and a loss defined over these distances is minimized.
> Triplet-Based methods are challenging to optimize (a main issue is the need for finding informative triplets).
>
> We propose to **optimize the triplet loss on a different space of triplets, consisting of an anchor data point and similar and dissimilar proxy points which are learned as well**. These proxies approximate the original data points, so that a triplet loss over the proxies is a tight upper bound of the original loss.

Mentioned in this [blog post](/doc/2020/01/training_a_speaker_embedding_fr):

> "**Proxy based triplet learning**": instead of generating triplets, we learn an embedding for each class and use the learnt embedding as a proxy for triplets as part of the training. In other words, we can train end to end without the computationally expensive step of resampling triplets after each network update.

Near the conclusion:

> Our formulation of Proxy-NCA loss produces a loss very
similar to the standard cross-entropy loss used in classification.
However, we arrive at our formulation from a different
direction: we are not interested in the actual classifier and
indeed discard the proxies once the model has been trained.
Instead, the proxies are auxiliary variables, enabling more
effective optimization of the embedding model parameters.
**As such, our formulation not only enables us to surpass the
state of the art in zero-shot learning, but also offers an explanation
to the effectiveness of the standard trick of training
a classifier, and using its penultimate layer’s output as the
embedding.**

2020-02-09 About