Semanlink - Intelligence AND Event

> The most
successful previous approaches like InferSent (Conneau
et al., 2017), Universial Sentence Encoder
(USE) (Cer et al., 2018) and SBERT (Reimers and
Gurevych, 2019) heavily relied on labeled data to
train sentence embedding models.
>
> TSDAE can
achieve up to 93.1% of the performance of indomain
supervised approaches. Further, we
show that TSDAE is **a strong domain adaptation
and pre-training method for sentence
embeddings**, significantly outperforming other
approaches like Masked Language Model.

> During training, TSDAE
encodes corrupted sentences into fixed-sized
vectors and requires the decoder to reconstruct the
original sentences from this sentence embedding.

- <https://www.sbert.net/examples/unsupervised_learning/TSDAE/README.html>
- [github](https://github.com/UKPLab/sentence-transformers/tree/master/examples/unsupervised_learning/TSDAE)
- [UKPLab/sentence-transformers: Sentence Embeddings with BERT & XLNet](doc:2020/07/ukplab_sentence_transformers_s)
- [twitter](https://twitter.com/KexinWang2049/status/1433361957579538432):

> **TSDAE can learn domain-specific sentence embeddings with unlabeled sentences**
>
> Most importantly, instead of STS (Semantic Textual Similarity), **we suggest evaluating unsupervised sentence embeddings on the domain-specific tasks&datasets, which is the real use case for them**. Actually, STS scores do not correlate with performance on specific tasks.

2021-09-01 A propos

[2010.02353] Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages

Tags:

2021-08-25 A propos

Knowledge Graphs in Natural Language Processing @ ACL 2021 | by Michael Galkin | Aug, 2021

Tags:

2021-08-06 A propos

AfricaNLP Workshop | Putting Africa on the NLP Map. ICLR 2020, Virtual Event

Tags:

2021-06-30 A propos

AfricaNLP Workshop

Tags:

2021-06-30 A propos

[1807.04905] Ultra-Fine Entity Typing

Tags:

2021-06-22 A propos

[2106.04098] Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Model

Tags:

2021-06-16 A propos

scikit-learn Pipelines meet Knowledge Graphs - The Python kgextension Package ESWC 2021

Tags:

2021-06-10 A propos

[2004.04906] Dense Passage Retrieval for Open-Domain Question Answering

Tags:

2021-06-03 A propos

Simple Unsupervised Keyphrase Extraction using Sentence Embeddings - ACL Anthology (2018)

Tags:

2021-05-31 A propos

[2009.12030] AutoETER: Automated Entity Type Representation for Knowledge Graph Embedding

Tags:

2021-05-17 A propos

[2011.05864] On the Sentence Embeddings from Pre-trained Language Models

Tags:

2021-04-19 A propos

exBERT: Extending Pre-trained Models with Domain-specific Vocabulary Under Constrained Training Resources - ACL Anthology

Tags:

**Focus on the Embedding of Domain-specific Vocabulary.**

> exBERT
adds a new domain-specific vocabulary and the corresponding
embedding layer, as well as a small
extension module to the original unmodified model

> a pretraining
method allowing **low-cost embedding of
domain-specific vocabulary in the context of an
existing large pre-trained model such as BERT**

> exBERT... explicitly incorporates
the new domain’s vocabulary, while being able to
**reuse the original pre-trained model’s weights as is**
to reduce required computation and training data. Specifically, exBERT extends BERT by augmenting
its embeddings for the original vocabulary with
new embeddings for the domain-specific vocabulary
via **a learned small “extension” module**. **The
output of the original and extension modules are
combined via a trainable weighted sum operation**

In a way similar to concept developed in

> [[1902.00751] Parameter-Efficient Transfer Learning for NLP](doc:2021/04/1902_00751_parameter_efficien), but not in the fine-tuning paradigm.

[Github](https://github.com/cgmhaicenter/exBERT)

2021-04-11 A propos

KGC 2021 Call for Presentations | The Knowledge Graph Conference

Tags:

The Knowledge Graph Conference

2021-01-30 A propos

Journée TAL grand public – GDR TAL

Tags:

2021-01-08 A propos

Event Extraction by Answering (Almost) Natural Questions

Tags:

2020-12-17 A propos

TextGraphs 2020

Tags:

2020-12-13 A propos

Knowledge Base Embedding By Cooperative Knowledge Distillation - ACL Anthology

Tags:

KD-MKB paper

2020-12-05 A propos

[2010.01057] LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention

Tags:

2020-11-26 A propos

Knowledge Graphs in NLP @ EMNLP 2020 | by Michael Galkin | Nov, 2020 | Medium

Tags:

2020-11-24 A propos

Text-Enhanced Representation Learning for Knowledge Graph (IJCAI 2016)

Tags:

2020-11-03 A propos

[2010.03496] Inductive Entity Representations from Text via Link Prediction

Tags:

BLP "BERT for Link Prediction". Central idea: **training an entity encoder with a
link prediction objective** (using the textual descriptions of entities when computing entity representations - hence not failing with entities unknown in training)

> a method for **learning representations
of entities**, that uses a **pre-trained Transformer** based
architecture as an entity encoder, and
**link prediction training on a knowledge graph
with textual entity descriptions**.

> using entity descriptions,
an entity encoder is trained for link prediction in
a knowledge graph. The encoder can then be used
without fine-tuning to obtain features for entity classification
and information retrieval

Cites [Xie et al](doc:2020/10/representation_learning_of_know) and [Kepler](doc:2020/11/1911_06136_kepler_a_unified_). They claim that their
objective targeted exclusively for link prediction (and not an objective that combines language modeling
and link prediction as Kepler)
performs better than Kepler's more complex one.

2020-11-03 A propos

Towards Unsupervised Text Classification Leveraging Experts and Word Embeddings - (ACL 2019)

Tags:

2020-10-05 A propos

Leveraging Just a Few Keywords for Fine-Grained Aspect Detection Through Weakly Supervised Co-Training - ACL Anthology (2019)

Tags:

2020-09-06 A propos

Knowledge Graphs in Natural Language Processing @ ACL 2020 | by Michael Galkin

Tags:

2020-07-12 A propos

Learning to Tag OOV Tokens by Integrating Contextual Representation and Background Knowledge (ACL Anthology 2020)

Tags:

Aim to leverage both contextual representation of input text (deep LMs)  and  knowledge derived
from curated KBs ([Wordnet](tag:wordnet)) to improve [slot tagging](tag:slot_tagging) in the presence of [out-of-vocab](tag:oov) words ([few-shot scenario](tag:few_shot_learning))

Method:

1. retrieve potentially relevant KB entities and
encode them into distributed representations that
describe global graph-structured information
2. BERT encoder
layer to capture context-aware representations of
the sequence and attend to the KB embeddings
using multi-level graph attention
3. integrate
BERT embeddings and the KB embeddings
to predict the slot type

Contributions:

1. feasibility of applying lexical ontology
to facilitate recognizing OOV words. First to consider the large-scale background
knowledge for enhancing context-aware
slot tagging models.
2. a knowledge integration mechanism that uses multi-level graph
attention to model explicit lexical relations.
3.experiments on two benchmark datasets

> our method makes a notable difference in a
scenario where samples are linguistically diverse,
and large vocab exists.

(Better improvements when using RNN than BERT, because BERT already contains a lot of background knowledge)

2020-07-04 A propos

Information Bottleneck for NLP (parsing & summarization)

Tags:

2020-06-29 A propos

[1910.00163] Specializing Word Embeddings (for Parsing) by Information Bottleneck

Tags:

2020-06-29 A propos

Representation Learning for Information Extraction from Form-like Documents – Google Research

Tags:

2020-06-15 A propos

[1909.04164] Knowledge Enhanced Contextual Word Representations

Tags:

2020-05-13 A propos

Bootstrapping Entity Alignment with Knowledge Graph Embedding | IJCAI

Tags:

2020-05-11 A propos

[1906.01195] Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs

Tags:

2020-04-30 A propos

CEUR-WS.org/Vol-2377 - Workshop on Deep Learning for Knowledge Graphs 2019

Tags:

2020-04-29 A propos

[2001.09522] TaxoExpan: Self-supervised Taxonomy Expansion with Position-Enhanced Graph Neural Network

Tags:

how to add a set of new concepts to an existing taxonomy.

[Tweet](https://twitter.com/mickeyjs6/status/1253772146142216194?s=20) [GitHub](https://github.com/mickeystroller/TaxoExpan)

> we study the taxonomy expansion task: given an
existing taxonomy and a set of new emerging concepts, we aim
to automatically expand the taxonomy to incorporate these new
concepts (without changing the existing relations in the given taxonomy).

> To the best of our knowledge, this is the first study on **how to
expand an existing directed acyclic graph (as we model a taxonomy
as a DAG) using self-supervised learning**.

Self-supervised framework, the existing taxonomy being used as training data: it learns a model to predict whether a query concept is the direct hyponym of an anchor concept.

> 2 techniques:
>
> 1. a **position-enhanced graph neural network that encodes the local structure of an anchor concept** in the existing taxonomy,
> 2. a noise-robust training objective that enables the learned model to be insensitive to the label noise in the self-supervision data.

Regarding 1: uses [GNN](/tag/graph_neural_networks.html) to model the "ego network" of concepts (potential “siblings”
and “grand parents” of the query concept).

> Regular
GNNs fail to distinguish nodes with different relative positions to
the query (i.e., some nodes are grand parents of the query while
the others are siblings of the query). To address this limitation, we
present a simple but effective enhancement to inject such position
information into GNNs using position embedding. We show that
such embedding can be easily integrated with existing GNN architectures
(e.g., [GCN](/tag/graph_convolutional_networks) and GAT) and significantly boosts the
prediction performance

Regarding point 2: uses InfoNCE loss, cf. [Contrastive Predictive Coding](/doc/?uri=https%3A%2F%2Farxiv.org%2Fabs%2F1807.03748)

> Instead of predicting
whether each individual ⟨query concept, anchor concept⟩ pair
is positive or not, we first group all pairs sharing the same query
concept into a single training instance and learn a model to select
the positive pair among other negative ones from the group.

(Hum, ça me rappelle quelque chose)

> assume each concept (in existing taxonomy + set of new concepts) has an initial embedding
vector learned from some text associated with this concept.

To keep things tractable, only attempts to find a single parent node of each new concept.

2020-04-25 A propos

BERT, ELMo, & GPT-2: How Contextual are Contextualized Word Representations? | SAIL Blog

Tags:

2020-03-28 A propos

Combining knowledge graphs, quickly and accurately (2020)

Tags:

2020-03-19 A propos

Information Retrieval for HR (2018)

Tags:

2020-02-14 A propos

Siamese CNN for job–candidate matching (slides)

Tags:

2020-02-10 A propos

Matching Resumes to Jobs via Deep Siamese Network | Companion Proceedings of the The Web Conference 2018

Tags:

2020-02-10 A propos

Paris NLP Season 4 Meetup #3 – Paris NLP (2020)

Tags:

2020-01-23 A propos

Semantic Text Matching for Long-Form Documents (2019)

Tags:

2020-01-23 A propos

Advancing Natural Language Processing (NLP) for Enterprise Domains

Tags:

2020-01-07 A propos

Highlights from CoNLL and EMNLP 2019

Tags:

EMNLP 2019

2019-12-07 A propos

Barack’s Wife Hillary: Using Knowledge Graphs for Fact-Aware Language Modeling (ACL 2019)

Tags:

2019-11-17 A propos

Graph Neural Networks for Natural Language Processing tutorial at EMNLP 2019

Tags:

2019-11-08 A propos

Knowledge Graphs and Knowledge modelling took center stage at ISWC 2019 | LinkedIn

Tags:

2019-11-02 A propos

Language and Perception in Deep Learning - Florian Strub DeepMind, Univ. Lille, Inria

Tags:

2019-10-07 A propos

Paris NLP Season 4 Meetup #1 at Algolia

Tags:

2019-10-07 A propos

Evolution of Representations in the Transformer (2019)

Tags:

2019-09-16 A propos

[1908.10084] Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

Tags:

2019-08-28 A propos

France is AI 2018: Lenka Zdeborova - Statistical physics modelling of machine learning - YouTube

Tags:

2019-08-15 A propos

ACL 2019: Highlights and Trends - Maria Khvalchik - Medium

Tags:

ACL 2019

2019-08-13 A propos

4th Workshop on Representation Learning for NLP

Tags:

2019-08-12 A propos

Trends in Natural Language Processing: ACL 2019 In Review - Mihail Eric

Tags:

ACL 2019

2019-08-05 A propos

[1905.07129] ERNIE: Enhanced Language Representation with Informative Entities

Tags:

2019-08-05 A propos

Knowledge graphs in Natural Language Processing @ ACL 2019 - Michael Galkin

Tags:

2019-08-05 A propos

A2N: Attending to Neighbors for Knowledge Graph Inference - ACL 2019

Tags:

2019-07-31 A propos

NAACL 2019 Highlights

Tags:

2019-07-30 A propos

Nominations for ACL 2019 Best Paper Awards - ACL 2019

Tags:

ACL 2019

2019-07-25 A propos

Natural Language Processing for Requirements Engineering: The Best Is Yet to Come

Tags:

2019-07-04 A propos

Papers - ACL 2019

Tags:

NLP conference

2019-06-12 A propos

A Hybrid Graph Model for Distant Supervision Relation Extraction | Springer for Research & Development (ESWC 2019)

Tags:

2019-05-30 A propos

Knowledge-Based Short Text Categorization Using Entity and Category Embedding | Springer for Research & Development (2019)

Tags:

2019-05-30 A propos

Knowledge Graph Technology and Applications 2019 - Simia

Tags:

2019-05-23 A propos

Knowledge Graph Conference 2019, Day 1 - Simia

Tags:

2019-05-09 A propos

International Workshop on Deep Learning for Graphs and Structured Data Embedding

Tags:

2019-04-30 A propos

thunlp/OpenKE: An Open-Source Package for Knowledge Embedding (KE)

Tags:

2019-04-23 A propos

France is AI 2018: Emmanuel Bacry - Detecting weak signals in pharmaco epidemiology - YouTube

Tags:

2019-04-23 A propos

2019 Knowledge Graph Conference - Knowledge Graphs for AI in the Enterprise

Tags:

Knowledge Graph Conference 2019

2019-04-11 A propos

Microsoft Academic

Tags:

2019-02-25 A propos

Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction - ACL Anthology

Tags:

2019-02-09 A propos

Paris NLP Season 3 Meetup #3 at Doctrine – Paris NLP

Tags:

Paris NLP meetup

Trois présentations :

- une première décrivant l'utilisation de données textuelles dans le cadre de la conception d'une opération marketing (définition de l'orientation à donner à l'image d'une marque). Les techniques NLP utilisées sont simples, mais leur utilisation comme outil pour aider les créatifs du marketing est originale.

- une deuxième dans le domaine juridique, très intéressante en termes de techniques mises en oeuvre. L'application vise à la valorisation de bases de contrats (un corpus de textes, sur lequel on souhaite effectuer des recherches complexes), en s'appuyant à la fois sur les techniques récentes de représentation de textes, et sur un knowledge graph (ontologie de termes juridiques). Pour ce qui est de le représentation de textes, ils utilisent Google BERT. Ce que permet BERT, c'est une forme de transfert d'apprentissage : BERT est un réseau de neurones profond entrainé de façon non supervisée, par et chez Google, sur une énorme quantité de textes, de façon à emmagasiner une connaissance sur une langue ("pre-trained language model"). Ces données (c'est à dire ce réseau pré-entrainé) sont mises à disposition par Google. Chacun peut ainsi affiner l'entrainement du réseau sur son propre corpus de textes, et ses propres données labellisées concernant le problème qu'il souhaite effectivement résoudre (par exemple, dans le cas juridique, la reconnaissance d'entités dans les contrats).

Le speaker rapporte des résultats sensiblement améliorés par rapport à ce qu'ils obtenaient auparavant en ce qui concerne la qualité de la représentation de phrases, et des problèmes de type classification de phrases ou reconnaissance d'entités (il note que la représentation de textes longs reste un problème ouvert). Les temps d'entrainement de BERT sur leur problème ne sont pas exorbitants (il parle de quelques heures de GPU, pas de jours ou de semaines de TPU comme pour l'entrainement initial).

- le dernier speaker quant à lui a présenté deux papiers de recherche, justement sur les techniques au cœur de BERT ("Transformer architecture").

2019-01-29 A propos

Romain Vial (Hyperlex) at Paris NLP meetup, slides

Tags:

2019-01-24 A propos

[1901.02860] Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context

Tags:

2019-01-11 A propos

10 Exciting Ideas of 2018 in NLP

Tags:

2018-12-19 A propos

Monolingual data in NMT

Tags:

2018-12-04 A propos

Paris NLP Season 3 Meetup #2 – Paris NLP

Tags:

2018-12-04 A propos

Paris NLP - blog

Tags:

Paris NLP meetup

2018-11-29 A propos

Highlights of EMNLP 2018 – Chris Zhu – Medium

Tags:

EMNLP 2018

2018-11-25 A propos

Interesting Stuff at EMNLP (part II) – Valentin Malykh – Medium

Tags:

EMNLP 2018

2018-11-25 A propos

Interesting Stuff in EMNLP (part I) – Valentin Malykh – Medium

Tags:

EMNLP 2018

2018-11-25 A propos

Google AI Blog: Google at EMNLP 2018

Tags:

2018-11-25 A propos

Associative Multichannel Autoencoder for Multimodal Word Representation (2018)

Tags:

2018-11-15 A propos

EMNLP 2018 Thoughts and Notes · Supernatural Language Processing

Tags:

EMNLP 2018

2018-11-13 A propos

Trying to Understand Recurrent Neural Networks for Language Processing (slides)

Tags:

2018-11-11 A propos

Word Mover's Embedding: From Word2Vec to Document Embedding (2018)

Tags:

2018-11-10 A propos

EMNLP 2018 Highlights: Inductive bias, cross-lingual learning, and more

Tags:

2018-11-08 A propos

Adapting the Neural Encoder-Decoder Framework from Single to Multi-Document Summarization

Tags:

2018-11-06 A propos

Deep Chit-Chat: deep learning for chatbots (EMNLP 2018 Tutorial)

Tags:

by Dr Wei Wu (Microsoft Xiaolce - chatbot with 200 millions users in China) and Dr Rui Yan (Peking Univ)

- Chit-chat (casual, non goal oriented) open-domain. Must be relevant to the context and diverse (informative) to be engaging.
- why creating a chat? to prove an AI can speak like a human, commercial reasons, link to services.

Task oriented vs non task oriented: this tutorial is about the second one.

Retrieval based vs generation based.

Basic knowledge of DL for chatbots:

- word embeddings
- sentence embeddings (CNN, RNN)
- dialogue modeling: seq-to-seq with attention

Response selection for retrieval based chatbots:

- single turn response selection (slides 37-57)
    - framework 1: matching with seq embeddings
    - framework 2: matching with message-response interaction (46)
    - extension of 1: KnowledgeMatching with External Knowledge (53)
    - extension of 2: RepresentationsMatching with Multiple Levels of Representations (54)
    - insights from comparison between 1 and 2 (57)
- multi turn response selection (62)
    - context is now: mess + history
    - again, 2 frameworks

Emerging directions (79):

- matching with better representations
    - Self-Attention (82)
    - fusing multiple types of repr. But how to fuse matters (83)
    - pre-training

Learning a matching model for response selection (84)

Generation based models for chatbots:

- single turn generarion (89)
    - Basic generation model
        - seq2seq
        - Attention
        - Bi-directional modeling
- multi turn generation
    - Contexts are important
    - Context sensitive models
    - Hierarchical context modeling
    - Latent variable modeling
    - Hierarchical memory networks

Diversity in conversations (99)

Content introducing (106)

Additional elements (113)

- Topics in cnversation
- Emotions

Persona in chat:

- Persona
- ...
- Knowledge
- Common sense

RL and Adversarial learning in conversations (125)

Evaluation (132)

Future trends:

- Reasoning in dialogues
- X-grounded dialogues

2018-11-06 A propos

Joint Models in NLP - Slides - Tutorial (EMNLP 2018) - Yue Zhang

Tags:

**Joint models: solve 2 tasks at once.**

Related tasks: POS tagging, NER, chuncking. Pipeline tasks

Motivations:

- reduce error propagation
- information exchange between tasks

Challenges:

- Joint learning
- Search

History: statistical models. 2 kinds:

- Graph-Based Methods
    - Traditional solution:
        - Score each candidate, select the highest-scored output
        - Search-space typically exponential
- Transition-Based Methods
    - Transition-Based systems: Automata
        - State: partial result during decoding, Action: operations that can be applied for state transition
        - Output constructed incrementally

- Deep learning based model
    - Neural transition based models
    - Neural graph-based models
        - Cross task
            - Seminal work: Collobert, Ronan, et al. "Natural language processing (almost) from scratch."
            - Not all tasks are mutually beneficial
            - Ramachandran, et al.  “Unsupervised pretraining for sequence to sequence learning.”
            - Peters, Matthew E., et al. "Deep contextualized word representations." (ELMo)
            - "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding."
            - ULMFIT
            - Correlation between multi-task learning and pretraining
        - Cross lingual
        - Cross domain
        - Cross standard

2018-11-06 A propos

PROCEEDINGS of the BlackboxNLP Workshop

Tags:

BlackboxNLP (2018 workshop)

2018-11-06 A propos

Analyzing and interpreting neural networks for NLP (Workshop's Home page)

Tags:

BlackboxNLP (2018 workshop)

2018-11-06 A propos

Writing code for Natural language processing Research

Tags:

2018-11-05 A propos

Transfer learning with language models

Tags:

2018-11-05 A propos

Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction

Tags:

2018-11-04 A propos

Conference Schedule - EMNLP 2018

Tags:

EMNLP 2018

2018-11-04 A propos

Self-Governing Neural Networks for On-Device Short Text Classification - Sujith Ravi | Zornitsa Kozareva (2018)

Tags:

2018-11-02 A propos

EMNLP (2018) - ACL Anthology - Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

Tags:

EMNLP 2018

2018-11-02 A propos

Deep Latent-Variable Models for Natural Language - Tutorial - harvardnlp

Tags:

2018-11-01 A propos

Facebook Research at EMNLP – Facebook Research

Tags:

2018-11-01 A propos

Trying to Understand Recurrent Neural Networks for Language Processing (tweets)

Tags:

2018-11-01 A propos

Writing Code for NLP Research, AllenNLP's tutorial at #emnlp2018

Tags:

2018-10-31 A propos

Tutorials - EMNLP 2018

Tags:

2018-10-31 A propos

Time-Contrastive Networks: Self-Supervised Learning from Video (2017)

Tags:

Self-supervised approach for learning representations and robotic behaviors entirely from unlabeled videos recorded from multiple viewpoints, and study how this representation can be used in two robotic imitation settings: imitating object interactions from videos of humans, and imitating human poses.

> We train our representations using a metric learning loss, where multiple simultaneous viewpoints of the same observation are attracted in the embedding space, while being repelled from temporal neighbors which are often visually similar but functionally different. In other words, the model simultaneously learns to recognize what is common between different-looking images, and what is different between similar-looking images.
> This signal causes our model to discover attributes that do not change across viewpoint, but do change across time, while ignoring nuisance variables such as occlusions, motion blur, lighting and background. We demonstrate that this representation can be used by a robot to directly mimic human poses without an explicit correspondence, and that it can be used as a reward function within a reinforcement learning algorithm.

2018-10-27 A propos

Teaching Machines to Understand Natural Language (2018)

Tags:

2018-10-26 A propos

Small Data is Big in AI : Train-spotting at France is AI

Tags:

2018-10-26 A propos

Grounded Language Learning: Where Robotics and NLP Meet (IJCAI 2018)

Tags:

2018-10-26 A propos

[1503.08895] End-To-End Memory Networks

Tags:

2018-10-23 A propos

#franceisai, petit récapitulatif des choses vues et entendues ce matin

Tags:

France is AI 2018

2018-10-18 A propos

Time, Context and Causality in Recommender Systems

Tags:

2018-10-18 A propos

Conference - France is AI

Tags:

France is AI 2018

2018-10-15 A propos

Paris NLP Season 3 Meetup #1 | Meetup

Tags:

2018-09-27 A propos

[1809.00782] Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text

Tags:

2018-09-06 A propos

[1601.03764] Linear Algebraic Structure of Word Senses, with Applications to Polysemy

Tags:

> Here it is shown that multiple word senses reside
in linear superposition within the word
embedding and simple sparse coding can recover
vectors that approximately capture the
senses

> Each extracted word sense is accompanied by one of about  2000 “discourse atoms” that gives a succinct description of which other words co-occur with that word sense.

> The success of the approach is mathematically explained using a variant of
the random walk on discourses model

("random walk": a generative model for language). Under the assumptions of this model,  there
exists a linear relationship between the vector of a
word w and the vectors of the words in its contexts (It is not the average of the words in w's context, but in a given corpus the matrix of the linear relationship does not depend on w. It can be estimated, and so we can compute the embedding of a word from the contexts it belongs to)

[Related blog post](/doc/?uri=https%3A%2F%2Fwww.offconvex.org%2F2016%2F07%2F10%2Fembeddingspolysemy%2F)

2018-08-28 A propos

2018 Conference on Empirical Methods in Natural Language Processing - EMNLP 2018

Tags:

EMNLP 2018

2018-08-23 A propos

ACL 2018 Highlights: Understanding Representations and Evaluation in More Challenging Settings - AYLIEN

Tags:

2018-07-26 A propos

Paris NLP Meetup #6

Tags:

2018-07-26 A propos

IJCAI Session Notes: Learning Common Sense · Liza

Tags:

2018-07-23 A propos

Journee:TAL | PFIA 2018

Tags:

2018-07-11 A propos

A Tri-Partite Neural Document Language Model for Semantic Information Retrieval (2018 - ESWC conference)

Tags:

2018-06-08 A propos

Module google/universal-sentence-encoder | TensorFlow

Tags:

2018-05-23 A propos

Improving Word Embedding Compositionality using Lexicographic Definitions

Tags:

2018-05-10 A propos

That Makes Sense: Joint Sense Retrofitting from Contextual and Ontological Information

Tags:

post-processing method for generating low-dimensional sense embedding. Emploies the ontological and contextual information simultaneously.

(Poster at the Web Conf) [Github](https://github.com/y95847frank/Joint-Retrofitting)

Calcule des "sense embeddings", en partant de word embeddings pré-calculés (par ex avec word2vec), et de données de type lexicographiques (ex wordnet), en contraignant, pour un sens, la distance entre sense et word embedding.

Abstract:

> While recent word embedding models demonstrate their abilities to capture syntactic and semantic information, the demand for sense level embedding is getting higher. In this study, we propose a novel joint sense embedding learning model that retrofits the word representation into sense representation from contextual and ontological information. The experiments show the effectiveness and robustness of our model that outperforms previous approaches in four public available benchmark datasets.

> Given a trained word embedding and a lexical ontology that contains sense level relationships (e.g., synonym, hypernym, etc.), our model generates new sense vectors via constraining the distance between the sense vector and its word form vector, its sense neighbors and its contextual neighbors

[Influenced by](/doc/?uri=https%3A%2F%2Farxiv.org%2Fabs%2F1411.4166) (which post-processes and modifies word vectors to incorporate knowledge from semantic lexicons, while this creates new sense vectors)

2018-05-10 A propos

Weakly-supervised Relation Extraction by Pattern-enhanced Embedding Learning

Tags:

2018-05-10 A propos

TUTORIAL: Graph-based Text Representations (SLIDES)

Tags:

2018-05-10 A propos

TUTORIAL: Graph-based Text Representations: Boosting Text Mining, NLP and Information Retrieval with Graphs

Tags:

2018-05-10 A propos

TUTORIAL: Representation Learning on Networks - TheWebConf 2018

Tags:

Network representation learning (NRL): Approaches that automatically learn to encode network structure into low-dimensional embeddings, using techniques based on deep learning and nonlinear dimensionality reduction

**Goal of representation learning for networks: efficient task-independant feature learning for ML in networks.** But it's hard. DL toolbox are designed for single sequences or grids (for instance CNN for images, RNN or word2vec are fixed size), but networks are far more complex!

from the abstract:

> In this tutorial, we will cover key advancements in NRL over the last decade, with an emphasis on fundamental advancements made in the last two years. We will discuss classic matrix factorization-based methods (e.g., Laplacian eigenmaps), random-walk based algorithms (e.g., DeepWalk and node2vec), as well as very recent advancements in graph convolutional networks (GCNs). We will cover methods to embed individual nodes (see [node embeddings](/tag/node_embeddings)) as well as approaches to embed entire (sub)graphs, and in doing so, we will present a unified framework for NRL.

2018-05-05 A propos

Smart-MD: Neural Paragraph Retrieval of Medical Topics

Tags:

2018-04-28 A propos

L’inventeur du Web exhorte à réguler l’intelligence artificielle

Tags:

2018-04-28 A propos

Improving the Compositionality of Word Embeddings (2017)

Tags:

(MS thesis, a [paper at TheWebConf 2018](/doc/?uri=https%3A%2F%2Fdoi.org%2F10.1145%2F3178876.3186007))

> This thesis explores a method to find better encodings of meaning a computer can work with. We specifically want to combine encodings of word meanings in such a way that a good encoding of their joint meaning is created. The act of combining multiple representations of meaning into a new representation of meaning is called semantic composition.

Analysis of four word embeddings (Word2Vec, GloVe, fastText and Paragram)  in terms of their semantic compositionality. A method to tune these embeddings towards better compositionality, using a simple neural network architecture with definitions and lemmas from WordNet.

> Since dictionary definitions are semantically similar to their associated lemmas, they are the ideal candidate for our tuning method, as well as evaluating for compositionality. Our architecture allows for the embeddings to be composed using simple arithmetic operations, which makes these embeddings specifically suitable for production applications such as web search and data mining. We also explore more elaborate and involved compositional models, such as recurrent composition and convolutional composition.

2018-02-13 A propos

Workshop on Deep Learning for Knowledge Graphs and Semantic Technologies

Tags:

2018-02-01 A propos

RESEARCH TRACK: Web Content Analysis, Semantics and Knowledge

Tags:

[CFP](https://www2018.thewebconf.org/call-for-papers/research-tracks-cfp/web-content-analysis/)

> In previous years, ‘content analysis’ and ‘semantic and knowledge’ were in separate track. This year, we combined these tracks to emphasize the close relationship between these topics; **the use of content to curate knowledge and the use of knowledge to guide content analysis and intelligent usage**.

Some of the accepted papers:
### [Large-Scale Hierarchical Text Classification with Recursively Regularized Deep Graph-CNN](https://doi.org/10.1145/3178876.3186005)

[Hierarchical Text Classification](/tag/nlp_hierarchical_text_classification): Text classification to a hierarchical taxonomy of topics, using graph representation of text, and CNN over this graph

Renvoie à ce qui a été vu dans le tutorial "Graph-based Text Representations"

from the abstract:

> a graph-CNN based deep learning model to first convert texts to graph-of-words, and then use graph convolution operations to convolve the word graph. Graph-of-words representation of texts has the advantage of capturing non-consecutive and long-distance semantics. CNN models have the advantage of learning different level of semantics. To further leverage the hierarchy of labels, we regularize the deep architecture with the dependency among labels

Conversion of text to graph: potentially given a single document

### [Weakly-supervised Relation Extraction by Pattern-enhanced Embedding Learning](https://doi.org/10.1145/3178876.3186024 )

Extraction de relations de corpus de textes de façon semi-supervisée, dans un contexte où on a peu de données labellisées décrivant les relations.

Par exemple, des données labellisées indique que le texte "Beijing, capital of China" correspond à la relation entre entités : ("Beijing", "Capital Of", "China), et on voudrait pouvoir extraire les entités et relations pertinentes à partir de texte tel que "Paris, France's capital,..."

Le papier décrit une méthode qui combine deux modules, l'un basé sur l'extraction automatique de patterns (par ex "[Head], Capital Of [Tail]") et l'autre sur la "sémantique distributionnelle" (du type "word embeddings"). Ces deux modules collaborent, le premier permettant de créer des instances de relations augmentant la base de connaissance sur lequel entrainer le second, et le second aidant le premier à déterminer des patterns informatifs ("co-entrainement")

### [Scalable Instance Reconstruction in Knowledge Bases via Relatedness Affiliated Embedding](https://doi.org/10.1145/3178876.3186017)

Knowledge base completion problem: usually, it is formulated as a link prediction problem, but not here. A novel knowledge embedding model ("Joint Modelling and Learning of Relatedness and Embedding")

### [Improving Word Embedding Compositionality using Lexicographic Definitions](https://doi.org/10.1145/3178876.3186007)

comment obtenir les meilleures représentations de texte à partir de représentations de mots (word embeddings) ? L'auteur utilise des ressources lexicographiques (wordnet) pour ses tests : l'embedding obtenu pour la définition d'un mot est-il proche de celui du mot ?

Le papier s'appuie sur une [thèse du même auteur](https://esc.fnwi.uva.nl/thesis/centraal/files/f1554608041.pdf), claire et bien écrite.

### [CESI: Canonicalizing Open Knowledge Bases using Embeddings and Side Information](https://doi.org/10.1145/3178876.3186030)

Amélioration de l'extraction de triplets (nom phrase, property, nom phrase) à partir de texte en calculant des embeddings pour les "nom phrases" (~entités)

### [Short-Text Topic Modeling via Non-negative Matrix Factorization Enriched with Local Word-Context Correlations](https://doi.org/10.1145/3178876.3186009)

Topic modeling for short texts, leveraging the word-context semantic correlations in the training

### [Towards Annotating Relational Data on the Web with Language Models](https://doi.org/10.1145/3178876.3186029)

### A paper by [David Blei](/tag/david_blei): (Dynamic Embeddings for Language Evolution)

2018-01-27 A propos

TUTORIAL: Representation Learning on Networks - TheWebConf 2018

Tags:

2018-01-27 A propos

WORKSHOP: BigNet @ WWW 2018 Workshop on Learning Representations for Big Networks

Tags:

2018-01-27 A propos

Combining word and entity embeddings for entity linking (ESWC 2017)

Tags:

2017-12-30 A propos

Machine Learning with Knowledge Graphs - VideoLectures.NET

Tags:

2014-10-06 A propos

Topic modeling with network regularization

Tags:

2014-04-23 A propos

At SemTechBiz, Knowledge Graphs Are Everywhere - semanticweb.com

Tags:

2013-06-18 A propos

Using SKOS vocabularies for improving Web Search

Tags:

2013-05-30 A propos

SemTechBiz Puts Spotlight On Financial Industry Business Ontology - semanticweb.com

Tags:

2013-04-27 A propos

Describing Customizable Products on the Web of Data, LDOW 2013

Tags:

2013-04-25 A propos

From Linked Data to Linked Entities: a Migration Path - Giovanni Bartolomeo, Stefano Salsano

Tags:

2012-04-20 A propos

LINDEN: Linking Named Entities with Knowledge Base via Semantic Knowledge

Tags:

2012-04-19 A propos

Factorizing Yago: scalable machine learning for the sw

Tags:

2012-04-18 A propos

A Spectrometry of Linked Data

Tags:

Entity mining is still a troublesome open problem. In past years many approaches allowed to automate the generation of equivalence links between references using schema matching or various heuristics based on the recognition of similar property values. In contrast, few of them considered the analysis of the network of equivalence links (“equivalence network”) as an indication of the likelihood and strength of the equivalence.

Could a URI reference (URIRef) be thought as exactly “attached” to its referent? Could it make sense to talk about entity “identifiers” or would it be better to talk about more ambiguous “references”, i.e., placeholders for any model that satisfies the formal semantics of the Semantic Web (Hayes)? Booth observes that the aforementioned question, which in the past has been often regarded as fundamental in the debate about identity on the Web, is relatively unimportant. As long as an entity, identified by whatsoever URIRef, is associated to at least one description containing machine understandable information, this information can be automatically processed and used by applications.

2012-04-16 A propos

NERD meets NIF: Lifting NLP Extraction Results to the Linked Data Cloud

Tags:

2012-04-16 A propos

Automated interlinking of speech radio archives

Tags:

2012-04-14 A propos

Question answering over Linked Data - Interacting with Linked Data

Tags:

2012-01-05 A propos

A Semantic Web Representation of a Product Range Specification based on Constraint Satisfaction Problem in the Automotive Industry

Tags:

2011-10-25 A propos

A Semantic Web Representation of a Product Range Specification based on Constraint Satisfaction Problem in the Automotive Industry

Tags:

2011-08-20 A propos

CSP - A vocabulary to represent Constraint Satisfaction Problems. | DERI Vocabularies

Tags:

2011-06-29 A propos

Networked Knowledge Organization Systems and Services | The 7th European Networked Knowledge Organization Systems (NKOS) Workshop

Tags:

2008-05-06 A propos

COMM: Core Ontology on Multimedia

Tags:

2008-04-23 A propos

Structured Objects in OWL: Representation and Reasoning. In Proc. of the 17th Int. World Wide Web Conference (WWW 2008), Beijing

Tags:

Very good presentation at WWW 2008. Nominated for the best paper award<br/>
Abstract: Applications of semantic technologies often require the representation of and reasoning with structured objects—that is, objects composed of parts connected in complex ways. Although OWL is a general and powerful language, its class descriptions and axioms cannot be used to describe arbitrarily connected structures. An OWL representation of structured objects can thus be underconstrained, which reduces the inferences that can be drawn and causes performance problems in reasoning. To address these problems, we extend OWL with description graphs, which allow for the description of structured objects in a simple and precise way. To represent conditional aspects of the domain, we also allow for SWRL-like rules over description graphs. Based on an observation about the nature of structured objects, we ensure decidability of our formalism. We also present a hypertableau-based decision procedure, which we implemented in the HermiT reasoner. To evaluate its performance, we have extracted description graphs from the GALEN and FMA ontologies, classified them successfully, and even detected a modeling error in GALEN.

2008-04-23 A propos

chillyinside.com » Blog Archive » 2006 Jena User Conference

Tags:

2006-05-26 A propos

Jena User Conference - some interesting UI related papers/presentations from Shabajee, Paul on 2006-05-17 (public-semweb-ui@w3.org from May 2006)

Tags:

2006-05-25 A propos