Semanlink - Chris Manning

1. new neural tensor
network (**NTN**) suitable for reasoning over relationships between two entities. Generalizes several previous neural network models and provides a more
powerful way to model relational information than a standard neural network layer.
2. a new way to represent entities in knowledge bases, as the
average of  their constituting word vectorss, allowing the sharing of statistical strength between the words describing
each entity (e.g., Bank of China and China).
3. incorporation of word vectors which are trained on large unlabeled text

> We **learn to modify word representations
via grounding in world knowledge**. This essentially allows us to analyze word embeddings and
query them for specific relations. Furthermore, the resulting vectors could be used in other tasks
such as named entity recognition or relation classification in natural language

**Makes use of entity name**: NTN
first learns word vectors from an auxiliary news corpus, and
then initializes the representation of each entity by averaging
the vectors of words contained in its name. For example,
**the embedding of AlfredHitchcock is initialized by the
average word vectors of “alfred” and “hitchcock”**... This kind of methods model textual information
separately from KG facts, and hence fail to leverage
interactions between them.[src](doc:2019/05/knowledge_graph_embedding_a_su)

2019-08-03 About

[1907.03950] Learning by Abstraction: The Neural State Machine

Tags:

2019-07-10 About

[1906.04341] What Does BERT Look At? An Analysis of BERT's Attention

Tags:

2019-06-21 About

A Structural Probe for Finding Syntax in Word Representations

Tags:

2019-06-10 About

Language, trees, and geometry in neural networks

Tags:

Chris Manning

2019-06-09 About

Effect of Non-linear Deep Architecture in Sequence Labeling

Tags:

2018-10-22 About

Representations for Language: From Word Embeddings to Sentence Meanings (2017) - Slides

Tags:

2018-08-28 About

Representations for Language: From Word Embeddings to Sentence Meanings (2017) - YouTube

Tags:

2018-05-29 About

Christopher Manning - "Building Neural Network Models That Can Reason" (TCSDLS 2017-2018) - YouTube

Tags:

2018-04-18 About

Deep Learning, Structure and Innate Priors - A Discussion between Yann LeCun and Christopher Manning | Abigail See

Tags:

2018-02-25 About

Distributed Word Representations for Information Retrieval

Tags:

2017-10-01 About

CS224n: Natural Language Processing with Deep Learning

Tags:

2017-09-10 About

Latent semantic indexing ("Introduction to Information Retrieval" Manning 2008)

Tags:

VSM : problem with synonymy and polysemy (eg. synonyms are accorded separate dimensions)

Could we use the co-occurrences of terms to capture the latent semantic associations of terms and alleviate these problems?

Concluding remarks:

- computational cost of the SVD is significant
    - biggest obstacle to the widespread adoption to LSI.
    - One approach to this obstacle: build the LSI representation on a randomly sampled subset of the documents, following which the remaining documents are ``folded in'' (cf Gensim tutorial "[Random Projection (used as an option to speed up LSI)](https://radimrehurek.com/gensim/models/rpmodel.html)")
- As we reduce k, recall tends to increase, as expected.
- **Most surprisingly**, a value of k in the low hundreds can actually increase precision. **This appears to suggest that for a suitable value of *k*, LSI addresses some of the challenges of synonymy**.
- LSI works best in applications where there is little overlap between queries and documents. (--??)

The experiments also documented some modes where LSI failed to match the effectiveness of more traditional indexes and score computations.

LSI shares two basic drawbacks of vector space retrieval:
    
- no good way of expressing negations
- no way of enforcing Boolean conditions.

LSI can be viewed as soft clustering by interpreting each dimension of the reduced space as a cluster and the value that a document has on that dimension as its fractional membership in that cluster.

2017-07-19 About

GloVe: Global Vectors for Word Representation

Tags:

2017-05-18 About

Computational Linguistics and Deep Learning

Tags:

2016-09-10 About

Natural Language Understanding-focused awards announced

Tags:

2013-07-10 About

Maxent models, Conditional estimation and Optimization

Tags: