Semanlink - IBM AND Deep Learning

Ben Hoover sur X : "Energy Transformer (ET): A novel architecture combining 3 prominent ideas in AI..."

Tags:

2023-12-02 About

[2306.04640] ModuleFormer: Modularity Emerges from Mixture-of-Experts

Tags:

2023-09-16 About

[2010.00711] A Survey of the State of Explainable AI for Natural Language Processing

Tags:

2022-09-08 About

Active Learning for BERT: An Empirical Study - ACL Anthology

Tags:

2022-09-02 About

Advancing Natural Language Processing (NLP) for Enterprise Domains

Tags:

2020-01-07 About

[1909.04120] Span Selection Pre-training for Question Answering

Tags:

> a **new pre-training task inspired by reading
comprehension** and an **effort to avoid encoding general knowledge in the transformer network itself**

Current transformer architectures store general knowledge -> large models, long pre-training time. Better to offload the requirement of general knowledge to a sparsely activated network.

"Span selection" as an additional auxiliary task: the query is a sentence drawn from a corpus
with a term replaced with a special token: [BLANK]. The term replaced by the blank is the answer term. The passage is
relevant as determined by a BM25 search, and answer-bearing (containing the answer
term). Unlike BERT’s cloze task, where the answer must be drawn from the model itself, the answer is found in a passage
using language understanding.

> **We hope to progress to a model of general purpose language modeling that uses an indexed long
term memory to retrieve world knowledge, rather than holding it in the densely activated transformer encoder layers.**

2019-09-18 About

IBM's 'Rodent Brain' Chip Could Make Our Phones Hyper-Smart | WIRED

Tags:

2015-08-18 About