NLP@Amazon
http://www.semanlink.net/tag/nlp_amazon
Documents tagged with NLP@Amazon[2309.15427] Graph Neural Prompting with Large Language Models
http://www.semanlink.net/doc/2023/09/2309_15427_graph_neural_promp
> Can we learn beneficial knowledge from KGs
and integrate them into pre-trained LLMs?
> we propose to
leverage the factual knowledge from KGs to enhance LLMs,
while still benefiting from circumventing the burdensome
training expenses by using pre-trained LLMs
> Graph Neural Prompting
(GNP), a plug-and-play method to assist pre-trained
LLMs in learning beneficial knowledge from KGs
>
> GNP
encodes the pertinent grounded knowledge and complex
structural information to derive Graph Neural Prompt, an
embedding vector that can be sent into LLMs to provide
guidance and instructions
> - GNP first utilizes
a GNN to capture and encode the
intricate graph knowledge into **entity/node embeddings**.
> - Then,
a cross-modality pooling module is present to determine
the **most relevant node embeddings in relation to the text
input**, and consolidate these node embeddings into **a holistic
graph-level embedding**.
> - After that, GNP encompasses a
**domain projector** to bridge the inherent disparities between
the graph and text domains.
> - Finally, a **self-supervised link
prediction objective** is introduced to enhance the model
comprehension of relationships between entities and capture
graph knowledge in a self-supervised manner.
2023-09-28T08:52:07ZUsing graph neural networks to recommend related products - Amazon Science
http://www.semanlink.net/doc/2022/10/using_graph_neural_networks_to_
2022-10-18T08:08:40Z[2004.05119] Beyond Fine-tuning: Few-Sample Sentence Embedding Transfer
http://www.semanlink.net/doc/2022/03/2004_05119_beyond_fine_tuning
> Fine-tuning (FT) pre-trained sentence embedding models on small datasets has been shown to have limitations. In this paper we show that concatenating the embeddings from the pre-trained model with those from a simple sentence embedding model trained only on the target data, can improve over the performance of FT for few-sample tasks
2022-03-31T21:04:02ZDomain Adaptation with BERT-based Domain Classification and Data Selection - ACL Anthology (2019)
http://www.semanlink.net/doc/2022/03/domain_adaptation_with_bert_bas
2022-03-16T17:36:19Z[2110.10778] Contrastive Document Representation Learning with Graph Attention Networks
http://www.semanlink.net/doc/2022/03/2110_10778_contrastive_docume
> most of the pretrained
Transformers models can only handle relatively
short text. It is still a challenge when it
comes to modeling very long documents. In
this work, we propose to use a graph attention
network on top of the available pretrained
Transformers model to learn document embeddings
2022-03-10T13:54:40Z[2004.11892] Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering
http://www.semanlink.net/doc/2022/02/2004_11892_template_based_que
[Gihub](doc:2021/12/awslabs_unsupervised_qa_templa)
> we expand
upon the recently introduced task of unsupervised
question answering ([Lewis et al., 2019, Unsupervised Question Answering by Cloze Translation](doc:2021/12/1906_04980_unsupervised_quest)) to
examine the extent to which synthetic training data
alone can be used to train a QA model.
focus on extractive, **factoid QA, where answers are named entities** -> focus on creating a relevant question from a
(context, answer) pair in an unsupervised manner
> We improve over [Lewis et al, 2019] by proposing a simple, intuitive, retrieval
and template-based question generation
approach
>
> Question Generation Pipeline: the original
context sentence containing a given answer is used as
a query to retrieve a related sentence containing matching
entities, which is input into our question-style converter
to create QA training data.
2022-02-11T14:06:18Zawslabs/unsupervised-qa: Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering
http://www.semanlink.net/doc/2021/12/awslabs_unsupervised_qa_templa
Code and synthetic data from our [ACL 2020 paper](doc:2022/02/2004_11892_template_based_que)
> We propose an unsupervised approach to training QA models with generated pseudo-training data. We show that generating questions for QA training by applying a simple template on a related, retrieved sentence rather than the original context sentence improves downstream QA performance by allowing the model to learn more complex context-question relationships.
2021-12-08T00:51:21ZX-BERT: eXtreme Multi-label Text Classification using Bidirectional Encoder Representations from Transformers
http://www.semanlink.net/doc/2021/01/x_bert_extreme_multi_label_tex
> Challenges in extending BERT to the XMC problem:
- difficulty of capturing [dependencies or correlations among labels](tag:classification_relations_between_classes.html)
- tractability to scale to the extreme label setting because of the Softmax bottleneck scaling linearly with the output space.
> X-BERT leverages both the label and input text to build label representations, which induces semantic label clusters to better model label dependencies. At the heart of X-BERT is a procedure to finetune BERT models to capture the contextual relations between input text and the induced label clusters. Finally, an ensemble of the different BERT models trained on heterogeneous label clusters leads to our best final mode
2021-01-10T19:23:20Z