Semanlink - sl:creationDate : 2023-09-28

Search Tag:

Search Doc:

Preferences...

Tag Cloud

Alibaba - Arxiv Doc - Guillaume Lample - Jerry Liu - LLM + KG - NLP@Amazon - Open-source LLM - Text to SQL - Tutorial -

sl:creationDate : 2023-09-28

Finetuning LLaMa + Text-to-SQL

Tags:

2023-09-28 About

Guillaume Lample sur X : "Mistral 7B is out. It outperforms Llama 2 13B on every benchmark we tried..."

Tags:

2023-09-28 About

[2309.15427] Graph Neural Prompting with Large Language Models

Tags:

> Can we learn beneficial knowledge from KGs
and integrate them into pre-trained LLMs?

> we propose to
leverage the factual knowledge from KGs to enhance LLMs,
while still benefiting from circumventing the burdensome
training expenses by using pre-trained LLMs

> Graph Neural Prompting
(GNP), a plug-and-play method to assist pre-trained
LLMs in learning beneficial knowledge from KGs
>
> GNP
encodes the pertinent grounded knowledge and complex
structural information to derive Graph Neural Prompt, an
embedding vector that can be sent into LLMs to provide
guidance and instructions

> - GNP first utilizes
a GNN to capture and encode the
intricate graph knowledge into **entity/node embeddings**. 
> - Then,
a cross-modality pooling module is present to determine
the **most relevant node embeddings in relation to the text
input**, and consolidate these node embeddings into **a holistic
graph-level embedding**.
> - After that, GNP encompasses a
**domain projector** to bridge the inherent disparities between
the graph and text domains.
> - Finally, a **self-supervised link
prediction objective** is introduced to enhance the model
comprehension of relationships between entities and capture
graph knowledge in a self-supervised manner.

2023-09-28 About

Yam Peleg sur X : "Qwen-14B (Alibaba) The most powerful open-source model for it's size. And the longest trained: 3T tokens..."

Tags:

2023-09-28 About