Semanlink - AI teams ; Arxiv Doc AND LLM

[2404.11018] Many-Shot In-Context Learning

Tags:

2024-04-21 About

[2307.15936] A Theory for Emergence of Complex Skills in Language Models

Tags:

2024-02-24 About

[2310.01352] RA-DIT: Retrieval-Augmented Dual Instruction Tuning

Tags:

2023-10-07 About

[2309.15427] Graph Neural Prompting with Large Language Models

Tags:

> Can we learn beneficial knowledge from KGs
and integrate them into pre-trained LLMs?

> we propose to
leverage the factual knowledge from KGs to enhance LLMs,
while still benefiting from circumventing the burdensome
training expenses by using pre-trained LLMs

> Graph Neural Prompting
(GNP), a plug-and-play method to assist pre-trained
LLMs in learning beneficial knowledge from KGs
>
> GNP
encodes the pertinent grounded knowledge and complex
structural information to derive Graph Neural Prompt, an
embedding vector that can be sent into LLMs to provide
guidance and instructions

> - GNP first utilizes
a GNN to capture and encode the
intricate graph knowledge into **entity/node embeddings**. 
> - Then,
a cross-modality pooling module is present to determine
the **most relevant node embeddings in relation to the text
input**, and consolidate these node embeddings into **a holistic
graph-level embedding**.
> - After that, GNP encompasses a
**domain projector** to bridge the inherent disparities between
the graph and text domains.
> - Finally, a **self-supervised link
prediction objective** is introduced to enhance the model
comprehension of relationships between entities and capture
graph knowledge in a self-supervised manner.

2023-09-28 About

[2309.12307] LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models

Tags:

2023-09-26 About

[2306.04640] ModuleFormer: Modularity Emerges from Mixture-of-Experts

Tags:

2023-09-16 About

[2307.08621] Retentive Network: A Successor to Transformer for Large Language Models

Tags:

2023-07-20 About

[2305.14128] Dr.ICL: Demonstration-Retrieved In-context Learning

Tags:

2023-07-14 About

[2305.07185] MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers

Tags:

2023-07-01 About

[2303.17651] Self-Refine: Iterative Refinement with Self-Feedback

Tags:

2023-04-03 About

[2212.09741] One Embedder, Any Task: Instruction-Finetuned Text Embeddings

Tags:

2023-02-17 About

[2302.04761] Toolformer: Language Models Can Teach Themselves to Use Tools

Tags:

> Toolformer, **a model
trained to decide which APIs to call, when to
call them, what arguments to pass, and how to
best incorporate the results into future token
prediction**.

> fulfills the
following desiderata:
> - The use of tools should be learned in a
self-supervised way without requiring large
amounts of human annotations
>- The LM should be able to decide for itself when
and how to use which tool.

> Approach based
on the recent idea of using large LMs with incontext
learning (Brown et al., 2020) to generate
entire datasets from scratch.
>
> Given just a handful of human-written examples
of how an API can be used, 
> - we let a LM annotate
a huge language modeling dataset with potential
API calls. 
> - We then use a self-supervised loss to
determine which of these API calls actually help
the model in predicting future tokens. 
>- Finally, we
finetune the LM itself on the API calls that it considers
useful.

[Jay Hack @mathemagic1an sur twitter](https://twitter.com/mathemagic1an/status/1624870248221663232):

> from a small seed set of human inputs (essentially demonstrating usage of APIs), the training set for this behavior is generated by the LLM itself.
> 
> So what does this mean? We've found a promising way to tightly integrate arbitrary APIs with our best-performing models.

2023-02-13 About

[2203.14465] STaR: Bootstrapping Reasoning With Reasoning

Tags: