Semanlink - Enseignement supérieur

Parents:

Education

Enseignement supérieur

Related Tags:

Descendants

Ministère de l'enseignement supérieur et de la recherche
Normale Sup
- NLP@ENS
Universités américaines
- Berkeley
- Cornell
- Harvard
  - Harvard Business Review
  - NLP@Harvard
- MIT
  - NLP@MIT
  - SIMILE
    - Piggy Bank
    - SIMILE Exhibit
    - SIMILE Timeline
- Princeton
  - NLP@Princeton
    - Sanjeev Arora
      - A La Carte Embedding
        
        A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors (2018)
        
        Tags:
        
        A La Carte Embedding
        
        2018-09-18 About
        
        Simple and efficient semantic embeddings for rare words, n-grams, and language features – Off the convex path
        
        Tags:
        
        A La Carte Embedding
        Good
        N-grams
        Rare words (NLP)
        Sanjeev Arora
        Sentence Embeddings
        Word embeddings
        
        A La Carte embeddings > Distributional methods for capturing meaning, such as word embeddings, often require observing many examples of words in context. But most humans can infer a reasonable meaning from very few or even a single occurrence... > > “Porgies live in shallow temperate marine waters” > > Inducing word embedding from their contexts: a surprising linear relationship
        
        2018-09-18 About
      - SIF embeddings
        
        A Self-Training Approach for Short Text Clustering - (Hadifar 2019)
        
        Tags:
        
        Short Text Clustering
        SIF embeddings
        
        > The method we propose, learns discriminative features from both an autoencoder and a sentence embedding ([SIF embeddings](tag:sif_embeddings)), then uses assignments from a clustering algorithm as supervision to update weights of the encoder network.
        
        2021-05-20 About
        
        [1703.02507] Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features
        
        Tags:
        
        Arxiv Doc
        Sent2Vec
        SIF embeddings
        
        2019-03-25 About
        
        nlp-notebooks/Simple Sentence Similarity.ipynb at master · nlptown/nlp-notebooks
        
        Tags:
        
        NLP sample code
        Sentence Similarity
        SIF embeddings
        Word Mover’s Distance
        Yves Peirsman
        
        [blog post](/doc/?uri=http%3A%2F%2Fnlp.town%2Fblog%2Fsentence-similarity%2F)
        
        2019-02-07 About
        
        Unsupervised Random Walk Sentence Embeddings: A Strong but Simple Baseline (Ethayarajh 2018)
        
        Tags:
        
        Random walk
        Sentence Embeddings
        SIF embeddings
        
        > we show that word vector length has a confounding effect on the probability of a sentence being generated in Arora et al.’s model ([SIF embeddings](tag:sif_embeddings)). We propose a random walk model that is robust to this confound... Our approach beats Arora et al.’s by up to 44.4% on textual similarity tasks... Unlike Arora et al.’s method, ours requires no hyperparameter tuning [Github](https://github.com/kawine/usif)
        
        2018-10-08 About
        
        Deep-learning-free Text and Sentence Embedding, Part 2 – Off the convex path
        
        Tags:
        
        Sanjeev Arora
        Sentence Embeddings
        SIF embeddings
        
        > Can we design a text embedding with the simplicity and transparency of SIF while also incorporating word order information? yes we can.
        
        2018-06-25 About
        
        Deep-learning-free Text and Sentence Embedding, Part 1 – Off the convex path
        
        Tags:
        
        Sanjeev Arora
        Sentence Embeddings
        SIF embeddings
        
        > introduction to extremely simple ways of computing sentence embeddings, which on many standard tasks, beat many state-of-the-art deep learning methods. Related to [this paper](/doc/?uri=https%3A%2F%2Fopenreview.net%2Fforum%3Fid%3DSyK00v5xx) (BTW, contains a good intro to text embeddings)
        
        2018-06-25 About
        
        A Simple but Tough-to-Beat Baseline for Sentence Embeddings (2017)
        
        Tags:
        
        Good
        Sanjeev Arora
        Sentence Embeddings
        SIF embeddings
        Singular Value Decomposition
        
        > Use word embeddings computed using one of the popular methods on unlabeled corpus like Wikipedia, represent the sentence by a weighted average of the word vectors, and then modify them a bit using PCA/SVD [github project](https://github.com/PrincetonML/SIF) [blog post](/doc/?uri=http%3A%2F%2Fwww.offconvex.org%2F2018%2F06%2F17%2Ftextembeddings%2F) See also [youtube: Sanjeev Arora on "A theoretical approach to semantic representations"](https://www.youtube.com/watch?v=KR46z_V0BVw)
        
        2018-05-10 About
    - Wordnet
  - Wordnet
- Stanford
  - AI@Stanford
    - NLP@Stanford
      - Alpaca
        
        How to use Alpaca-LoRA to fine-tune a model like ChatGPT
        
        Tags:
        
        Alpaca
        ChatGPT
        ChatGPT fine-tuning
        LoRA
        
        2023-05-26 About
        
        Daniel Vila Suero sur Twitter : "Data quality is key for LLMs, but we're building Open Source LLMs with data of "unknown" quality... Introducing Alpaca GarbageCollector..."
        
        Tags:
        
        Alpaca
        Dataset quality
        SetFit (SBERT fine-tuning)
        Tweet
        
        > a cross-lingual SetFit model to identify potential bad instructions in Alpaca-like datasets
        
        2023-04-05 About
        
        Alpaca Finetuning of Llama on a 24G Consumer GPU
        
        Tags:
        
        Alpaca
        Fine-tuning
        
        [GitHub](https://github.com/aspctu/alpaca-lora) fork of [tloen/alpaca-lora: Instruct-tune LLaMA on consumer hardware](doc:2023/03/tloen_alpaca_lora_instruct_tun)
        
        2023-03-27 About
        
        tloen/alpaca-lora: Instruct-tune LLaMA on consumer hardware
        
        Tags:
        
        Alpaca
        GitHub project
        GPT: frugal alternatives
        LLaMA
        
        Uses [LoRA: Low-Rank Adaptation of Large Language Models](doc:2023/03/2106_09685_lora_low_rank_ada) see [Alpaca Finetuning of Llama on a 24G Consumer GPU](doc:2023/03/alpaca_finetuning_of_llama_on_a)
        
        2023-03-22 About
        
        LLM Zoo at Home: LLaMA & Alpaca | bergis universe of software, hardware and ideas
        
        Tags:
        
        Alpaca
        LLaMA
        
        2023-03-20 About
      - ColBERT
        
        ColBERT gist:c1182551fa609736d47df4af82f7c5ab
        
        Tags:
        
        ColBERT
        Sample code
        
        > a quick gist that does synthetic data gen, fine-tuning, eval. Just add your own documents, or try it on a PG essay. @JoshPurtell
        
        2024-03-08 About
        
        ColBERT Inference in the Browser
        
        Tags:
        
        ColBERT
        Demo
        Vespa
        
        Demo of ColBERT query-passage scoring interpretability - try with the following: "what are the mentioned EICPS?" and passage "There is a security risk related to EICPS 67" - MaxSim Score: 20.71 - Estimated Relevance: 64.71% - highlights: There related - then "what are the mentioned animals?" and "There is a security risk related to lions" - MaxSim Score: 9.18 - Estimated Relevance: 28.68% - highlights: related lions ``` Effects of climate change on marine ecosystems MaxSim Score: 27.90 Estimated Relevance: 87.17% Effects of global warming on marine ecosystems MaxSim Score: 24.62 Estimated Relevance: 76.94% Effects of global warming on life in the oceans MaxSim Score: 19.64 Estimated Relevance: 61.39% Effects of global warming on life on Mars MaxSim Score: 13.65 Estimated Relevance: 42.65% ```
        
        2024-03-08 About
        
        Announcing Vespa Long-Context ColBERT | Vespa Blog
        
        Tags:
        
        @jobergum
        ColBERT
        Long documents
        Vespa
        
        2024-03-03 About
        
        Omar Khattab sur X : "ColBERT in 81 languages by generalizing from English training! ..."
        
        Tags:
        
        ColBERT
        Multilingual NLP
        Omar Khattab
        Tweet
        
        2024-02-28 About
        
        Omar Khattab sur X : "A thread on late interaction, how it works efficiently, and why/where it's been shown to improve quality"
        
        Tags:
        
        ColBERT
        Omar Khattab
        Twitter thread
        
        2024-02-05 About
        
        Craig Macdonald sur X : "a thread of our main ColBERT research findings"
        
        Tags:
        
        ColBERT
        Twitter thread
        
        2024-02-01 About
        
        long document ranking · Issue #11 · stanford-futuredata/ColBERT
        
        Tags:
        
        ColBERT
        Long documents
        
        [Instructions on using ColBERT](https://github.com/stanford-futuredata/ColBERT/issues/73) passage and query length, etc
        
        2024-01-31 About
        
        Exploring ColBERT with RAGatouille | Simon Willison’s TILs
        
        Tags:
        
        ColBERT
        RAGatouille
        
        2024-01-29 About
        
        Jo Kristian Bergum sur X : (on "why using ColBERT?")
        
        Tags:
        
        @jobergum
        ColBERT
        Twitter thread
        
        > The idea that you can accurately boil down the nuances of ~256 tokens (2/3rds of a page) into a single vector is a pretty wild proposition > Inspired by @lateinteraction , we hacked into the ColBERT model's contextualized late-interaction similarities to produce [interpretable snippets](https://x.com/jobergum/status/1750282246072746178?s=20)!
        
        2024-01-28 About
        
        bclavie/RAGatouille
        
        Tags:
        
        Benjamin Clavié
        ColBERT
        GitHub project
        RAG (Retrieval-Augmented Generation)
        RAGatouille
        
        > RAGatouille's purpose is make it easy to use state-of-the-art methods in your RAG pipeline, without having to worry about the details or the years of literature! At the moment, RAGatouille focuses on making ColBERT simple to use. [Using ColBERT in-memory: Index-Free Encodings & Search](https://github.com/bclavie/RAGatouille/blob/0.0.5b1/examples/06-index_free_use.ipynb) ``` from ragatouille import RAGPretrainedModel RAG = RAGPretrainedModel.from_pretrained "colbert-ir/colbertv2. 0" ) # Your documents, a plain old list of chunked strings. documents = [...] # In-memory indexing supports metadata too! meta = ['attribute': ' really cool value'}...] # All the magic happens here RAG.encode documents, document_metadatas=meta) # Query your in-memory index RAG. search_encoded_docs(query = "A great question", k=3) # All further encode() calls add to the existing documents... RAG.encode(extra_documents, document_metadatas=extra_meta) # ... until you clear them RAG.clear_encoded ```
        
        2024-01-26 About
        
        Omar Khattab sur X : "A🧵on beating the hardware lottery for retrieval: the internals of the late interaction stack. ColBERT..."
        
        Tags:
        
        ColBERT
        Omar Khattab
        Twitter thread
        
        2023-12-29 About
        
        raphaelsty/neural-cherche: Neural Search
        
        Tags:
        
        Cherche (Raphaël)
        ColBERT
        GitHub project
        Neural Search
        Raphaël Sourty
        SPLADE
        
        > a library to fine-tune neural search models such as Splade, ColBERT, and SparseEmbed on a specific dataset
        
        2023-11-17 About
        
        Omar Khattab sur X : "This isn't the main point of this great new paper by @sophiaalthammer et al. But it's incredible how ColBERT at 1000 training queries is better than DPR trained at *50,000* queries!"
        
        Tags:
        
        ColBERT
        Omar Khattab
        Tweet
        
        mentions [Annotating Data for Fine-Tuning a Neural Ranker? Current Active Learning Strategies are not Better than Random Selection](doc:2023/09/2309_06131_annotating_data_fo) > why is ColBERT so data-efficient? > Answer: > > - ColBERT neither needs to learn how to condense each document (unlike DPR) > - nor how to do matching (unlike MonoBERT). > - Just needs to learn contextual term representations—a much lower burden on the encoders.
        
        2023-09-14 About
        
        LangChain Retrieval Webinar - YouTube
        
        Tags:
        
        ColBERT
        DSPy (Demonstrate-Search-Predict)
        LangChain
        Omar Khattab
        Webinar
        YouTube video
        
        [ColBERT](tag:colbert) retrieval model and the [DSP](doc:2023/02/stanfordnlp_dsp_𝗗𝗦𝗣_demons) programming model
        
        2023-05-27 About
        
        [2304.01982] Rethinking the Role of Token Retrieval in Multi-Vector Retrieval
        
        Tags:
        
        Arxiv Doc
        ColBERT
        Multi-Vector Retrieval
        Neural Search
        NLP@Google
        
        > Multi-vector retrievers like [ColBERT](tag:colbert) are powerful, but they come at the cost of complicated inference. In this paper, we ask: "can token retrieval alone achieve great performance in multi-vector retrieval?" [tweet](https://twitter.com/leejnhk/status/1643632578824396805?s=20) > The key insight of XTR is that the token-retrieval in multi-vector models should be **trained to retrieve the most salient and informative document tokens**, so that the score between a query and document can be computed using only the retrieved information, just like how single-vector retrieval models work > This is an *amazing* way to re-engineer the scoring mechanism of late interaction / ColBERT retrievers! [src: ColBERT's author Omar Khattab](https://twitter.com/lateinteraction/status/1643439889902637056?s=20) - scoring using only retrieved document terms - imputing missing token scores using their upper bound
        
        2023-04-05 About
        
        Omar Khattab sur Twitter : "New features in 𝗖𝗼𝗹𝗕𝗘𝗥𝗧 ‣ Index Updater. Simple API to add/remove documents..."
        
        Tags:
        
        ColBERT
        Tweet
        
        2023-01-30 About
        
        [2203.13088] Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reduction
        
        Tags:
        
        Arxiv Doc
        ColBERT
        
        [tweet](https://twitter.com/s_hofstaetter/status/1508803785317138435)
        
        2022-03-30 About
        
        Integrate ORQA and REALM for Open Domain Question Answering · Issue #312 · deepset-ai/haystack
        
        Tags:
        
        ColBERT
        Haystack
        REALM
        
        mntions [[2002.08909] REALM: Retrieval-Augmented Language Model Pre-Training](doc:2020/12/2002_08909_realm_retrieval_a)
        
        2022-01-12 About
        
        [2004.12832] ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
        
        Tags:
        
        Arxiv Doc
        ColBERT
        
        (The 1st Colbert paper)
        
        2022-01-12 About
        
        [2007.00814] Relevance-guided Supervision for OpenQA with ColBERT
        
        Tags:
        
        Arxiv Doc
        ColBERT
        
        Colbert-QA
        
        2022-01-07 About
        
        [2112.01488] ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction
        
        Tags:
        
        Arxiv Doc
        ColBERT
        Domain specific corpora
        
        2021-12-05 About
        
        Building Scalable, Explainable, and Adaptive NLP Models with Retrieval | SAIL Blog
        
        Tags:
        
        ColBERT
        Good
        NLP@Stanford
        Not Encoding Factual Knowledge in Language Model
        Retrieval augmented LM
        Retrieval-based NLP
        
        > The black-box nature of large language models like T5 and GPT-3 makes them inefficient to train and deploy, opaque in their knowledge representations and in backing their claims with provenance, and static in facing a constantly evolving world and diverse downstream contexts. **This post explores retrieval-based NLP, where models retrieve information pertinent to solving their tasks from a plugged-in text corpus**. > > Retrieval-based NLP methods view tasks as “open-book” exams: knowledge is encoded explicitly in the form of a text corpus like Wikipedia, the medical literature, or a software’s API documentation. When solving a language task, **the model learns to search for pertinent passages** and to then use the retrieved information for crafting knowledgeable responses. In doing so, **retrieval helps decouple the capacity that language models have for understanding text from how they store knowledge**
        
        2021-10-07 About
        
        A Moderate Proposal for Radically Better AI-powered Web Search
        
        Tags:
        
        ColBERT
        Neural Search
        NLP@Stanford
        Web search
        
        2021-07-10 About
      - Coursera: NLP class
        
        Introduction to Information Retrieval, Cambridge University Press (2008) Manning, Raghavan, and Schütze
        
        Tags:
        
        AI: books & journals
        Chris Manning
        Coursera: NLP class
        Good
        Information retrieval
        Information retrieval: techniques
        NLP@Stanford
        
        2012-04-01 About
        
        SRILM - The SRI Language Modeling Toolkit
        
        Tags:
        
        Coursera: NLP class
        Language Model
        NLP tools
        
        SRILM is a toolkit for building and applying statistical language models (LMs), primarily for use in speech recognition, statistical tagging and segmentation, and machine translation.
        
        2012-03-24 About
        
        Unix for Poets
        
        Tags:
        
        Coursera: NLP class
        Unix
        
        2012-03-19 About
        
        Porter Stemming Algorithm
        
        Tags:
        
        Coursera: NLP class
        Stemming
        
        2012-03-18 About
        
        Regular-Expressions.info - Regex Tutorial, Examples and Reference - Regexp Patterns
        
        Tags:
        
        Coursera: NLP class
        Regex
        
        2012-03-07 About
      - cs224n
        
        Prompting, Instruction Finetuning, and RLHF (CS224N)
        
        Tags:
        
        ChatGPT
        ChatGPT: training
        cs224n
        Instruction tuning
        Prompting/In-context learning
        RL from Human Feedback
        Slides
        
        2023-02-16 About
        
        cs224n Lecture 13: Integrating Knowledge in Language Models
        
        Tags:
        
        cs224n
        Language Models + Knowledge
        Slides
        
        2022-07-08 About
        
        Softmax classifier (CS231n Convolutional Neural Networks for Visual Recognition)
        
        Tags:
        
        Andrej Karpathy
        Cross-Entropy
        cs224n
        NLP@Stanford
        Softmax
        
        > the cross-entropy objective wants the predicted distribution to have all of its mass on the correct answer.
        
        2020-06-04 About
        
        CS224n: Natural Language Processing with Deep Learning Stanford / Winter 2019
        
        Tags:
        
        cs224n
        NLP@Stanford
        
        2019-09-30 About
        
        CS224n: Natural Language Processing with Deep Learning
        
        Tags:
        
        cs224n
        NLP@Stanford
        
        2019-04-03 About
      - GloVe
        
        I.A.B sur Twitter : "When we "know the meaning" of a word, what is it that we know? For example, what does knowing the words "dolphin" and "tiger" entail?..."
        
        Tags:
        
        GloVe
        Meaning in NLP
        Tweet
        Word embeddings
        
        2022-04-15 About
        
        Pablo Castro sur Twitter : "Random finding of the day for word embeddings: vec("apple")-vec("apples") yields a vector close to ipad, ipod, etc. (apples removes the "fruitness" from apple)
        
        Tags:
        
        Fun fact
        GloVe
        Tweet
        Word embeddings
        
        2020-12-18 About
        
        An overview of word embeddings and their connection to distributional semantic models - AYLIEN (2016)
        
        Tags:
        
        Distributional semantics
        GloVe
        Good
        Sebastian Ruder
        Word2vec
        Word embedding: evaluation
        Word embeddings
        
        > While on the surface DSMs and word embedding models use varying algorithms to learn word representations – the former count, the latter predict – both types of model fundamentally act on the same underlying statistics of the data, i.e. the co-occurrence counts between words... > These results are in contrast to the general consensus that word embeddings are superior to traditional methods. Rather, they indicate that it typically makes no difference whatsoever whether word embeddings or distributional methods are used. What really matters is that your hyperparameters are tuned and that you utilize the appropriate pre-processing and post-processing steps.
        
        2017-07-20 About
        
        Don’t count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors (2014)
        
        Tags:
        
        GloVe
        Word2vec
        Word embedding: evaluation
        Word embeddings
        
        (good presentation in the intro of context-counting vs. context-predicting vectors)
        
        2017-05-18 About
        
        How is GloVe different from word2vec? - Quora
        
        Tags:
        
        GloVe
        Word2vec
        Word embeddings
        
        Both learn geometrical encodings (vectors) of words from their co-occurrence information. Word2vec is a "predictive" model, whereas GloVe is a "count-based" model.
        
        2017-05-18 About
        
        GloVe: Global Vectors for Word Representation
        
        Tags:
        
        Chris Manning
        GloVe
        NLP@Stanford
        Richard Socher
        
        2017-05-18 About
      - Stanford classifier
        
        java - Method(s) to output confidence score from Stanford Classifier? - Stack Overflow
        
        Tags:
        
        Stack Overflow
        Stanford classifier
        
        2014-04-08 About
        
        [java-nlp-user] Stanford NER: confidence scores
        
        Tags:
        
        Stanford classifier
        
        2014-04-08 About
        
        The Stanford classifier
        
        Tags:
        
        Stanford classifier
        
        2014-03-16 About
      - Stanford NER
        
        Named Entity Recognition and the Stanford NER Software (slides)
        
        Tags:
        
        Slides
        Stanford NER
        
        2018-05-20 About
        
        Stanford Named Entity Recognizer
        
        Tags:
        
        Stanford NER
        
        general implementation of (arbitrary order) linear chain Conditional Random Field (CRF) sequence models
        
        2018-05-20 About
        
        How to Train your Own Model with NLTK and Stanford NER Tagger? (for English, French, German…)
        
        Tags:
        
        NLTK
        Stanford NER
        
        2018-05-20 About
      - Stanford POS Tagger
        
        dicksontsai/stanford-nlp-local-extension: Chrome extension for sending content to localhost server running Stanford NLP tools.
        
        Tags:
        
        Chrome extension
        GitHub project
        Sample code
        Servlet
        Stanford POS Tagger
        
        2020-07-03 About
        
        Dive Into NLTK, Part V: Using Stanford Text Analysis Tools in Python – Text Mining Online
        
        Tags:
        
        Java in python
        NLTK
        Stanford POS Tagger
        
        [en cas de pb](https://gist.github.com/alvations/e1df0ba227e542955a8a) **including how to use Java NLP Tools in python** ``` export CLASSPATH=/Users/fps/_fps/DeveloperTools/stanford-postagger-full/stanford-postagger.jar # ATTENTION, stanford-postagger.jar, pas stanford-postagger-3.8.0.jar export STANFORD_MODELS=/Users/fps/_fps/DeveloperTools/stanford-postagger-full/models python ``` ``` from nltk.tag import StanfordPOSTagger st = StanfordPOSTagger('english-bidirectional-distsim.tagger') st.tag('What is the airspeed of an unladen swallow ?'.split()) st = StanfordPOSTagger('french.tagger') st.tag('Les plats servis sont toujours les mêmes et la qualité des plats est en nette baisse'.split()) ``` [('Les', 'DET'), ('plats', 'NOUN'), ('servis', 'ADJ'), ('sont', 'VERB'), ('toujours', 'ADV'), ('les', 'DET'), ('mêmes', 'ADJ'), ('et', 'CONJ'), ('la', 'DET'), ('qualité', 'NOUN'), ('des', 'DET'), ('plats', 'NOUN'), ('est', 'VERB'), ('en', 'ADP'), ('nette', 'ADJ'), ('baisse', 'NOUN')]
        
        2017-07-11 About
        
        NLTK: Installing Third Party Software · nltk Wiki
        
        Tags:
        
        NLTK
        Stanford POS Tagger
        
        2017-07-11 About
        
        Source code for nltk.tag.stanford — NLTK documentation
        
        Tags:
        
        NLTK
        Stanford POS Tagger
        
        2017-07-11 About
        
        nltk.tag.stanford module — NLTK documentation
        
        Tags:
        
        NLTK
        Stanford POS Tagger
        
        A module for interfacing with the Stanford taggers.
        
        2017-07-11 About
        
        Stanford Log-linear Part-Of-Speech Tagger
        
        Tags:
        
        Stanford POS Tagger
        
        2017-07-11 About
      - Stanza
        
        stanfordnlp/stanza: Official Stanford NLP Python Library for Many Human Languages
        
        Tags:
        
        NLP@Stanford
        Python library
        Stanza
        
        2021-09-20 About
  - Coursera: NLP class
  - NLP@Stanford
  - Protégé
  - TAP
- University of Maryland
Universités françaises
- Sorbonne
  - Jussieu

0 Document (Long List)

Properties

rdf:type : sl:Tag
skos:prefLabel : Enseignement supérieur