Semanlink - Entreprise

Parents:

Entreprise

Related Tags:

Economie

Descendants

Alibaba
- NLP@Alibaba
Amazon
- Amazon Alexa
- Amazon Mechanical Turk
- AWS
  - Amazon S3
  - AWS Machine Learning
- Kindle
Apple
- AI@Apple
- Airport
- Apple Developer Connection
- Apple-Intel
- Apple Java
  - Java 1.5 Mac OS X
- Apple M1/M2
  - M2 Max
    - My M2Max mac
- Apple Neural Engine
- Apple Software
  - Hypercard
  - Mac OS X
    - Java 1.5 Mac OS X
    - Leopard
    - Mac OS X Web serving
    - OS X 10.6 - Snow leopard
    - OS X app
      - iphoto
        
        sparqling days / iphoto rdf export
        
        Tags:
        
        iphoto
        RDF
        
        2005-04-21 About
        
        RDF Exporter Plugin for iPhoto
        
        Tags:
        
        iphoto
        RDF
        
        2005-04-21 About
    - OSX Tip
    - OS X Unix
    - Spotlight (OSX)
    - Tiger
  - QuickTime
  - Safari
  - Siri
- Apple sucks
  - Steve please come back!
- FBI v. Apple
- Hypercard
- iOS
- iphone
  - Apple CarPlay
  - FBI v. Apple
  - iphone app
    - summly
- iPod
- iTunes
- JDD Apple
- Macintosh
  - Bricolage Mac
  - iMac
  - Mac dev
    - AppleScript
  - Mac Mini
  - Mac software
    - iMovie
- Mac OS X
- Meta Content Framework
- QuickTime
- Steve Jobs
  - Steve please come back!
    John H. Meyer 🚀 sur Twitter : "@emerywells That's actually what I built it for👀 Context: I unfortunately lost my dad unexpectedly at the young age of 50, back in 2017. There was a lot left un-said, and a lot I wish I could've spoken to him about in my adult life.…"
    Tags:
    
    Black Mirror
    ChatGPT: use-case
    Steve Jobs
    Tweet
    2023-03-21 About
    Steve Jobs the Lost Interview - Documentary Mania
    Tags:
    
    Steve Jobs
    2018-12-22 About
    Steve Jobs and the actually usable computer - W3C Blog
    Tags:
    
    Steve Jobs
    Tim Berners-Lee
    TBL about Steve Jobs
    
    2012-07-06 About
    Apple, America and a Squeezed Middle Class - NYTimes.com
    Tags:
    
    Apple
    Chine : technologie
    Délocalisations
    Outsourcing
    Steve Jobs
    Travail
    2012-01-26 About
- Tim Cook
  - FBI v. Apple
ASML
Biotech industry
- Monsanto
  - Glyphosate
- Syngenta
Bosch
Boston Dynamics
Caterpillar
Compagnies pétrolières
- BP
- ExxonMobil
- Texaco
EDF
Ford
Foxconn
France Télécom
Fujitsu
General Motors
Google
- Accelerated Mobile Pages (AMP)
- AI@Google
  - DeepMind
    - AlphaFold
    - Alphago
      - AlphaZero
        
        Jim Fan sur Twitter : "Do you know that DeepMind has actually open-sourced the heart of AlphaGo & AlphaZero?... "
        
        Tags:
        
        Alphago
        AlphaZero
        DeepMind
        Open Source
        Tweet
        
        2023-02-15 About
    - Chichilla
  - Gemini
  - Google AI Blog
  - Google Brain
  - Google car
  - NLP@Google
    - ALBERT
    - Bard
    - BERT
      - ALBERT
      - BERT + KB
        BLP
        CKB
        
        raphaelsty/ckb: Contextual knowledge bases
        
        Tags:
        
        CKB
        GitHub project
        Raphaël Sourty
        Semanlink Tag Finder
        
        Une implémentation de [BLP](tag:blp) [[2010.03496] Inductive Entity Representations from Text via Link Prediction](doc:2020/11/2010_03496_inductive_entity_r)
        
        2020-11-09 About
        
        Daniel Daza sur Twitter : "BioBLP, a method for learning embeddings on multimodal knowledge graphs...."
        
        Tags:
        
        BLP
        Tweet
        
        2023-06-07 About
        
        [2010.03496] Inductive Entity Representations from Text via Link Prediction
        
        Tags:
        
        Arxiv Doc
        Attention + Knowledge Graphs
        BLP
        Discuté avec Raphaël
        Entity embeddings
        Good
        Link Prediction
        Text-Aware KG embedding
        TheWebConf 2021
        
        BLP "BERT for Link Prediction". Central idea: **training an entity encoder with a link prediction objective** (using the textual descriptions of entities when computing entity representations - hence not failing with entities unknown in training) > a method for **learning representations of entities**, that uses a **pre-trained Transformer** based architecture as an entity encoder, and **link prediction training on a knowledge graph with textual entity descriptions**. > using entity descriptions, an entity encoder is trained for link prediction in a knowledge graph. The encoder can then be used without fine-tuning to obtain features for entity classification and information retrieval Cites [Xie et al](doc:2020/10/representation_learning_of_know) and [Kepler](doc:2020/11/1911_06136_kepler_a_unified_). They claim that their objective targeted exclusively for link prediction (and not an objective that combines language modeling and link prediction as Kepler) performs better than Kepler's more complex one.
        
        2020-11-03 About
        
        KnowBert
        
        [1909.04164] Knowledge Enhanced Contextual Word Representations
        
        Tags:
        
        Allen Institute for AI (AI2)
        Arxiv Doc
        Contextualized word representations
        EMNLP 2019
        Entities and LM
        Good
        Grounded Language Learning
        KD-MKB biblio
        KG-augmented Language Models
        KnowBert
        Knowledge-augmented Language Models
        Knowledge-driven embeddings
        Knowledge Graphs in NLP
        Multiple Knowledge Bases
        
        General method to **embed multiple knowledge bases into pre-trained language models** (KB in the sense as fixed collection of entity nodes) > The key idea is to explicitly model entity spans in the input text and use an **entity linker** to retrieve relevant entity embeddings from a KB to form knowledge enhanced entity-span representations. > Then, update contextual word representations via a form of **word-to-entity attention**. > In contrast to previous approaches, the entity linkers and self-supervised language modeling objective are jointly trained end-to-end in a multitask setting that **combines a small amount of entity linking supervision with a large amount of raw text**.
        
        2020-05-13 About
        
        LUKE
        
        [2110.08151] mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Models
        
        Tags:
        
        Arxiv Doc
        Entities and LM
        Ikuya Yamada
        LUKE
        Masked entity prediction
        Multilingual NLP
        
        [Ikuya Yamada sur Twitter : "Is entity representation effective to improve multilingual language models?..."](doc:2022/04/ikuya_yamada_sur_twitter_is_) > Recent studies have shown that multilingual pretrained language models can be effectively improved with cross-lingual alignment information from Wikipedia entities. However, **existing methods only exploit entity information in pretraining and do not explicitly use entities in downstream tasks**. In this study, we explore the **effectiveness of leveraging entity representations for downstream cross-lingual tasks**. > > the key insight is that incorporating entity representations into the input allows us to extract more language-agnostic features. [Github](https://github.com/studio-ousia/luke) > Entity representations are known to enhance language models in mono-lingual settings (Zhang et al., 2019: [ERNIE](tag:ernie.html); Peters et al., 2019: [[1909.04164] Knowledge Enhanced Contextual Word Representations](doc:2020/05/1909_04164_knowledge_enhanced); Wang et al., 2021 [[1911.06136] KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation](doc:2020/11/1911_06136_kepler_a_unified_); Xiong et al., 2020; Yamada et al., 2020: [[2010.01057] LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention](doc:2020/11/2010_01057_luke_deep_context)) presumably by introducing real-world knowledge. We show that using entity representations facilitates cross-lingual transfer by providing languageindependent features. > > Multilingual extension of LUKE. The model is trained with the multilingual masked language modeling (MLM) task as well as the masked entity prediction (MEP) task with Wikipedia entity embeddings > We investigate two ways of using the entity representations in cross-lingual transfer tasks: > 1. perform entity linking for the input text, and append the detected entity tokens to the input sequence. The entity tokens are expected to provide language independent features to the model > 2. use the entity [MASK] token from the MEP task as a languageindependent feature extractor.
        
        2022-04-17 About
        
        Ikuya Yamada sur Twitter : "Is entity representation effective to improve multilingual language models?..."
        
        Tags:
        
        Entities and LM
        Ikuya Yamada
        LUKE
        Multilingual Language Models
        Tweet
        
        [[2110.08151] mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Models](doc:2022/04/2110_08151_mluke_the_power_o) > mLUKE, an extension of [LUKE](tag:luke) based on 1M Wikidata entity embeddings shared across languages > mLUKE solves downstream tasks by using its language-agnostic entity embeddings as inputs. > entity representations are shared across languages during pretraining -> they are much more language-agnostic than word representations
        
        2022-04-13 About
        
        Studio Ousia sur Twitter : "Now using LUKE is easier than ever!" / Twitter
        
        Tags:
        
        LUKE
        Named Entity Recognition
        Tweet
        
        2022-03-15 About
        
        [2010.01057] LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention
        
        Tags:
        
        Arxiv Doc
        BERT + KB
        EMNLP 2020
        Entity embeddings
        Ikuya Yamada
        LUKE
        Self-Attention
        Text-Aware KG embedding
        
        > LUKE is based on bidirectional Transformer, treats words and entities in a text as independent tokens, and outputs contextualized representations of them. The representations can be used to address downstream tasks similarly to BERT. [src](https://twitter.com/ikuyamada/status/1312947499141750786) > LUKE is trained using a novel pretraining task that involves predicting randomly masked words (equivalent to BERT’s masked language model) and entities in an entity-annotated corpus obtained from Wikipedia. (Hum, ça me rappelle quelque chose) > LUKE also uses a new *entity-aware* self-attention mechanism that considers the types of tokens (words or entities) when computing attention scores. [github](https://github.com/studio-ousia/luke), [at Hugging Face](https://twitter.com/AkariAsai/status/1389428550298525696), [doc](https://huggingface.co/transformers/model_doc/luke.html), [tweet](https://twitter.com/ikuyamada/status/1392742990586683392?s=20)
        
        2020-11-26 About
        
        [2205.00820] Entity-aware Transformers for Entity Search
        
        Tags:
        
        Arxiv Doc
        BERT + KB
        Discuté avec Raphaël
        Entities and LM
        Entity Retrieval
        KG-aware low-resource learning
        SIGIR 2022
        
        > **Do BERT-based entity retrieval models benefit from additional entity information stored in knowledge graphs?** To address this research question, we map entity embeddings into the same input space as a pre-trained BERT model and inject these entity embeddings into the BERT model. This entity-enriched language model is then employed on the entity retrieval task. > we observe empirically that the entity-enriched BERT models **enable fine-tuning on limited training data**, which otherwise would not be feasible due to the known instabilities of BERT in few-sample fine-tuning Uses [Wikipedia2Vec](tag:wikipedia2vec) as graph embedding method
        
        2022-07-12 About
        
        [1911.03681] E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT
        
        Tags:
        
        Arxiv Doc
        BERT
        BERT + KB
        Discuté avec Raphaël
        Entity embeddings
        Text, KG and embeddings
        
        > way of **injecting factual knowledge about entities into the pretrained BERT model**. (Feeding entity vectors into BERT as if they were wordpiece vectors without additional encoder pretraining) > > **We align [Wikipedia2Vec](tag:wikipedia2vec) entity vectors (Yamada et al., 2016) with BERT's native wordpiece vector space and use the aligned entity vectors as if they were wordpiece vectors**. The resulting entity-enhanced version of BERT (called E-BERT) is similar in spirit to [ERNIE](tag:ernie) (Zhang et al., 2019) and [KnowBert](tag:knowbert) (Peters et al., 2019), but it **requires no expensive further pretraining of the BERT encoder**. > > Our vector space alignment strategy is inspired by cross-lingual word vector alignment Related work on Entity-enhanced BERT: > ([ERNIE](doc:2019/08/_1905_07129_ernie_enhanced_la) and [Knowbert](doc:2020/05/1909_04164_knowledge_enhanced)) are based on the design principle that BERT be adapted to entity vectors. They introduce new encoder layers to feed pretrained entity vectors into the Transformer, and they require additional pretraining to integrate the new parameters. In contrast, E-BERT’s design principle is that entity vectors be adapted to BERT. > > Two other knowledge-enhanced MLMs are [KEPLER](doc:2020/11/1911_06136_kepler_a_unified_) (Wang et al., 2019c) and K-Adapter (Wang et al., 2020)... Their factual knowledge does not stem from entity vectors – instead, they are trained in a multi-task setting on relation classification and knowledge base completion. Not to be cofounded with [[2009.02835] E-BERT: A Phrase and Product Knowledge Enhanced Language Model for E-commerce](doc:2020/12/2009_02835_e_bert_a_phrase_a)
        
        2021-01-12 About
        
        [2009.02835] E-BERT: A Phrase and Product Knowledge Enhanced Language Model for E-commerce
        
        Tags:
        
        Arxiv Doc
        Aspect Detection
        BERT + KB
        Domain-Specific BERT
        Product data & NLP
        
        E-BERT, pre-training framework for product data. 1. to benefit from phrase-level knowledge: Adaptive Hybrid Masking, a new masking strategy, which allows the model to adaptively switch from learning preliminary word knowledge to learning complex phrases 2. leveraging product-level knowledge: training E-BERT to predict a product’s associated neighbors (product association) Resources used: - description of millions of products from the amazon dataset (title, description, reviews) - e-commerce phrases: extracted from above dataset using [AutoPhrase](doc:2020/12/autophrase_automated_phrase_mi) - product association graph: pairs of substitutable and complementary products extracted from amazon dataset Not to be confounded with [[1911.03681] E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT](doc:2021/01/1911_03681_e_bert_efficient_)
        
        2020-12-14 About
        
        Supporting content decision makers with machine learning | Dec, 2020 | Netflix TechBlog
        
        Tags:
        
        BERT + KB
        Entity embeddings
        Link Prediction
        Netflix
        Transfer learning
        
        2020-12-11 About
        
        [2010.01057] LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention
        
        Tags:
        
        Arxiv Doc
        BERT + KB
        EMNLP 2020
        Entity embeddings
        Ikuya Yamada
        LUKE
        Self-Attention
        Text-Aware KG embedding
        
        > LUKE is based on bidirectional Transformer, treats words and entities in a text as independent tokens, and outputs contextualized representations of them. The representations can be used to address downstream tasks similarly to BERT. [src](https://twitter.com/ikuyamada/status/1312947499141750786) > LUKE is trained using a novel pretraining task that involves predicting randomly masked words (equivalent to BERT’s masked language model) and entities in an entity-annotated corpus obtained from Wikipedia. (Hum, ça me rappelle quelque chose) > LUKE also uses a new *entity-aware* self-attention mechanism that considers the types of tokens (words or entities) when computing attention scores. [github](https://github.com/studio-ousia/luke), [at Hugging Face](https://twitter.com/AkariAsai/status/1389428550298525696), [doc](https://huggingface.co/transformers/model_doc/luke.html), [tweet](https://twitter.com/ikuyamada/status/1392742990586683392?s=20)
        
        2020-11-26 About
        
        Learning to Tag OOV Tokens by Integrating Contextual Representation and Background Knowledge (ACL Anthology 2020)
        
        Tags:
        
        ACL 2020
        BERT + KB
        Few-shot learning
        Knowledge Graphs in NLP
        OOV
        Slot filling
        
        Aim to leverage both contextual representation of input text (deep LMs) and knowledge derived from curated KBs ([Wordnet](tag:wordnet)) to improve [slot tagging](tag:slot_tagging) in the presence of [out-of-vocab](tag:oov) words ([few-shot scenario](tag:few_shot_learning)) Method: 1. retrieve potentially relevant KB entities and encode them into distributed representations that describe global graph-structured information 2. BERT encoder layer to capture context-aware representations of the sequence and attend to the KB embeddings using multi-level graph attention 3. integrate BERT embeddings and the KB embeddings to predict the slot type Contributions: 1. feasibility of applying lexical ontology to facilitate recognizing OOV words. First to consider the large-scale background knowledge for enhancing context-aware slot tagging models. 2. a knowledge integration mechanism that uses multi-level graph attention to model explicit lexical relations. 3.experiments on two benchmark datasets > our method makes a notable difference in a scenario where samples are linguistically diverse, and large vocab exists. (Better improvements when using RNN than BERT, because BERT already contains a lot of background knowledge)
        
        2020-07-04 About
      - BERT + Sentence Embeddings
        Sentence-BERT
        sBERT fine-tuning
        SetFit (SBERT fine-tuning)
        
        UKP Lab sur X : "a lightweight solution for few-shot domain-specific sentence classification: AdaSent!..."
        
        Tags:
        
        Adapter modules (LM finetuning)
        EMNLP 2023
        Few-shot text classification
        Sentence-BERT: Domain adaptation
        Sentence embeddings: Domain adaptation
        SetFit (SBERT fine-tuning)
        Tweet
        
        AdaSent is an approach to creating domain-specialized sentence encoders for few-shot sentence classification > Reusable general sentence adapter across domains > AdaSent decouples DAPT (Domain Adaptative Pre-Training) & SEPT (Sentence Embedding Pre-Training) **by storing the sentence encoding abilities into an adapter**, which is trained only once in the general domain and plugged into various DAPT-ed PLMs [Github](https://github.com/UKPLab/AdaSent)
        
        2023-12-09 About
        
        Daniel Vila Suero sur Twitter : "Data quality is key for LLMs, but we're building Open Source LLMs with data of "unknown" quality... Introducing Alpaca GarbageCollector..."
        
        Tags:
        
        Alpaca
        Dataset quality
        SetFit (SBERT fine-tuning)
        Tweet
        
        > a cross-lingual SetFit model to identify potential bad instructions in Alpaca-like datasets
        
        2023-04-05 About
        
        ValueError "invalid literal for int() with base 10" in trainer.evaluate (dataset created from pandas) · Issue #228 · huggingface/setfit
        
        Tags:
        
        fps' post
        SetFit (SBERT fine-tuning)
        
        see <https://github.com/huggingface/setfit/blob/main/notebooks/zero-shot-classification.ipynb> > Note: some datasets on the Hugging Face Hub don't have a ClassLabel feature for the label column. In these cases, you should compute the candidate labels manually by first computing the id2label mapping as follows:
        
        2022-12-13 About
        
        Lewis Tunstall sur Twitter : "The SetFit library for few-shot learning with Sentence Transformers now supports *multi-label text classification*..."
        
        Tags:
        
        Multi-label Text classification
        SetFit (SBERT fine-tuning)
        Tweet
        
        Multilabel support [github issue](https://github.com/huggingface/setfit/issues/65)
        
        2022-10-14 About
        
        huggingface/setfit: Efficient few-shot learning with Sentence Transformers
        
        Tags:
        
        GitHub project
        SetFit (SBERT fine-tuning)
        
        2022-10-12 About
        
        [2209.11055] Efficient Few-Shot Learning Without Prompts
        
        Tags:
        
        Arxiv Doc
        Few-shot learning
        Moshe Wasserblat
        Nils Reimers
        SetFit (SBERT fine-tuning)
        
        [tweet](https://twitter.com/_akhaliq/status/1573109469646561280?s=20&t=RTpK9dh90az0zT1Xg2ohpQ): > So if I have 4 classes and say 2 labels per class, I would first fine tune an ST on these 4 pairs and then vectorize the 8 total examples for fine-tuning the classifier
        
        2022-09-23 About
        
        Sentence Transformer Fine-Tuning (SetFit): Outperforming GPT-3 on few-shot Text-Classification while being 1600 times smaller | by Moshe Wasserblat (2021-12)
        
        Tags:
        
        Few-shot text classification
        Moshe Wasserblat
        sBERT fine-tuning
        SetFit (SBERT fine-tuning)
        
        Finetuning d'un SBERT sur une tâche de classification (in fine, produit un SBERT) > **Few-shot text classification based on fine-tuning a Sentence Transformer with task-specific data** that can easily be implemented with the sentence-transformers library > Surprisingly, we did not find any work that performed an end-to-end ST fine-tuning for text classification in a Siamese manner. [COLAB](https://colab.research.google.com/github/MosheWasserb/SetFit/blob/main/SetFit_SST_2.ipynb) [Nils Reimers sur Twitter](doc:2022/03/nils_reimers_sur_twitter_gre)
        
        2022-03-31 About
        
        Nils Reimers sur Twitter : "Great post on SetFit"
        
        Tags:
        
        Nils Reimers
        SetFit (SBERT fine-tuning)
        Tweet
        
        About [Sentence Transformer Fine-Tuning (SetFit): Outperforming GPT-3 on few-shot Text-Classification while being 1600 times smaller | by Moshe Wasserblat](doc:2022/03/sentence_transformer_fine_tunin) > - Outperforms GPT-3 in few-shot text-classification (50 labeled examples, secret test set) > - 1600 times smaller > - Can be run on your CPU > - No limitation on the number of training examples > - Just few lines of code needed
        
        2022-03-31 About
        
        LlamaIndex 🦙 sur X : "Fine-tuning Embedding Models for RAG with LoRA'
        
        Tags:
        
        LlamaIndex
        LoRA
        sBERT fine-tuning
        
        2024-04-23 About
        
        [2008.11228] A simple method for domain adaptation of sentence embeddings
        
        Tags:
        
        Arxiv Doc
        Domain adaptation (NLP)
        sBERT fine-tuning
        Sentence-BERT: Domain adaptation
        Universal Sentence Encoder
        
        2022-04-01 About
        
        [2004.05119] Beyond Fine-tuning: Few-Sample Sentence Embedding Transfer
        
        Tags:
        
        ACL 2020
        Arxiv Doc
        Low-Resource NLP
        NLP@Amazon
        NLP@Microsoft
        sBERT fine-tuning
        
        > Fine-tuning (FT) pre-trained sentence embedding models on small datasets has been shown to have limitations. In this paper we show that concatenating the embeddings from the pre-trained model with those from a simple sentence embedding model trained only on the target data, can improve over the performance of FT for few-sample tasks
        
        2022-03-31 About
        
        Sentence Transformer Fine-Tuning (SetFit): Outperforming GPT-3 on few-shot Text-Classification while being 1600 times smaller | by Moshe Wasserblat (2021-12)
        
        Tags:
        
        Few-shot text classification
        Moshe Wasserblat
        sBERT fine-tuning
        SetFit (SBERT fine-tuning)
        
        Finetuning d'un SBERT sur une tâche de classification (in fine, produit un SBERT) > **Few-shot text classification based on fine-tuning a Sentence Transformer with task-specific data** that can easily be implemented with the sentence-transformers library > Surprisingly, we did not find any work that performed an end-to-end ST fine-tuning for text classification in a Siamese manner. [COLAB](https://colab.research.google.com/github/MosheWasserb/SetFit/blob/main/SetFit_SST_2.ipynb) [Nils Reimers sur Twitter](doc:2022/03/nils_reimers_sur_twitter_gre)
        
        2022-03-31 About
        
        Sentence Embedding Fine-tuning for the French Language | by La Javaness R&D | Feb, 2022 | Medium
        
        Tags:
        
        CamemBERT
        NLP: French
        NLP sample code
        sBERT fine-tuning
        
        2022-03-31 About
        
        [2203.14655] Few-Shot Learning with Siamese Networks and Label Tuning
        
        Tags:
        
        Arxiv Doc
        Few-shot text classification
        sBERT fine-tuning
        Sentence-BERT
        Siamese networks
        
        > the problem of building text classifiers with little or no training data. > > In recent years, an approach based on neural textual entailment models has been found to give strong results on a diverse range of tasks. (cf. #[NLI](tag:nli), using the input text as the premise and the text representing the label as the hypothesis) > In this work, we show that **with proper pre-training, Siamese Networks that embed texts and labels** offer a competitive alternative. > > We introduce **label tuning: fine-tuning the label embeddings only**. While giving lower performance than model fine-tuning (which updates all params of the model), this approach has the architectural advantage that a single encoder can be shared by many different tasks (we only fine-tune the label embeddings) > The drop in quality can be compensated by using a variant of **[Knowledge distillation](tag:knowledge_distillation)** [Github](https://tinyurl.com/label-tuning), [Tweet](doc:2022/03/thomas_muller_sur_twitter_pa)
        
        2022-03-30 About
        
        Sentence-BERT: Domain adaptation
        GPL (Generative Pseudo Labeling)
        
        Domain Adaptation with Generative Pseudo-Labeling (GPL) | Pinecone
        
        Tags:
        
        GPL (Generative Pseudo Labeling)
        Pinecone
        
        2023-04-09 About
        
        [2205.11498] Domain Adaptation for Memory-Efficient Dense Retrieval
        
        Tags:
        
        Arxiv Doc
        Dense retriever
        Domain adaptation
        GPL (Generative Pseudo Labeling)
        Nils Reimers
        
        Refers to [Binary Passage Retriever (BPR)](doc:2021/06/2106_00882_efficient_passage_)
        
        2022-09-26 About
        
        Domain transfer with GGPL: German Generative Pseudo Labeling 🥨 | by Matthias Richter | Jun, 2022 | ML6team
        
        Tags:
        
        GPL (Generative Pseudo Labeling)
        Multilingual NLP
        
        2022-06-02 About
        
        Nils Reimers sur Twitter : "GPL goes multi-lingual..."
        
        Tags:
        
        GPL (Generative Pseudo Labeling)
        Nils Reimers
        Tweet
        
        [Domain transfer with GGPL: German Generative Pseudo Labeling](doc:2022/06/domain_transfer_with_ggpl_germ)
        
        2022-06-01 About
        
        Ramsri Goutham Golla sur Twitter : "Hi @Nils_Reimers For GPL you used "msmarco-distilbert-base-tas-b" model and ..."
        
        Tags:
        
        GPL (Generative Pseudo Labeling)
        Nils Reimers
        Tweet
        
        2022-04-27 About
        
        Domain Adaptation — Sentence-Transformers documentation
        
        Tags:
        
        Documentation
        Domain adaptation (NLP)
        GPL (Generative Pseudo Labeling)
        Sentence-BERT: Domain adaptation
        TSDAE
        
        2022-03-31 About
        
        NAVER LABS Europe : "@Nils_Reimers of @huggingface on 'Unsupervised domain adaptation for neural search'"
        
        Tags:
        
        GPL (Generative Pseudo Labeling)
        J'y étais
        Neural Search
        Nils Reimers
        Unsupervised Domain Adaptation (NLP)
        
        2022-03-09 About
        
        [2112.07577] GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval
        
        Tags:
        
        Arxiv Doc
        Dense Passage Retrieval
        GPL (Generative Pseudo Labeling)
        Nils Reimers
        Synthetic QA data
        Unsupervised Domain Adaptation (NLP)
        
        An unsupervised domain adaptation technique for dense retrieval models 1. synthetic queries are generated for each passage from the target corpus (using an existing pre-trained [T5](tag:text_to_text_transfer_transformer) encoder-decoder) 2. the generated queries are used for mining negative passages (retrieving the most similar paragraphs using an existing dense retrieval model == hard negatives!) 3. the query-passage pairs are labeled by a cross-encoder and used to train the domain-adapted dense retriever (using method described in [Hofstätter et al., 2020](doc:2021/12/2010_02666_improving_efficien)) [Nils Reimers sur Twitter](doc:2021/12/nils_reimers_sur_twitter_do_), [GitHub](https://github.com/UKPLab/gpl), by the author of [TSDAE](doc:2021/09/2104_06979_tsdae_using_trans) Claims to improve "Doc2Query" [Document Expansion by Query Prediction](doc:2022/01/1904_08375_document_expansion): ([src](https://twitter.com/KexinWang2049/status/1471435779415150598)) > - GPL: Uses doc2query to construct synthetic data and does knowledge distillation (i.e. training) on that data. > - Doc2query: Generates queries to extend the documents and use BM25 on top of them w/o training.
        
        2021-12-15 About
        
        sBERT fine-tuning
        
        UKP Lab sur X : "a lightweight solution for few-shot domain-specific sentence classification: AdaSent!..."
        
        Tags:
        
        Adapter modules (LM finetuning)
        EMNLP 2023
        Few-shot text classification
        Sentence-BERT: Domain adaptation
        Sentence embeddings: Domain adaptation
        SetFit (SBERT fine-tuning)
        Tweet
        
        AdaSent is an approach to creating domain-specialized sentence encoders for few-shot sentence classification > Reusable general sentence adapter across domains > AdaSent decouples DAPT (Domain Adaptative Pre-Training) & SEPT (Sentence Embedding Pre-Training) **by storing the sentence encoding abilities into an adapter**, which is trained only once in the general domain and plugged into various DAPT-ed PLMs [Github](https://github.com/UKPLab/AdaSent)
        
        2023-12-09 About
        
        [2008.11228] A simple method for domain adaptation of sentence embeddings
        
        Tags:
        
        Arxiv Doc
        Domain adaptation (NLP)
        sBERT fine-tuning
        Sentence-BERT: Domain adaptation
        Universal Sentence Encoder
        
        2022-04-01 About
        
        Domain Adaptation — Sentence-Transformers documentation
        
        Tags:
        
        Documentation
        Domain adaptation (NLP)
        GPL (Generative Pseudo Labeling)
        Sentence-BERT: Domain adaptation
        TSDAE
        
        2022-03-31 About
        
        Unsupervised Training for Sentence Transformers | Pinecone
        
        Tags:
        
        Pinecone
        Sentence-BERT: Domain adaptation
        TSDAE
        Unsupervised Sentence Embedding Learning
        
        Blog post about [[2104.06979] TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learning](doc:2021/09/2104_06979_tsdae_using_trans) > Fine-tuning with TSDAE simply cannot compete in terms of performance against supervised methods. However, **the point and value of TSDAE is that it allows us to fine-tune models for use-cases where we have no data**. Specific domains with unique terminology or low resource languages.
        
        2021-11-24 About
        
        SentenceBERT and Words
        
        Few-Shot Text Classification (Cloudera 2020)
        
        Tags:
        
        Few-shot text classification
        SentenceBERT and Words
        
        > Sentence-BERT has been optimized… well, for sentences! It’s reasonable to suspect that SBERT’s representations of single words or short phrases like “Business” or “Science & Technology” won’t be as semantically relevant as representations derived from a word-level method, like word2vec or GloVe
        
        2022-11-24 About
        
        skeskinen/bert.cpp: ggml implementation of BERT
        
        Tags:
        
        .cpp
        BERT
        Frugal LLMs
        Quantization
        Sentence-BERT
        Sentence Embeddings
        
        > ggml inference of BERT neural net architecture with pooling and normalization from SentenceTransformers (sbert.net). High quality sentence embeddings in pure C++ (with C API). > > The main goal of bert.cpp is to run the BERT model using **4-bit integer quantization on CPU**
        
        2023-05-09 About
        
        Multilingual Sentence Transformers | Pinecone
        
        Tags:
        
        Multilingual embeddings
        Pinecone
        Sentence-BERT
        
        Focus on **Multilingual Knowledge Distillation** > recent method introduced by Nils Reimers and Iryna Gurevych in 2020 > The teacher model is an already fine-tuned sentence transformer used for creating embeddings in a single language (most likely English). The student model is a transformer that has been pretrained on a multilingual corpus.
        
        2023-01-13 About
        
        Unsupervised Learning — Sentence-Transformers documentation
        
        Tags:
        
        Sentence-BERT
        TSDAE
        Unsupervised Sentence Embedding Learning
        
        > In our paper TSDAE we compare approaches for sentence embedding tasks, and in GPL we compare them for semantic search tasks (given a query, find relevant passages). While the unsupervised approach achieve acceptable performances for sentence embedding tasks, they perform poorly for semantic search tasks.
        
        2022-08-20 About
        
        Train and Fine-Tune Sentence Transformers Models
        
        Tags:
        
        Sentence-BERT
        
        2022-08-13 About
        
        sentence bert model in onnx format · Issue #46 · UKPLab/sentence-transformers
        
        Tags:
        
        ONNX
        Raphaël Sourty
        Sentence-BERT
        
        2022-06-13 About
        
        [2205.15952] Knowledge Graph -- Deep Learning: A Case Study in Question Answering in Aviation Safety Domain
        
        Tags:
        
        Arxiv Doc
        Knowledge Graph + Deep Learning
        Question Answering
        Sentence-BERT
        
        2022-06-11 About
        
        [2205.04260] EASE: Entity-Aware Contrastive Learning of Sentence Embedding
        
        Tags:
        
        Arxiv Doc
        Contrastive Learning
        Entities
        Entities and LM
        Entity type
        Ikuya Yamada
        Sentence-BERT
        Sentence Embeddings
        
        > we explore a type of supervision that has been under-explored in the literature: entity hyperlink annotations from Wikipedia. > > entities have been shown to be a strong indicator of text semantics > > a method for mining hard negatives based on the entity type Uses wikipedia2vec > the reliance on Wikipedia for training data may limit the application of the models to specific domains (e.g., general or encyclopedia domains). To apply EASE to other domains, one may need to annotate text from the domain either manually or automatically.
        
        2022-05-11 About
        
        [2203.14655] Few-Shot Learning with Siamese Networks and Label Tuning
        
        Tags:
        
        Arxiv Doc
        Few-shot text classification
        sBERT fine-tuning
        Sentence-BERT
        Siamese networks
        
        > the problem of building text classifiers with little or no training data. > > In recent years, an approach based on neural textual entailment models has been found to give strong results on a diverse range of tasks. (cf. #[NLI](tag:nli), using the input text as the premise and the text representing the label as the hypothesis) > In this work, we show that **with proper pre-training, Siamese Networks that embed texts and labels** offer a competitive alternative. > > We introduce **label tuning: fine-tuning the label embeddings only**. While giving lower performance than model fine-tuning (which updates all params of the model), this approach has the architectural advantage that a single encoder can be shared by many different tasks (we only fine-tune the label embeddings) > The drop in quality can be compensated by using a variant of **[Knowledge distillation](tag:knowledge_distillation)** [Github](https://tinyurl.com/label-tuning), [Tweet](doc:2022/03/thomas_muller_sur_twitter_pa)
        
        2022-03-30 About
        
        Document Matching for Job Descriptions | Semantic Scholar (2021)
        
        Tags:
        
        NLP@Stanford
        NLP + Human Resources
        Sentence-BERT
        
        > We train a document encoder to match online job descriptions to one of many standardized job roles from Singapore’s Skills Framework. The encoder generates semantically meaningful document encodings from textual descriptions of job roles, which are then compared using Cosine Similarity to determine matching. During training, we implement the methodology used by Sentence-BERT, fine tuning pre-trained BERT models using a siamese network architecture on labelled document pairs.
        
        2022-03-09 About
        
        Nils Reimers sur Twitter : "Creating intent classes for chatbots is challenging This tutorial shows how to use sentence-transformers to find potentially overlapping intent classes and how to improve your data annotation work." / Twitter
        
        Tags:
        
        Annotations (ML)
        Chatbots
        Nils Reimers
        Sentence-BERT
        Spectral clustering
        Tutorial
        Tweet
        
        2022-02-19 About
        
        Nils Reimers sur Twitter : "how to use the fast clustering algorithm from sentence-transformers..."
        
        Tags:
        
        Nils Reimers
        Sentence-BERT
        Short Text Clustering
        Tweet
        
        Clustering millions of sentences to optimize the ML-workflow
        
        2022-02-19 About
        
        sentence-transformers/fast_clustering.py at master · UKPLab/sentence-transformers
        
        Tags:
        
        Clustering
        Nils Reimers
        Sentence-BERT
        
        > This is a more complex example on performing clustering on large scale dataset. This examples find in a large set of sentences local communities, i.e., groups of sentences that are highly similar. You can freely configure the threshold what is considered as similar. A high threshold will only find extremely similar sentences, a lower threshold will find more sentence that are less similar. A second parameter is 'min_community_size': Only communities with at least a certain number of sentences will be returned. The method for finding the communities is extremely fast, for clustering 50k sentences it requires only 5 seconds (plus embedding comuptation). In this example, we download a large set of questions from Quora and then find similar questions in this set.
        
        2022-02-18 About
        
        gsarti/scibert-nli · Hugging Face
        
        Tags:
        
        Hugging Face
        NLP for scientific documents
        Sentence-BERT
        
        SciBERT fine-tuned on the SNLI and the MultiNLI datasets using the sentence-transformers library to produce universal sentence embeddings
        
        2022-01-29 About
        
        Semantic Search — Sentence-Transformers documentation
        
        Tags:
        
        Documentation
        Semantic Search
        Sentence-BERT
        
        **symmetric** semantic search vs **asymmetric** semantic search > - Suitable models for symmetric semantic search: Pre-Trained Sentence Embedding > - Suitable models for asymmetric semantic search: Pre-Trained MS MARCO Models
        
        2022-01-29 About
        
        Using pretrained SBERT model in cross-encoder · Issue #726 · UKPLab/sentence-transformers
        
        Tags:
        
        Cross-Encoders
        Sentence-BERT
        
        > so would it be a good idea to finetune a SBERT model on a cross-encoder task? > > The SBERT models are regular transformers model and hence can be used as base for cross encoders. Sometimes it could be helpful, otherwise it is better to use the original models. ([Nils Reimers](tag:nils_reimers))
        
        2021-12-17 About
        
        Advance BERT model via transferring knowledge from Cross-Encoders to Bi-Encoders | by Chien Vu | Towards Data Science
        
        Tags:
        
        BERT
        Cross-Encoders
        Data Augmentation
        Sentence-BERT
        Sentence Similarity
        
        Data Augmentation Method to improve SBERT Bi-Encoders for Pairwise Sentence Scoring Tasks (Semantic sentence tasks)
        
        2021-12-17 About
        
        Semantic search through a vectorized Wikipedia (SentenceBERT) with the Weaviate vector search engine
        
        Tags:
        
        Semantic Search
        Sentence-BERT
        Similarity queries / Vector search
        Weaviate
        Wikipedia
        
        2021-12-05 About
        
        Unsupervised_Extractive_Summarization - a Hugging Face Space by Hellisotherpeople
        
        Tags:
        
        Extractive Text Summarization
        GitHub project
        Sentence-BERT
        
        Unsupervised Extractive Text Summarization and Semantic Search [Github](https://github.com/Hellisotherpeople/CX_DB8)
        
        2021-12-03 About
        
        How to Fine-Tune Sentence-BERT for Question Answering | Capital One
        
        Tags:
        
        Question Answering
        Sentence-BERT
        Tutorial
        
        > tutorial on using the sentence-transformers library to fine-tune Sentence-BERT for question matching
        
        2021-11-21 About
        
        Multilingual Sentence Transformers | Pinecone
        
        Tags:
        
        Knowledge distillation
        Multilingual Language Models
        Pinecone
        Sentence-BERT
        Transformers
        
        How to make a text encoder multilingual using sentence transformers and multilingual knowledge distillation.
        
        2021-11-04 About
        
        Next-Gen Sentence Embeddings with Multiple Negatives Ranking Loss | Pinecone
        
        Tags:
        
        Multiple negatives ranking (MNR) loss
        Pinecone
        Sentence-BERT
        Sentence Embeddings
        
        > the world of sentence embeddings was ignited with the introduction of SBERT in 2019. Since then, many more sentence transformers have been introduced. These models quickly made the original SBERT obsolete. How did these newer sentence transformers manage to outperform SBERT so quickly? The answer is **multiple negatives ranking (MNR) loss**. > In short; **fine-tune your models with MNR loss, and do it with the [sentence-transformers](tag:sbert) library**. (mentionned in a [tweet](https://twitter.com/Nils_Reimers/status/1453001422400856086) by [Nils Reimers](tag:nils_reimers))
        
        2021-10-27 About
        
        Sentence Embeddings and Transformers | Pinecone
        
        Tags:
        
        Pinecone
        Sentence-BERT
        
        2021-10-23 About
        
        Sahajtomar/french_semantic · Hugging Face
        
        Tags:
        
        CamemBERT
        Discuté avec Raphaël
        huggingface / transformers
        NLP: French
        Sentence-BERT
        
        2021-10-14 About
        
        Semantic Search with S-BERT is all you need
        
        Tags:
        
        Howto
        Semantic Search
        Sentence-BERT
        
        > SentenceTransformers is designed in such way that fine-tuning your own sentence / text embeddings models is easy.
        
        2021-06-05 About
        
        Nils Reimers sur Twitter : "SBERT Release v1.1.0"
        
        Tags:
        
        Nils Reimers
        Sentence-BERT
        Tweet
        
        2021-04-22 About
        
        Nils Reimers sur Twitter : "New models for Neural Information Retrieval..."
        
        Tags:
        
        Information retrieval
        Nils Reimers
        Sentence-BERT
        Tweet
        
        2021-04-17 About
        
        SentenceTransformers Documentation
        
        Tags:
        
        Documentation
        Sentence-BERT
        
        2021-03-25 About
        
        Zero-Shot Learning in Modern NLP | Joe Davison Blog (2020-05)
        
        Tags:
        
        Hugging Face
        Natural Language Inference
        Sentence-BERT
        Text Classification
        Zero-Shot Learning
        Zero-shot Text Classifier
        
        > state-of-the-art NLP models for sequence classification without large annotated training sets. Simple idea: use a single model (eg. [Sentence-BERT](tag:sbert)) to embed both the text data and the class names into the same space. Pb: Sentence-BERT is designed to learn effective sentence-level, not single- or multi-word representations like our class names -> the label embeddings may not be as semantically salient as word-level embedding methods (i.e. word2vec). Solution 1: Learn a projection from sentence level embeddings of words to word2vec embeddings, use it for encoding when learning classifier. Can be adapted to few short learning Solution 2: "Classification as [#Natural Language Inference](tag:nli)". > A method which not only embeds sequences and labels into the same latent space where their distance can be measured, but that can actually tell us something about the compatibility of two distinct sequences out of the box.
        
        2021-02-23 About
        
        UKPLab/sentence-transformers: Sentence Embeddings with BERT & XLNet
        
        Tags:
        
        GitHub project
        huggingface / transformers
        Sentence-BERT
        XLNet
        
        [paper](doc:2019/08/_1908_10084_sentence_bert_sen)
        
        2020-07-14 About
        
        Richer Sentence Embeddings using Sentence-BERT — Part I
        
        Tags:
        
        Sentence-BERT
        
        Simplistic (and often used) methods for sentence embeddings with BERT are too simplistic to be good (avearaging the word vectors, or using the \[CLS\] special vector (start of sequence). [About this paper](/doc/2019/08/_1908_10084_sentence_bert_sen)
        
        2020-01-06 About
        
        [1908.10084] Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
        
        Tags:
        
        Arxiv Doc
        EMNLP 2019
        huggingface / transformers
        Nearest neighbor search
        Sentence-BERT
        Sentence Similarity
        Siamese networks
        
        > Sentence-BERT (SBERT), a modification of the pretrained BERT network that use siamese and triplet network structures to derive **semantically meaningful sentence embeddings** that can be compared using cosine-similarity. Important because - BERT ist unsuitable for semantic similarity search as well as for unsupervised tasks like clustering. - simple methods such as using the CLS token give low quality sentence embeddings However, the purpose of SBERT sentence embeddings are **not to be used for transfer learning for other tasks**. [Related blog post](/doc/2020/01/richer_sentence_embeddings_usin); [Github](https://github.com/UKPLab/sentence-transformers)
        
        2019-08-28 About
        
        PromptBERT improving BERT sentence embeddings with prompts - Ethan Kim
        
        Tags:
        
        BERT + Sentence Embeddings
        
        2022-09-16 About
        
        [2201.04337] PromptBERT: Improving BERT Sentence Embeddings with Prompts
        
        Tags:
        
        Arxiv Doc
        BERT + Sentence Embeddings
        Prompting/In-context learning
        
        [PromptBERT improving BERT sentence embeddings with prompts - Ethan Kim](doc:2022/09/promptbert_improving_bert_sente)
        
        2022-09-16 About
        
        [2011.05864] On the Sentence Embeddings from Pre-trained Language Models
        
        Tags:
        
        Anisotropy in LM space
        Arxiv Doc
        BERT + Sentence Embeddings
        EMNLP 2020
        
        > **the sentence embeddings from the pre-trained language models without fine-tuning have been found to poorly capture semantic meaning of sentences.** > > We find that **BERT always induces a non-smooth anisotropic semantic space of sentences**, which harms its performance of semantic similarity. To address this issue, we propose to transform the anisotropic sentence embedding distribution to a smooth and isotropic Gaussian distribution through normalizing flows that are learned with an unsupervised objective > normalizing flows (Dinh et al., 2015): invertible function parameterized by neural networks. > **During training, only the flow network is optimized while the BERT parameters remain unchanged** > When combined with external supervision from natural language inference tasks (Bowman et al., 2015; Williams et al., 2018), our method outperforms the [Sentence-BERT](tag:sbert) embeddings [GitHub](https://github.com/bohanli/BERT-flow)
        
        2021-04-19 About
        
        SimCSE: Simple Contrastive Learning of Sentence Embeddings
        
        Tags:
        
        BERT + Sentence Embeddings
        Contrastive Learning
        Sentence Embeddings
        
        (by one of the authors of [KEPLER](doc:2020/11/1911_06136_kepler_a_unified_)) a contrastive sentence embedding framework, which can be used to produce sentence embeddings, from either unlabeled or labeled data. > 1. **an unsupervised approach, which takes an input sentence and predicts itself in a contrastive objective, with only standard dropout** used as noise > 2. we draw inspiration from the recent success of learning sentence embeddings from natural language inference (NLI) datasets and incorporate annotated pairs from NLI datasets into contrastive learning by using “entailment” pairs as positives and “contradiction” pairs as hard negatives Cites [[2011.05864] On the Sentence Embeddings from Pre-trained Language Models](doc:2021/04/2011_05864_on_the_sentence_em) (question of the anisotropic semantic space of BERT's sentences)
        
        2021-04-18 About
        
        How to use BERT for finding similar sentences or similar news? · Issue #876 · huggingface/transformers
        
        Tags:
        
        BERT + Sentence Embeddings
        huggingface / transformers
        
        links to [UKPLab/sentence-transformers](doc:2020/07/ukplab_sentence_transformers_s) [Another answer](https://github.com/huggingface/transformers/issues/2986)
        
        2020-07-12 About
      - BERT-based entity linking
        
        [1911.03814] Scalable Zero-shot Entity Linking with Dense Entity Retrieval
        
        Tags:
        
        Arxiv Doc
        BERT-based entity linking
        BLINK
        Entity Retrieval
        NLP@Facebook
        Retriever-Reader
        Zero-shot Entity Linking
        
        > a two stage approach, based on fine-tuned BERT architectures. In the first stage, we do retrieval in a dense space defined by a bi-encoder that independently embeds the mention context and the entity descriptions (Humeau et al., 2019; Gillick et al., 2019). Each retrieved candidate is then examined more carefully with a cross-encoder that concatenates the mention and entity text,
        
        2020-05-02 About
      - BERT fine-tuning
        BERT fine-tuning: instabilities
        
        On Stability of Few-Sample Transformer Fine-Tuning | Kaggle
        
        Tags:
        
        BERT fine-tuning: instabilities
        Fine-tuning
        Kaggle
        
        [[2006.05987] Revisiting Few-sample BERT Fine-tuning](doc:2022/03/2006_05987_revisiting_few_sam)
        
        2022-08-29 About
        
        [2006.05987] Revisiting Few-sample BERT Fine-tuning
        
        Tags:
        
        ADAM (ML)
        Arxiv Doc
        BERT fine-tuning
        BERT fine-tuning: instabilities
        NLP@Stanford
        
        > A study of fine-tuning of BERT contextual representations, with focus on commonly observed instabilities in few-sample scenarios. > The most commonly used optimizer for fine-tuning BERT is BERTADAM, a modified version of the ADAM first-order stochastic optimization method. It differs from the original ADAM algorithm (Kingma & Ba, 2014) in omitting a bias correction step. > > ... We observe that the bias correction omission influences the learning rate, especially early in the fine-tuning process, and is one of the primary reasons for instability in fine-tuning BERT and this is bad when finetuning with less than 10K samples. Pb included in many > open source libraries, including the official implementation huggingface’s Transformers How to solve pb in HuggingFace? > HuggingFace Transformers AdamW has correct_bias parameter set to True by default. Still it's worth noting the importance this parameter serves. [src](doc:2022/08/on_stability_of_few_sample_tran)
        
        2022-03-21 About
        
        On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines (2021)
        
        Tags:
        
        BERT fine-tuning
        BERT fine-tuning: instabilities
        Catastrophic forgetting
        Vanishing gradient
        
        > **an analysis of the fine-tuning instability of BERT-based models and a simple method to fix it** > > Despite the strong empirical performance of fine-tuned models, fine-tuning is an unstable process: training the same model with multiple random seeds can result in a large variance of the task performance. > > 2 potential reasons identified in (Devlin et al., 2019; Lee et al., 2020; Dodge et al., 2020) : > - catastrophic forgetting > - small size of the fine-tuning datasets. > > we show that both hypotheses fail to explain the fine-tuning instability, which is caused by optimization difficulties / **vanishing gradients**). > > A simple but strong baseline that makes fine-tuning BERT-based models significantly more stable than the previously proposed approaches. > > [Github](https://github.com/uds-lsv/bert-stable-fine-tuning)
        
        2021-10-30 About
        
        [2106.10199] BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
        
        Tags:
        
        Allen Institute for AI (AI2)
        Arxiv Doc
        BERT fine-tuning
        Parameter-efficient fine-tuning (PEFT)
        Yoav Goldberg
        
        > BitFit, a sparse-finetuning method where only the bias-terms of the model (or a subset of them) are being modified. We show that **with small-to-medium training data, applying BitFit on pre-trained BERT models is competitive with (and sometimes better than) fine-tuning the entire model.** > **these findings support the hypothesis that finetuning is mainly about exposing knowledge induced by language-modeling training, rather than learning new task-specific linguistic knowledge** -- ???!!! > The focus on modifying a small group of parameters eases deployment, as the vast majority of the parameters of the model are shared between various NLP tasks [GitHub](https://github.com/benzakenelad/BitFit)
        
        2022-09-01 About
        
        [2006.05987] Revisiting Few-sample BERT Fine-tuning
        
        Tags:
        
        ADAM (ML)
        Arxiv Doc
        BERT fine-tuning
        BERT fine-tuning: instabilities
        NLP@Stanford
        
        > A study of fine-tuning of BERT contextual representations, with focus on commonly observed instabilities in few-sample scenarios. > The most commonly used optimizer for fine-tuning BERT is BERTADAM, a modified version of the ADAM first-order stochastic optimization method. It differs from the original ADAM algorithm (Kingma & Ba, 2014) in omitting a bias correction step. > > ... We observe that the bias correction omission influences the learning rate, especially early in the fine-tuning process, and is one of the primary reasons for instability in fine-tuning BERT and this is bad when finetuning with less than 10K samples. Pb included in many > open source libraries, including the official implementation huggingface’s Transformers How to solve pb in HuggingFace? > HuggingFace Transformers AdamW has correct_bias parameter set to True by default. Still it's worth noting the importance this parameter serves. [src](doc:2022/08/on_stability_of_few_sample_tran)
        
        2022-03-21 About
        
        On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines (2021)
        
        Tags:
        
        BERT fine-tuning
        BERT fine-tuning: instabilities
        Catastrophic forgetting
        Vanishing gradient
        
        > **an analysis of the fine-tuning instability of BERT-based models and a simple method to fix it** > > Despite the strong empirical performance of fine-tuned models, fine-tuning is an unstable process: training the same model with multiple random seeds can result in a large variance of the task performance. > > 2 potential reasons identified in (Devlin et al., 2019; Lee et al., 2020; Dodge et al., 2020) : > - catastrophic forgetting > - small size of the fine-tuning datasets. > > we show that both hypotheses fail to explain the fine-tuning instability, which is caused by optimization difficulties / **vanishing gradients**). > > A simple but strong baseline that makes fine-tuning BERT-based models significantly more stable than the previously proposed approaches. > > [Github](https://github.com/uds-lsv/bert-stable-fine-tuning)
        
        2021-10-30 About
      - Bertology
        
        Unveiling Transformers with LEGO - YouTube
        
        Tags:
        
        Bertology
        Emergence (AI)
        LM: reasonning
        Reasoning
        Sebastien Bubeck
        Transformers
        Transformers explained
        YouTube video
        
        > To me, what's good about transformers is that they have relative filters. I mean **a standard NN tests an input against a fixed filter w, but here we test part of x against another part of x**. (#[Self-Attention](tag:self_attention)) > > This potentially allows for reasonning to emerge: the network can associate concepts that it encounters, compare them, make analogies > LEGO: Learning Equality and Group Operations. It's a very **basic reasoning task**, where a sentence is made of clauses defining variables as a function of some other variable, and the goal is to **resolve the value of the variables**.
        
        2022-06-30 About
        
        Chris Olah sur Twitter : "I'm excited to *finally* be making progress on understanding the first MLP layer in large transformer LMs. I've tried really hard and prior to SoLU had little success." / Twitter
        
        Tags:
        
        Bertology
        Chris Olah
        Tweet
        
        2022-06-27 About
        
        Anthropic sur Twitter : "a mathematical framework for trying to reverse engineer transformer language models..."
        
        Tags:
        
        Bertology
        Catherine Olsson
        Chris Olah
        Transformers
        Tweet
        
        2021-12-23 About
        
        BERT, ELMo, & GPT-2: How Contextual are Contextualized Word Representations? | SAIL Blog
        
        Tags:
        
        Bertology
        Contextualized word representations
        EMNLP 2019
        NLP@Stanford
        
        2020-03-28 About
        
        [2002.12327] A Primer in BERTology: What we know about how BERT works
        
        Tags:
        
        Arxiv Doc
        BERT
        Bertology
        Survey / Review
        
        (article praised on [twitter](https://twitter.com/dennybritz/status/1233343170596917248?s=20) by D Britz and Y. Goldberg)
        
        2020-02-28 About
        
        NLP's Clever Hans Moment has Arrived
        
        Tags:
        
        Benjamin Heinzerling
        Bertology
        
        Do neural networks learn what we think they learn? @benbenhh reviews research that suggests that they often instead fall prey to the so-called Clever Hans effect and discusses its implications for NLP.
        
        2020-01-10 About
        
        [2003.05473] Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking (CoNNL 2019)
        
        Tags:
        
        Arxiv Doc
        BERT
        Bertology
        Discuté avec Raphaël
        End-To-End Entity Linking
        Good
        
        Training BERT-base-uncased on English Wikipedia and then fine-tuned and evaluating it on an entity linking (EL) benchmark (EL implemented as a token classification over the entity vocabulary) > BERT+Entity is a straightforward extension on top of BERT, i.e. we initialize BERT with the publicly available weights from the BERT-base-uncased model and add an output classification layer on top of the architecture. Given a contextualized token, the classifier computes the probability of an entity link for each entry in the entity vocabulary. Can BERT’s architecture learn all entity linking steps jointly? To answer: > an extreme simplification of the **entity linking setup that works surprisingly well**: simply cast it as **a per token classification over the entire entity vocabulary** (over 700K classes in our case). > the model is the first that performs entity linking without any pipeline or any heuristics, compared to all prior approaches. We found that with our approach we can learn additional entity knowledge in BERT that helps in entity linking. **However, we also found that almost none of the downstream tasks really required entity knowledge**. ### Related work - > [Durrett and Klein (2014)](/doc/2020/01/a_joint_model_for_entity_analys) were the first to propose jointly modelling Mention detection, Candidate generation and Entity disambiguation in a graphical model and could show that each of those steps are interdependent and benefit from a joint objective This paper uses neural techniques instead of CRF. - > [Yamada](/showprop.do?pptyuri=http%3A%2F%2Fwww.semanlink.net%2F2001%2F00%2Fsemanlink-schema%23arxiv_author&pptyval=Ikuya%2BYamada) (2016, 2017) was the first to investigate neural text representations and entity linking, but their approach is limited to ED. cf. [#Wikipedia2Vec](tag:wikipedia2vec). Compare with [newer work by Yamada](doc:2020/09/1909_01259_neural_attentive_b)
        
        2020-01-09 About
        
        [1911.01464] Emerging Cross-lingual Structure in Pretrained Language Models
        
        Tags:
        
        Arxiv Doc
        Bertology
        Cross-lingual NLP
        NLP@Facebook
        Pre-Trained Language Models
        
        2019-11-06 About
        
        BERT is now part of Google Search, so let’s understand how it reasons
        
        Tags:
        
        BERT
        Bertology
        NLP: negation
        
        2019-10-31 About
        
        Evolution of Representations in the Transformer (2019)
        
        Tags:
        
        Bertology
        EMNLP 2019
        Information bottleneck method
        Information theory AND Deep Learning
        Transformers
        
        Blog post about [this paper](http://127.0.0.1:8080/semanlink/doc/2019/09/_1909_01380_the_bottom_up_evol)
        
        2019-09-16 About
        
        BERT's success in some benchmarks tests may be simply due to the exploitation of spurious statistical cues in the dataset. Without them it is no better then random. : MachineLearning
        
        Tags:
        
        BERT
        Bertology
        
        2019-07-24 About
        
        [1907.07355] Probing Neural Network Comprehension of Natural Language Arguments
        
        Tags:
        
        Arxiv Doc
        BERT
        Bertology
        
        what has BERT learned about argument comprehension? [Comments](/doc/2019/07/bert_s_success_in_some_benchmar)
        
        2019-07-24 About
        
        [1906.04341] What Does BERT Look At? An Analysis of BERT's Attention
        
        Tags:
        
        Arxiv Doc
        Attention mechanism
        BERT
        Bertology
        Chris Manning
        What's encoded by a NN
        
        2019-06-21 About
        
        [1906.02715] Visualizing and Measuring the Geometry of BERT
        
        Tags:
        
        Arxiv Doc
        BERT
        Bertology
        Geometry of language embeddings
        NLP@Google
        Thèse IRIT-Renault: biblio
        Tree embeddings
        
        > At a high level, linguistic features seem to be represented in separate semantic and syntactic subspaces. We find evidence of a fine-grained geometric representation of word senses. We also present empirical descriptions of syntactic representations in both attention matrices and individual word embeddings, as well as a mathematical argument to explain the geometry of these representations
        
        2019-06-07 About
        
        [1905.05950] BERT Rediscovers the Classical NLP Pipeline
        
        Tags:
        
        Arxiv Doc
        Bertology
        
        > We find that the model represents the steps of the traditional NLP pipeline in an interpretable and localizable way, and that the regions responsible for each step appear in the expected sequence: POS tagging, parsing, NER, semantic roles, then coreference. Qualitative analysis reveals that the model can and often does adjust this pipeline dynamically, revising lower-level decisions on the basis of disambiguating information from higher-level representations.
        
        2019-05-18 About
      - BERTopic
        
        Maarten Grootendorst sur X : "BERTopic + LLMs + DataMapPlot"
        
        Tags:
        
        BERTopic
        Tweet
        
        2024-01-06 About
        
        Maarten Grootendorst sur Twitter : "The v0.14 release of BERTopic is here. Fine-tune your topic keywords and labels with models from @OpenAI, @huggingface, @CohereAI, @spacy_io, and @LangChainAI... An overview thread"
        
        Tags:
        
        BERTopic
        Tweet
        
        2023-02-15 About
        
        BERTopic: The Future of Topic Modeling | Pinecone
        
        Tags:
        
        BERTopic
        Pinecone
        
        2022-05-12 About
        
        MaartenGr/BERTopic: Leveraging BERT and c-TF-IDF to create easily interpretable topics.
        
        Tags:
        
        BERT
        BERTopic
        GitHub project
        Topic Modeling
        
        > topic modeling technique that leverages 🤗 transformers and [c-TF-IDF](https://github.com/MaartenGr/cTFIDF) to create dense clusters allowing for easily interpretable topics whilst keeping important words in the topic descriptions. refers to [Top2Vec](doc:2022/03/ddangelov_top2vec_top2vec_lear) [youtube](https://www.youtube.com/watch?v=Qub3PrFvauI) [tweet](https://twitter.com/JayAlammar/status/1594681648121102336?s=20&t=R0G_LrajK9WBtzypwXtD7Q)
        
        2022-03-10 About
      - CamemBERT
        
        Sentence Embedding Fine-tuning for the French Language | by La Javaness R&D | Feb, 2022 | Medium
        
        Tags:
        
        CamemBERT
        NLP: French
        NLP sample code
        sBERT fine-tuning
        
        2022-03-31 About
        
        Sahajtomar/french_semantic · Hugging Face
        
        Tags:
        
        CamemBERT
        Discuté avec Raphaël
        huggingface / transformers
        NLP: French
        Sentence-BERT
        
        2021-10-14 About
        
        Paris NLP Season 4 Meetup #3 – Paris NLP (2020)
        
        Tags:
        
        CamemBERT
        Job matching
        Paris NLP meetup
        Siamese networks
        Triplet Loss
        
        - Siamese CNN for jobs-candidate matching: learning document embeddings with triplet loss. - Sesame street-based naming schemes must fade out, long live CamemBERT et le French fromage!
        
        2020-01-23 About
        
        CamemBERT
        
        Tags:
        
        CamemBERT
        
        language model for French based on the RoBERTa architecture pretrained on the French subcorpus of the OSCAR multilingual corpus
        
        2019-11-10 About
      - DistilBERT
        
        Hugging Face sur Twitter : DistilBERT-cased for Question Answering w/ just 3 lines of javascript
        
        Tags:
        
        DistilBERT
        Hugging Face
        JavaScript
        Question Answering
        Tweet
        
        2020-02-14 About
      - Domain-Specific BERT
        
        NLP | How to add a domain-specific vocabulary (new tokens) to a subword tokenizer already trained like BERT WordPiece | by Pierre Guillou | Medium
        
        Tags:
        
        Domain adaptation: vocabulary
        Domain-Specific BERT
        huggingface / transformers
        Language Model: domain adaptation
        
        2022-03-18 About
        
        [2007.15779] Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
        
        Tags:
        
        Arxiv Doc
        Biomedical NLP
        Domain-Specific BERT
        NLP@Microsoft
        
        > A prevailing assumption is that even domain-specific pretraining can benefit by starting from general-domain language models. In this paper, we challenge this assumption by showing that **for domains with abundant unlabeled text, such as biomedicine, pretraining language models from scratch results in substantial gains over continual pretraining of general-domain language models**
        
        2021-04-11 About
        
        exBERT: Extending Pre-trained Models with Domain-specific Vocabulary Under Constrained Training Resources - ACL Anthology
        
        Tags:
        
        BERT
        Biomedical NLP
        Domain adaptation (NLP)
        Domain-Specific BERT
        EMNLP 2020
        NLP@Harvard
        
        **Focus on the Embedding of Domain-specific Vocabulary.** > exBERT adds a new domain-specific vocabulary and the corresponding embedding layer, as well as a small extension module to the original unmodified model > a pretraining method allowing **low-cost embedding of domain-specific vocabulary in the context of an existing large pre-trained model such as BERT** > exBERT... explicitly incorporates the new domain’s vocabulary, while being able to **reuse the original pre-trained model’s weights as is** to reduce required computation and training data. Specifically, exBERT extends BERT by augmenting its embeddings for the original vocabulary with new embeddings for the domain-specific vocabulary via **a learned small “extension” module**. **The output of the original and extension modules are combined via a trainable weighted sum operation** In a way similar to concept developed in > [[1902.00751] Parameter-Efficient Transfer Learning for NLP](doc:2021/04/1902_00751_parameter_efficien), but not in the fine-tuning paradigm. [Github](https://github.com/cgmhaicenter/exBERT)
        
        2021-04-11 About
        
        [2009.02835] E-BERT: A Phrase and Product Knowledge Enhanced Language Model for E-commerce
        
        Tags:
        
        Arxiv Doc
        Aspect Detection
        BERT + KB
        Domain-Specific BERT
        Product data & NLP
        
        E-BERT, pre-training framework for product data. 1. to benefit from phrase-level knowledge: Adaptive Hybrid Masking, a new masking strategy, which allows the model to adaptively switch from learning preliminary word knowledge to learning complex phrases 2. leveraging product-level knowledge: training E-BERT to predict a product’s associated neighbors (product association) Resources used: - description of millions of products from the amazon dataset (title, description, reviews) - e-commerce phrases: extracted from above dataset using [AutoPhrase](doc:2020/12/autophrase_automated_phrase_mi) - product association graph: pairs of substitutable and complementary products extracted from amazon dataset Not to be confounded with [[1911.03681] E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT](doc:2021/01/1911_03681_e_bert_efficient_)
        
        2020-12-14 About
        
        Domain-Specific BERT Models · Chris McCormick
        
        Tags:
        
        Domain-Specific BERT
        
        Chances are you won’t be able to pre-train BERT on your own dataset, for the following reasons: 1. Pre-training BERT requires a huge corpus 2. Huge Model + Huge Corpus = Lots of GPUs
        
        2020-12-01 About
      - KeyBERT
        
        Maarten Grootendorst sur X : "Introducing KeyLLM. An extension to KeyBERT that can create, extract, and fine-tune keywords using Large Language Models!
        
        Tags:
        
        KeyBERT
        Keyword/keyphrase extraction
        
        2023-09-30 About
        
        MaartenGr/KeyBERT: Minimal keyword extraction with BERT
        
        Tags:
        
        KeyBERT
        Keyword/keyphrase extraction
        
        2022-10-06 About
      - KnowBert
        
        An efficient long-text semantic retrieval approach via utilizing presentation learning on short-text | Complex & Intelligent Systems (2023)
        
        Tags:
        
        BERT
        Information retrieval
        Long documents
        
        long-text retrieval model based on BERT (called LTR-BERT)
        
        2024-01-31 About
        
        Rethinking Query Expansion for BERT Reranking | Advances in Information Retrieval (2020)
        
        Tags:
        
        BERT
        Information retrieval
        Query Expansion
        
        using BERT for Information Retrieval: > We find that traditional word-based query expansion is not entirely applicable
        
        2023-10-29 About
        
        [2002.06275] TwinBERT: Distilling Knowledge to Twin-Structured BERT Models for Efficient Retrieval
        
        Tags:
        
        Arxiv Doc
        BERT
        Knowledge distillation
        Neural Search
        NLP@Microsoft
        
        2023-08-27 About
        
        skeskinen/bert.cpp: ggml implementation of BERT
        
        Tags:
        
        .cpp
        BERT
        Frugal LLMs
        Quantization
        Sentence-BERT
        Sentence Embeddings
        
        > ggml inference of BERT neural net architecture with pooling and normalization from SentenceTransformers (sbert.net). High quality sentence embeddings in pure C++ (with C API). > > The main goal of bert.cpp is to run the BERT model using **4-bit integer quantization on CPU**
        
        2023-05-09 About
        
        Classifying long textual documents (up to 25 000 tokens) using BERT | by Sinequa | (2020)
        
        Tags:
        
        BERT
        Long documents
        Representation of text + metadata
        Sinequa
        Text Classification
        
        > long text + additional textual metadata (such as title, abstract …) and categories (location, authors …).
        
        2023-04-07 About
        
        Prompt Tuning BERT🎯:CommonLit Readability | Kaggle
        
        Tags:
        
        BERT
        NLP sample code
        Prompting/In-context learning
        
        > Prompt-tuning is a simple yet effective mechanism for learning “soft prompts” to condition frozen language models to perform specific downstream tasks.Soft prompts are learned through backpropagation and can be tuned to incorporate signal from any number of labeled examples. Finally, we show that conditioning,a frozen model with soft prompts confers benefits in robustness to domain transfer, as compared to full model tuning.
        
        2022-09-16 About
        
        Active Learning for BERT: An Empirical Study - ACL Anthology
        
        Tags:
        
        Active learning
        BERT
        EMNLP 2020
        Imbalanced Data
        NLP@IBM
        Text Classification
        
        > The use of Actice Learning (AL) with deep pre-trained models has so far received little consideration. > > We study the potential of (i) various AL strategies; (ii) in conjunction with BERT, (iii) within a highly challenging – yet common – real-world scenario of class imbalance and scarce labeled data. focused on binary classification > AL can boost BERT performance, especially in the most realistic scenario in which the initial set of labeled examples is created using keyword-based queries, resulting in a biased sample of the minority class. [Github](https://github.com/IBM/low-resource-text-classification-framework)
        
        2022-09-02 About
        
        Using BERT For Classifying Documents with Long Texts | by Armand Olivares | Medium
        
        Tags:
        
        BERT
        Long documents
        
        2022-06-29 About
        
        [1909.00426] Global Entity Disambiguation with BERT
        
        Tags:
        
        Arxiv Doc
        BERT
        Entity Disambiguation
        Ikuya Yamada
        
        2022-04-18 About
        
        MaartenGr/BERTopic: Leveraging BERT and c-TF-IDF to create easily interpretable topics.
        
        Tags:
        
        BERT
        BERTopic
        GitHub project
        Topic Modeling
        
        > topic modeling technique that leverages 🤗 transformers and [c-TF-IDF](https://github.com/MaartenGr/cTFIDF) to create dense clusters allowing for easily interpretable topics whilst keeping important words in the topic descriptions. refers to [Top2Vec](doc:2022/03/ddangelov_top2vec_top2vec_lear) [youtube](https://www.youtube.com/watch?v=Qub3PrFvauI) [tweet](https://twitter.com/JayAlammar/status/1594681648121102336?s=20&t=R0G_LrajK9WBtzypwXtD7Q)
        
        2022-03-10 About
        
        [2109.06304] Phrase-BERT: Improved Phrase Embeddings from BERT with an Application to Corpus Exploration
        
        Tags:
        
        Arxiv Doc
        BERT
        Phrase embeddings
        
        2022-02-25 About
        
        Domain Transfer with BERT | Pinecone
        
        Tags:
        
        BERT
        Domain adaptation (NLP)
        Pinecone
        
        2022-01-04 About
        
        Making the Most of Data: Augmentation with BERT | Pinecone
        
        Tags:
        
        BERT
        Cross-Encoders
        Pinecone
        Synthetic data (NLP)
        
        2021-12-18 About
        
        Advance BERT model via transferring knowledge from Cross-Encoders to Bi-Encoders | by Chien Vu | Towards Data Science
        
        Tags:
        
        BERT
        Cross-Encoders
        Data Augmentation
        Sentence-BERT
        Sentence Similarity
        
        Data Augmentation Method to improve SBERT Bi-Encoders for Pairwise Sentence Scoring Tasks (Semantic sentence tasks)
        
        2021-12-17 About
        
        [2007.12603] IR-BERT: Leveraging BERT for Semantic Search in Background Linking for News Articles
        
        Tags:
        
        Arxiv Doc
        BERT
        BM25
        Embeddings in Information Retrieval
        Semantic Search
        
        2021-04-12 About
        
        [1902.00751] Parameter-Efficient Transfer Learning for NLP
        
        Tags:
        
        Adapter modules (LM finetuning)
        Arxiv Doc
        BERT
        Continual Learning
        NLP@Google
        Transfer learning in NLP
        
        **Adapter tuning for NLP**. A strategy for tuning a large text model on several downstream tasks, that permits training on tasks sequentially, and that adds only a small number of additional parameters per task. New modules added between layers of a pre-trained network. Parameters of the original network are frozen and therefore may be shared by many tasks. [GitHub google-research/adapter-bert](https://github.com/google-research/adapter-bert)
        
        2021-04-11 About
        
        exBERT: Extending Pre-trained Models with Domain-specific Vocabulary Under Constrained Training Resources - ACL Anthology
        
        Tags:
        
        BERT
        Biomedical NLP
        Domain adaptation (NLP)
        Domain-Specific BERT
        EMNLP 2020
        NLP@Harvard
        
        **Focus on the Embedding of Domain-specific Vocabulary.** > exBERT adds a new domain-specific vocabulary and the corresponding embedding layer, as well as a small extension module to the original unmodified model > a pretraining method allowing **low-cost embedding of domain-specific vocabulary in the context of an existing large pre-trained model such as BERT** > exBERT... explicitly incorporates the new domain’s vocabulary, while being able to **reuse the original pre-trained model’s weights as is** to reduce required computation and training data. Specifically, exBERT extends BERT by augmenting its embeddings for the original vocabulary with new embeddings for the domain-specific vocabulary via **a learned small “extension” module**. **The output of the original and extension modules are combined via a trainable weighted sum operation** In a way similar to concept developed in > [[1902.00751] Parameter-Efficient Transfer Learning for NLP](doc:2021/04/1902_00751_parameter_efficien), but not in the fine-tuning paradigm. [Github](https://github.com/cgmhaicenter/exBERT)
        
        2021-04-11 About
        
        [1901.04085] Passage Re-ranking with BERT
        
        Tags:
        
        Arxiv Doc
        BERT
        Learning to rank
        NLP@Facebook
        Ranking (information retrieval)
        
        a simple re-implementation of BERT for query-based passage re-ranking ["Slides of our WSDM 2021 tutorial "Pretrained Transformers for Text Ranking: BERT and Beyond"](doc:2021/03/rodrigo_nogueira_sur_twitter_)
        
        2021-03-26 About
        
        Rodrigo Nogueira sur Twitter : "Slides of our WSDM 2021 tutorial "Pretrained Transformers for Text Ranking: BERT and Beyond"
        
        Tags:
        
        BERT
        Learning to rank
        Ranking (information retrieval)
        Slides
        Tutorial
        Tweet
        
        2021-03-09 About
        
        kamalkraj/BERT-NER: Pytorch-Named-Entity-Recognition-with-BERT
        
        Tags:
        
        BERT
        GitHub project
        Named Entity Recognition
        
        Use google BERT to do CoNLL-2003 NER !
        
        2021-02-07 About
        
        [1911.03681] E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT
        
        Tags:
        
        Arxiv Doc
        BERT
        BERT + KB
        Discuté avec Raphaël
        Entity embeddings
        Text, KG and embeddings
        
        > way of **injecting factual knowledge about entities into the pretrained BERT model**. (Feeding entity vectors into BERT as if they were wordpiece vectors without additional encoder pretraining) > > **We align [Wikipedia2Vec](tag:wikipedia2vec) entity vectors (Yamada et al., 2016) with BERT's native wordpiece vector space and use the aligned entity vectors as if they were wordpiece vectors**. The resulting entity-enhanced version of BERT (called E-BERT) is similar in spirit to [ERNIE](tag:ernie) (Zhang et al., 2019) and [KnowBert](tag:knowbert) (Peters et al., 2019), but it **requires no expensive further pretraining of the BERT encoder**. > > Our vector space alignment strategy is inspired by cross-lingual word vector alignment Related work on Entity-enhanced BERT: > ([ERNIE](doc:2019/08/_1905_07129_ernie_enhanced_la) and [Knowbert](doc:2020/05/1909_04164_knowledge_enhanced)) are based on the design principle that BERT be adapted to entity vectors. They introduce new encoder layers to feed pretrained entity vectors into the Transformer, and they require additional pretraining to integrate the new parameters. In contrast, E-BERT’s design principle is that entity vectors be adapted to BERT. > > Two other knowledge-enhanced MLMs are [KEPLER](doc:2020/11/1911_06136_kepler_a_unified_) (Wang et al., 2019c) and K-Adapter (Wang et al., 2020)... Their factual knowledge does not stem from entity vectors – instead, they are trained in a multi-task setting on relation classification and knowledge base completion. Not to be cofounded with [[2009.02835] E-BERT: A Phrase and Product Knowledge Enhanced Language Model for E-commerce](doc:2020/12/2009_02835_e_bert_a_phrase_a)
        
        2021-01-12 About
        
        Keyword Extraction with BERT | Towards Data Science
        
        Tags:
        
        BERT
        Keyword/keyphrase extraction
        
        A minimal method for extracting keywords and keyphrases. [GitHub](https://github.com/MaartenGr/KeyBERT/) > uses BERT-embeddings and simple cosine similarity to find the sub-phrases in a document that are the most similar to the document itself.
        
        2020-12-06 About
        
        Salmon Run: Word Sense Disambiguation using BERT as a Language Model
        
        Tags:
        
        BERT
        huggingface / transformers
        NLP sample code
        Word-sense disambiguation
        
        2020-12-01 About
        
        Which flavor of BERT should you use for your QA task? | by Olesya Bondarenko | Towards Data Science
        
        Tags:
        
        BERT
        huggingface / transformers
        Question Answering
        Tutorial
        
        A guide to choosing and benchmarking BERT models for question answering
        
        2020-10-04 About
        
        BERT Word Embeddings Tutorial · Chris McCormick
        
        Tags:
        
        BERT
        Tutorial
        Word embeddings
        
        2020-07-06 About
        
        [1909.03193] KG-BERT: BERT for Knowledge Graph Completion
        
        Tags:
        
        Arxiv Doc
        BERT
        Discuté avec Raphaël
        Knowledge Graph Completion
        Pre-Trained Language Models
        Transformers
        
        Pre-trained language models for knowledge graph completion. **Triples are treated as textual sequences**. (Hum, j'ai déjà vu ça quelque part. Ah, peut-être [RDF2VEC](tag:rdf2vec)? // TODO à voir) Takes entity and relation descriptions of a triple as input and computes scoring function of the triple with the KG-BERT language model > we first treat entities, relations and triples as textual sequences and turn knowledge graph completion into a sequence classification problem. We then fine-tune BERT model on these sequences for predicting the plausibility of a triple or a relation. The method [GitHub](https://github.com/yao8839836/kg-bert)
        
        2020-03-22 About
        
        [1909.07606] K-BERT: Enabling Language Representation with Knowledge Graph
        
        Tags:
        
        Arxiv Doc
        BERT
        Knowledge Graph + Deep Learning
        Knowledge Graphs in NLP
        Language Model
        
        a knowledge-enabled language representation model (K-BERT) with knowledge graphs (KGs), in which triples are injected into the sentences as domain knowledge (Summarized in [Domain adaptation of word embeddings through the exploitation of in-domain corpora and knowledge bases (PhD Thesis 2021)](doc:2022/03/domain_adaptation_of_word_embed), p43)
        
        2020-03-08 About
        
        Unsupervised NER using BERT - Hands-on NLP model review - Quora
        
        Tags:
        
        BERT
        Named Entity Recognition
        
        [GitHub](https://github.com/ajitrajasekharan/unsupervised_NER)
        
        2020-03-06 About
        
        [2002.12327] A Primer in BERTology: What we know about how BERT works
        
        Tags:
        
        Arxiv Doc
        BERT
        Bertology
        Survey / Review
        
        (article praised on [twitter](https://twitter.com/dennybritz/status/1233343170596917248?s=20) by D Britz and Y. Goldberg)
        
        2020-02-28 About
        
        [2002.11402] Detecting Potential Topics In News Using BERT, CRF and Wikipedia
        
        Tags:
        
        Arxiv Doc
        BERT
        Conditional random fields
        Named Entity Recognition
        Wikipedia
        
        2020-02-27 About
        
        Distilling BERT models with spaCy - Towards Data Science (2019)
        
        Tags:
        
        BERT
        Knowledge distillation
        spaCy
        Transfer learning in NLP
        Yves Peirsman
        
        2020-02-15 About
        
        [2002.02925] BERT-of-Theseus: Compressing BERT by Progressive Module Replacing
        
        Tags:
        
        Arxiv Doc
        BERT
        Knowledge distillation
        NLP@Microsoft
        
        approach to compress BERT by progressive module replacing. > Compared to the previous knowledge distillation approaches for BERT compression, our approach leverages only one loss function and one hyper-parameter [Github](https://github.com/JetRunner/BERT-of-Theseus)
        
        2020-02-10 About
        
        Canwen Xu sur Twitter : "WTF? We brutally dismember BERT and replace all his organs?"
        
        Tags:
        
        BERT
        Knowledge distillation
        NLP@Microsoft
        
        [paper](/doc/2020/02/_2002_02925_bert_of_theseus_c)
        
        2020-02-10 About
        
        Building a Search Engine with BERT and TensorFlow - Towards Data Science
        
        Tags:
        
        BERT
        Nearest neighbor search
        NLP and Search
        Search Engines
        TensorFlow
        
        [somewhat related](/doc/2020/01/elasticsearch_meets_bert_build)
        
        2020-01-12 About
        
        Elasticsearch meets BERT: Building Search Engine with Elasticsearch and BERT
        
        Tags:
        
        BERT
        ElasticSearch
        GitHub project
        NLP and Search
        Text Similarity
        
        - Links to [this ES blog post](/doc/2020/01/text_similarity_search_in_elast) - [somewhat related](/doc/2020/01/building_a_search_engine_with_b)
        
        2020-01-10 About
        
        [2003.05473] Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking (CoNNL 2019)
        
        Tags:
        
        Arxiv Doc
        BERT
        Bertology
        Discuté avec Raphaël
        End-To-End Entity Linking
        Good
        
        Training BERT-base-uncased on English Wikipedia and then fine-tuned and evaluating it on an entity linking (EL) benchmark (EL implemented as a token classification over the entity vocabulary) > BERT+Entity is a straightforward extension on top of BERT, i.e. we initialize BERT with the publicly available weights from the BERT-base-uncased model and add an output classification layer on top of the architecture. Given a contextualized token, the classifier computes the probability of an entity link for each entry in the entity vocabulary. Can BERT’s architecture learn all entity linking steps jointly? To answer: > an extreme simplification of the **entity linking setup that works surprisingly well**: simply cast it as **a per token classification over the entire entity vocabulary** (over 700K classes in our case). > the model is the first that performs entity linking without any pipeline or any heuristics, compared to all prior approaches. We found that with our approach we can learn additional entity knowledge in BERT that helps in entity linking. **However, we also found that almost none of the downstream tasks really required entity knowledge**. ### Related work - > [Durrett and Klein (2014)](/doc/2020/01/a_joint_model_for_entity_analys) were the first to propose jointly modelling Mention detection, Candidate generation and Entity disambiguation in a graphical model and could show that each of those steps are interdependent and benefit from a joint objective This paper uses neural techniques instead of CRF. - > [Yamada](/showprop.do?pptyuri=http%3A%2F%2Fwww.semanlink.net%2F2001%2F00%2Fsemanlink-schema%23arxiv_author&pptyval=Ikuya%2BYamada) (2016, 2017) was the first to investigate neural text representations and entity linking, but their approach is limited to ED. cf. [#Wikipedia2Vec](tag:wikipedia2vec). Compare with [newer work by Yamada](doc:2020/09/1909_01259_neural_attentive_b)
        
        2020-01-09 About
        
        Named Entity Recognition with Bert – Depends on the definition
        
        Tags:
        
        BERT
        Hugging Face
        Named Entity Recognition
        NLP sample code
        
        > how you can finetune the Bert model to do state-of-the art named entity recognition Same author: [NER with Lime](/doc/2020/01/interpretable_named_entity_reco)
        
        2020-01-09 About
        
        [1902.10909] BERT for Joint Intent Classification and Slot Filling
        
        Tags:
        
        Arxiv Doc
        BERT
        Intent classification and slot filling
        NLP@Alibaba
        
        > Experimental results show that our proposed joint BERT model outperforms BERT models modeling intent classification and slot filling separately, demonstrating the efficacy of exploiting the relationship between the two tasks. Adding a CRF on top of the model doesn't improve the results.
        
        2020-01-09 About
        
        Lecture 14 – Contextual Vectors | Stanford CS224U: Natural Language Understanding | Spring 2019
        
        Tags:
        
        BERT
        Contextualized word representations
        ELMo
        NLP@Stanford
        
        2020-01-05 About
        
        BERT is now part of Google Search, so let’s understand how it reasons
        
        Tags:
        
        BERT
        Bertology
        NLP: negation
        
        2019-10-31 About
        
        Meet ALBERT: a new ‘Lite BERT’ from Google & Toyota with State of the Art NLP performance and 18x fewer parameters.
        
        Tags:
        
        ALBERT
        BERT
        NLP@Google
        Toyota
        
        2019-10-01 About
        
        Smaller, faster, cheaper, lighter: Introducing DistilBERT, a distilled version of BERT
        
        Tags:
        
        BERT
        Hugging Face
        Knowledge distillation
        
        2019-08-28 About
        
        [1905.07129] ERNIE: Enhanced Language Representation with Informative Entities
        
        Tags:
        
        ACL 2019
        Arxiv Doc
        BERT
        Discuté avec Raphaël
        ERNIE
        Knowledge Graphs in NLP
        Text, KG and embeddings
        Transformers
        
        > We argue that informative entities in **KGs can enhance language representation with external knowledge**. In this paper, we utilize both large-scale textual corpora and KGs to train an enhanced language representation model (ERNIE), which can take full advantage of lexical, syntactic, and knowledge information simultaneously. > ERNIE achieves significant improvements on various knowledge-driven tasks, and meanwhile is comparable with the state-of-the-art model BERT on other common NLP tasks [GitHub](https://github.com/thunlp/ERNIE) WARNING, there is another ERNIE (by [NLP@Baidu](tag:nlp_baidu)): Yu Sun, Shuohuan Wang, Yukun Li, Shikun Feng, Xuyi Chen, Han Zhang, Xin Tian, Danxiang Zhu, Hao Tian, and Hua Wu. 2019. Ernie: Enhanced representation through knowledge integration. This doesn't happen when you choose François-Paul as the name for your child.
        
        2019-08-05 About
        
        What is XLNet and why it outperforms BERT - Towards Data Science
        
        Tags:
        
        BERT
        XLNet
        
        2019-08-02 About
        
        BERT's success in some benchmarks tests may be simply due to the exploitation of spurious statistical cues in the dataset. Without them it is no better then random. : MachineLearning
        
        Tags:
        
        BERT
        Bertology
        
        2019-07-24 About
        
        [1907.07355] Probing Neural Network Comprehension of Natural Language Arguments
        
        Tags:
        
        Arxiv Doc
        BERT
        Bertology
        
        what has BERT learned about argument comprehension? [Comments](/doc/2019/07/bert_s_success_in_some_benchmar)
        
        2019-07-24 About
        
        [1906.04341] What Does BERT Look At? An Analysis of BERT's Attention
        
        Tags:
        
        Arxiv Doc
        Attention mechanism
        BERT
        Bertology
        Chris Manning
        What's encoded by a NN
        
        2019-06-21 About
        
        NLP: Contextualized word embeddings from BERT – Towards Data Science
        
        Tags:
        
        BERT
        
        2019-06-12 About
        
        [1906.02715] Visualizing and Measuring the Geometry of BERT
        
        Tags:
        
        Arxiv Doc
        BERT
        Bertology
        Geometry of language embeddings
        NLP@Google
        Thèse IRIT-Renault: biblio
        Tree embeddings
        
        > At a high level, linguistic features seem to be represented in separate semantic and syntactic subspaces. We find evidence of a fine-grained geometric representation of word senses. We also present empirical descriptions of syntactic representations in both attention matrices and individual word embeddings, as well as a mathematical argument to explain the geometry of these representations
        
        2019-06-07 About
        
        "I made a bet that a Naive Bayes classifier would work as well on humor recognition as a neural net with fine-tuned Bert embeddings. I won"
        
        Tags:
        
        BERT
        Humour
        Naive Bayes classifier
        Rada Mihalcea
        
        [Jeremy Howard's answer](https://forums.fast.ai/t/nlp-challenge-project/44153)
        
        2019-06-06 About
        
        Introducing FastBert — A simple Deep Learning library for BERT Models
        
        Tags:
        
        BERT
        fast.ai
        huggingface / transformers
        
        2019-05-23 About
        
        Robust Language Representation Learning via Multi-task Knowledge Distillation - Microsoft Research
        
        Tags:
        
        BERT
        Ensemble learning
        NLP@Microsoft
        
        Related to [this](/doc/?uri=https%3A%2F%2Farxiv.org%2Fabs%2F1901.11504).
        
        2019-05-19 About
        
        [1904.08398] DocBERT: BERT for Document Classification
        
        Tags:
        
        Arxiv Doc
        BERT
        Text Classification
        
        2019-04-18 About
        
        huggingface/pytorch-pretrained-BERT: The Big-&-Extending-Repository-of-Transformers: Pretrained PyTorch models for Google's BERT, OpenAI GPT & GPT-2, Google/CMU Transformer-XL.
        
        Tags:
        
        BERT
        GitHub project
        PyTorch
        
        2019-03-15 About
        
        [1901.11504] Multi-Task Deep Neural Networks for Natural Language Understanding
        
        Tags:
        
        Arxiv Doc
        BERT
        KD-MKB biblio
        Multi-task learning
        NLP@Microsoft
        NLU
        Pre-Trained Language Models
        
        outperforms BERT in nine of eleven benchmark NLP tasks
        
        2019-02-17 About
        
        Using BERT for state-of-the-art pre-training for natural language processing
        
        Tags:
        
        BERT
        Howto
        
        2019-02-14 About
        
        Jacob Devlin talks about BERT at the Stanford NLP seminar
        
        Tags:
        
        BERT
        Slides
        
        Includes new results such as the effect of the masking strategy, using synthetic training data,...
        
        2019-02-11 About
        
        Romain Vial (Hyperlex) at Paris NLP meetup, slides
        
        Tags:
        
        BERT
        Good
        Information extraction
        Knowledge Graph Completion
        Knowledge Graph Construction
        Long documents
        NLP + juridique
        Paris NLP meetup
        Slides
        
        > Hyperlex is a contract analytics and management solution powered by artificial intelligence. Hyperlex helps companies manage and make the most of their contract portfolio by identifying relevant information and data to manage key contractual commitments. > Take-home message: > > - Sentence representation starts to be well understood empirically > - Large document representation is still an open (and interesting) problem!
        
        2019-01-24 About
        
        The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) – Jay Alammar
        
        Tags:
        
        BERT
        Contextualized word representations
        ELMo
        Transfer learning in NLP
        ULMFiT
        
        2018-12-03 About
        
        Google AI Blog: Open Sourcing BERT: State-of-the-Art Pre-training for Natural Language Processing
        
        Tags:
        
        BERT
        
        2018-11-05 About
        
        GitHub - google-research/bert: TensorFlow code and pre-trained models for BERT
        
        Tags:
        
        BERT
        GitHub
        
        Code and pretrained weights for BERT. Includes scripts to reproduce results. BERT-Base can be fine-tuned on a standard GPU; for BERT-Large, a Cloud TPU is required
        
        2018-11-05 About
        
        [1810.04805] BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
        
        Tags:
        
        Arxiv Doc
        BERT
        Language Model
        Neural machine translation
        Sequence-to-sequence learning
        Transformers
        
        **The "Devlin et al 2019" paper** [Paper Dissected](https://datasciencetoday.net/index.php/en-us/nlp/211-paper-dissected-bert-pre-training-of-deep-bidirectional-transformers-for-language-understanding-explained)
        
        2018-10-12 About
    - Dialogflow
    - Gemini
    - Google LaMDA
    - PaLM
      - PaLM 2
        
        Google AI PaLM 2 – Google AI
        
        Tags:
        
        PaLM 2
        
        2023-05-15 About
    - SGNN
    - T5 (Text-To-Text Transfer Transformer)
      - Flan-T5
        
        Ramsri Goutham Golla sur Twitter : "The most practical open-source competitor to @OpenAI 's GPT-3 is Google's Flan-T5 Here are 5 Flan-T5 resources to try out easily, deploy, or fine-tune it! 🧵" / Twitter
        
        Tags:
        
        Flan-T5
        GPT-3
        Open-source LLM
        Tweet
        
        2023-02-04 About
        
        Shayne Longpre sur Twitter : "What’s the best completely public competitor to #ChatGPT? Flan-T5 beats all public models we tested..."
        
        Tags:
        
        ChatGPT: competitors
        Flan-T5
        Instruction tuning
        Tweet
        
        > It's promising these results don't use any [#RLHF](tag:reinforcement_learning_from_human_feedback) data, or human "alignment", which is expensive to collect and less publicly available. > Key takeaway: finetuning Flan-T5 is better and more compute-efficient than finetuning T5.[src](https://twitter.com/_jasonwei/status/1620864198262804481?s=20&t=hMXLCdqcOFAEbjsfwc_yog)
        
        2023-02-01 About
    - Universal Sentence Encoder
    - Word2vec
      - Doc2Vec
        
        A gentle introduction to Doc2Vec – ScaleAbout – Medium
        
        Tags:
        
        Doc2Vec
        
        2018-02-14 About
        
        Explanation for Doc2Vec - Quora
        
        Tags:
        
        Doc2Vec
        
        2018-02-14 About
        
        [1507.07998] Document Embedding with Paragraph Vectors
        
        Tags:
        
        Arxiv Doc
        Chris Olah
        Doc2Vec
        
        2017-08-20 About
        
        [1405.4053] Distributed Representations of Sentences and Documents
        
        Tags:
        
        Arxiv Doc
        Doc2Vec
        Tomas Mikolov
        
        Paragraph Vector: an unsupervised algorithm that learns fixed-length feature representations from variable-length pieces of texts, such as sentences, paragraphs, and documents.Represents each document by a dense vector which is trained to predict words in the document. Overcomes the weaknesses of the [Bag Of Words](/tag/bag_of_words) model (order of words, semantic of words)
        
        2017-07-10 About
      - Word2vec: howto
        
        More Fun With Word Vectors - Bag of Words Meets Bags of Popcorn | Kaggle
        
        Tags:
        
        Word2vec: howto
        
        > We found that the code above gives about the same (or slightly worse) results compared to the Bag of Words
        
        2017-07-20 About
        
        Can I use word2vec representation to train a weka classifier? - Quora
        
        Tags:
        
        Word2vec: howto
        
        2017-07-20 About
        
        Can I use word2vec to train a machine learning classifier? - Quora
        
        Tags:
        
        Word2vec: howto
        
        2017-07-20 About
        
        An Intuitive Understanding of Word Embeddings: From Count Vectors to Word2Vec
        
        Tags:
        
        gensim
        NLP sample code
        Using word embeddings
        Word2vec: howto
        Word embeddings
        
        Types of word embeddings: - Frequency based Embedding - Count Vector - TF-IDF Vector - Co-Occurrence Vector - Co_occurence matrix (with a fixed context window), size V*V or V * N (Vocab size * subset of V size) matrix. - PCA or SVD: keeping the k most important eigenvalues - Prediction based Embedding - CBOW (Continuous Bag Of Words). 1 hidden layer, one output layer. Predict the probability of a word given a context - Skip-gram. Predict the proba of the context given a word Sample code using gensim
        
        2017-06-09 About
  - Pix2Struct
  - Reformer
  - TensorFlow
    - Tensor2Tensor
    - TensorFlow 2.0
  - VertexAI
    - Google Document AI
- Android
- AngularJS
- Bigtable
- Chrome
  - Chrome extension
- Doc by Google
- Dremel
- Eric Schmidt
- Google: SEO
  - Google Rich Snippets
  - Google Structured Data Testing Tool
- Google + Advertising
  - AdSense
  - Jedi Blue
- Google App Engine
- Google car
- Google Cloud Platform
  - BigQuery
  - VertexAI
    Generative AI support on Vertex AI generally available | Google Cloud Blog
    Tags:
    
    Generative AI
    Google Cloud Platform
    NLP@Google
    VertexAI
    2023-06-09 About
    Tutorial: VS Code with Google Cloud AI Platform as a backend | by Kyle Ziegler | Medium
    Tags:
    
    Google Cloud Platform
    Tutorial
    VS Code
    2023-03-21 About
    Viacheslav Kovalevskyi – Medium
    Tags:
    
    Google Cloud Platform
    cf. [Deep Learning Images For Google Compute Engine, The Definitive Guide | by Viacheslav Kovalevskyi](doc:2022/01/deep_learning_images_for_google)
    
    2022-01-22 About
    Conteneurs de deep learning | Google Cloud
    Tags:
    
    Google Cloud Platform
    2022-01-22 About
    Vertex AI Workbench | Google Cloud
    Tags:
    
    Google Cloud Platform
    2022-01-22 About
    Deep Learning Images For Google Compute Engine, The Definitive Guide | by Viacheslav Kovalevskyi
    Tags:
    
    Google Cloud Platform
    had been useful to me when creating VMs for deep learning some years ago
    
    2022-01-22 About
    Colab+GCP Compute — how to link them together | by Gautham Senthilnathan | Medium
    Tags:
    
    Google Cloud Platform
    Google Colab
    2022-01-14 About
    Détecter le texte dans les fichiers (PDF/TIFF) | API Cloud Vision
    Tags:
    
    Google Cloud Platform
    http://www.semanlink.net/tag/pdf_extract
    OCR
    2020-06-15 About
    Building a real-time embeddings similarity matching system | Solutions | Google Cloud
    Tags:
    
    Approximate nearest-neighbor
    Doc by Google
    Good
    Google Cloud Platform
    Nearest neighbor search
    Sample code
    Similarity queries / Vector search
    Text Similarity
    - an overview of approximate similarity matching - an end-to-end example solution for performing real-time text semantic search
    
    2020-01-11 About
    GCP and Fast Ai v1: A full setup that’ll work – mc.ai
    Tags:
    
    fast.ai
    Google Cloud Platform
    2019-05-03 About
    DeepLearning Images Revision M13/14. PyTorch 1.0. Git Integration. Smaller Boot Time.
    Tags:
    
    fast.ai
    Google Cloud Platform
    PyTorch
    2018-12-12 About
    GCP | fast.ai course v3
    Tags:
    
    fast.ai
    Google Cloud Platform
    2018-12-05 About
    Deploy Your First Deep Learning Model On Kubernetes With Python, Keras, Flask, and Docker
    Tags:
    
    Docker
    Docker-Python
    Flask
    Google Cloud Platform
    Keras
    Kubernetes
    2018-10-07 About
    Cloud Datalab – Outil interactif d'analyse de données | Google Cloud
    Tags:
    
    Google Cloud Platform
    2018-08-22 About
    How to get started with GCP | Google Cloud Platform
    Tags:
    
    Google Cloud Platform
    Tutorial
    2017-07-12 About
- Google dev
- Google Fusion Tables
- Google Groups
- Google Hummingbird
- Google Knowledge Graph
- Google Looker Studio
- Google Maps
- Google Patents
- Google Play
- GooglePlus
- Google ranking
  - PageRank
- Google Refine
- Google Research
  - Google Colab
  - TensorFlow
  - Wikilinks Corpus
- Google Rich Cards
- Google Rich Snippets
- Google Spreadsheets
- Google sucks
- GoogleTechTalks
- Google über alles
- Google Visualization API
- Google Web Toolkit
  - OwlSight
- Googling
- Jedi Blue
- Knowledge Vault
- Webmasters @ Google
Honda
HP
- Steve Cayzer
  - semblog
IBM
- AI@IBM
  - NLP@IBM
- IBM developerWorks
- IBM SPSS Text Analytics for Surveys
- IBM Watson
  - Watson Speech-to-Text
Intel
- Apple-Intel
John Deere
Makolab
- C2GWeb, Product description and Makolab
  - GAO
- Makolab Semantic Day
Microsoft
- AI@Microsoft
  - NLP@Microsoft
    - Bhaskar Mitra
    - E5
    - LoRA
      - DoRA
        
        Jeremy Howard sur X : "FSDP/QDoRA with Llama3 : I believe that this combination is likely to create better task-specific models than anything else at any cost."
        
        Tags:
        
        AI on consumer hardware
        Continued pre-training
        DoRA
        Jeremy Howard
        LLama3
        Parameter-efficient fine-tuning (PEFT)
        Quantization
        Task-specific models
        Twitter thread
        
        2024-04-23 About
      - QLoRA
        
        Answer.AI - You can now train a 70b language model at home
        
        Tags:
        
        AI on consumer hardware
        Jeremy Howard
        QLoRA
        
        2024-03-09 About
    - Microsoft Concept Graph
    - Semantic Kernel
  - Sebastien Bubeck
- Ballmer
- Internet Explorer
  - Javascript RDF Parser in IE
- Microsoft Azure
  - Azure Cognitive Search
  - Azure OpenAI
- Microsoft Media Center
- Microsoft Research
  - Bhaskar Mitra
  - DSSM (Deep Semantic Similarity Model)
  - Microsoft Concept Graph
  - Satori
  - Sebastien Bubeck
    GraphRAG: Unlocking LLM discovery on narrative private data - Microsoft Research
    Tags:
    
    LLM generated Knowledge Graphs
    Microsoft Research
    NLP@Microsoft
    RAG + KG
    > GraphRAG uses **LLM generated knowledge graphs** to provide substantial improvements in question-and-answer performance when conducting document analysis of complex information. > power of **prompt augmentation** when performing **discovery** on private datasets (data that the LLM is not trained on and has never seen before, such as an enterprise’s proprietary research, business documents..) > GraphRAG uses the LLM to **create a knowledge graph based on the private dataset**. This graph is then used alongside graph machine learning to perform **prompt augmentation** at query time. > the GraphRAG approach [can] **discover entities in the query**. This allows the LLM to ground itself in the graph and results in superior answer that contains provenance through links to the original supporting text GraphRAG can answer queries such as "**what are the top five themes in the data?**"
    
    2024-03-07 About
    BioGPT
    Tags:
    
    Biomedical NLP
    Generative Transformer LM
    Microsoft Research
    NLP@Microsoft
    OpenAI GPT
    2023-02-02 About
    Revolutionizing Document AI with Multimodal Document Foundation Models - Microsoft Research
    Tags:
    
    Document AI
    LayoutLM
    Microsoft Research
    NLP@Microsoft
    2023-01-30 About
    Everything you always wanted to know about extreme classification (but were afraid to ask) - Microsoft Research - 2019
    Tags:
    
    Extreme Multi-label Classification
    Manik Varma
    Microsoft Research
    NLP@Microsoft
    2020-08-12 About
    [2001.09522] TaxoExpan: Self-supervised Taxonomy Expansion with Position-Enhanced Graph Neural Network
    Tags:
    
    Arxiv Doc
    Discuté avec Raphaël
    Graph neural networks
    Knowledge Graph Completion
    Microsoft Research
    NLP@Microsoft
    Taxonomies
    Taxonomy expansion task
    Text-Aware KG embedding
    TheWebConf 2020
    how to add a set of new concepts to an existing taxonomy. [Tweet](https://twitter.com/mickeyjs6/status/1253772146142216194?s=20) [GitHub](https://github.com/mickeystroller/TaxoExpan) > we study the taxonomy expansion task: given an existing taxonomy and a set of new emerging concepts, we aim to automatically expand the taxonomy to incorporate these new concepts (without changing the existing relations in the given taxonomy). > To the best of our knowledge, this is the first study on **how to expand an existing directed acyclic graph (as we model a taxonomy as a DAG) using self-supervised learning**. Self-supervised framework, the existing taxonomy being used as training data: it learns a model to predict whether a query concept is the direct hyponym of an anchor concept. > 2 techniques: > > 1. a **position-enhanced graph neural network that encodes the local structure of an anchor concept** in the existing taxonomy, > 2. a noise-robust training objective that enables the learned model to be insensitive to the label noise in the self-supervision data. Regarding 1: uses [GNN](/tag/graph_neural_networks.html) to model the "ego network" of concepts (potential “siblings” and “grand parents” of the query concept). > Regular GNNs fail to distinguish nodes with different relative positions to the query (i.e., some nodes are grand parents of the query while the others are siblings of the query). To address this limitation, we present a simple but effective enhancement to inject such position information into GNNs using position embedding. We show that such embedding can be easily integrated with existing GNN architectures (e.g., [GCN](/tag/graph_convolutional_networks) and GAT) and significantly boosts the prediction performance Regarding point 2: uses InfoNCE loss, cf. [Contrastive Predictive Coding](/doc/?uri=https%3A%2F%2Farxiv.org%2Fabs%2F1807.03748) > Instead of predicting whether each individual ⟨query concept, anchor concept⟩ pair is positive or not, we first group all pairs sharing the same query concept into a single training instance and learn a model to select the positive pair among other negative ones from the group. (Hum, ça me rappelle quelque chose) > assume each concept (in existing taxonomy + set of new concepts) has an initial embedding vector learned from some text associated with this concept. To keep things tractable, only attempts to find a single parent node of each new concept.
    
    2020-04-25 About
    Towards a Conscious AI: A Computer Architecture inspired by Neuroscience - Microsoft Research
    Tags:
    
    Conscience artificielle
    Emotions
    Free will
    Global workspace theory
    Microsoft Research
    Neuroscience
    > This talk describes neuroscientist Bernard Baars’ Global Workspace Model (GWM) of the brain, its implications for understanding consciousness, and a novel computer architecture that it inspires. The Model gives **insight for the design of machines that truly experience (as opposed to simulate) the ecstasy of joy and the agony of pain**. It also gives **a reasonable explanation of free will in a completely deterministic world**.
    
    2019-11-12 About
    Deep InfoMax: Learning good representations through mutual information maximization - Microsoft Research
    Tags:
    
    Computer vision
    Microsoft Research
    Mutual information
    Representation learning
    Self-Supervised Learning
    2019-08-09 About
    Bringing the power of machine reading comprehension to specialized documents - Microsoft Research
    Tags:
    
    Automobile manuals
    Microsoft Research
    NLP@Microsoft
    Reading Comprehension
    Transfer learning
    User manuals
    A transfer learning–based approach for adapting models that have proven good at answering general interest– type questions to documents in specialized domains using only limited amounts of domain-specific example data
    
    2019-07-28 About
    Automatic Tag Recommendation Algorithms for Social Recommender Systems - Microsoft Research (2009)
    Tags:
    
    Automatic tagging
    Microsoft Research
    2018-08-07 About
    Bringing low-resource languages and spoken dialects into play with Semi-Supervised Universal Neural Machine Translation - Microsoft Research
    Tags:
    
    Cross-lingual NLP
    Microsoft Research
    NLP@Microsoft
    2018-05-18 About
    Knowledge Graph and Text Jointly Embedding (2014)
    Tags:
    
    Knowledge Graph Completion
    Knowledge Graph Embeddings
    Microsoft Research
    NLP@Microsoft
    Thèse IRIT-Renault: biblio initiale
    Word + Entity embeddings
    method of **jointly embedding knowledge graphs and a text corpus** so that **entities and words/phrases are represented in the same vector space**. Promising improvement in the accuracy of predicting facts, compared to separately embedding knowledge graphs and text (in particular, enables the prediction of facts containing entities out of the knowledge graph) [cité par J. Moreno](/doc/?uri=https%3A%2F%2Fhal.archives-ouvertes.fr%2Fhal-01626196%2Fdocument)
    
    2018-01-05 About
    Enriching Word Embeddings Using Knowledge Graph for Semantic Tagging in Conversational Dialog Systems - Microsoft Research (2015)
    Tags:
    
    Microsoft Research
    NLP@Microsoft
    Semantic tagging
    Word + Entity embeddings
    > new simple, yet effective approaches to learn domain specific word embeddings. ## Intro > Adapting word embeddings, such as jointly capturing syntactic and semantic information, can further enrich semantic word representations for several tasks, e.g., sentiment analysis (Tang et al. 2014), named entity recognition (Lebret, Legrand, and Collobert 2013), entity-relation extraction (Weston et al. 2013), etc. (Yu and Dredze 2014) has introduced a lightly supervised word embedding learning extending word2vec. They incorporate prior information to the objective function as a regularization term considering synonymy relations between words from Wordnet (Fellbaum 1999). > In this work, we go one step further and investigate if enriching the word2vec word embeddings trained on unstructured/ unlabeled text with domain specific semantic relations obtained from knowledge sources (e.g., knowledge graphs, search query logs, etc.) can help to discover relation aware word embeddings. Unlike earlier work, **we encode the information about the relations between phrases, thereby, entities and relation mentions are all embedded into a low dimensional vector space**. ## Related work (Learning Word Embeddings with Priors) - word2vec - Relational Constrained Model (RTM) (Yu and Dredze 2014) While CBOW learns lexical word embeddings from provided text, the RTM learns embeddings of words based on their similarity to other words provided by a knowledge resource (eg. wordnet) - Joint model (Yu and Dredze 2014) combines CBOW and RTM through linear combination
    
    2017-10-02 About
    A Ranking Approach to Keyphrase Extraction - Microsoft Research (2009)
    Tags:
    
    Keyword/keyphrase extraction
    Learning to rank
    Microsoft Research
    NLP@Microsoft
    Ranking SVM
    Previously, automatic keyphrase extraction was formalized as classification and learning methods for classification were utilized. This paper points out that it is more essential to **cast the keyphrase extraction problem as ranking** and employ a **learning to rank** method to perform the task. As example, it employs Ranking SVM, a state-of-art method of learning to rank, in keyphrase extraction
    
    2017-06-27 About
    Challenges of the email domain for text classification
    Tags:
    
    email classification
    Microsoft Research
    NLP@Microsoft
    JD Brutlag, C Meek - ICML, 2000 - research.microsoft.com<br/> Interactive classification of email into a userdefined hierarchy of folders is a natural domain for application of text classification methods. This domain presents several challenges. First, the user's changing mailfiling habits mandate classification technology ...
    
    2015-10-20 About
    Rethinking the Web as a Personal Archive
    Tags:
    
    http://www.semanlink.net/tag/semanlink2_related
    Microsoft Research
    Personal archives
    WWW 2013
    2013-05-21 About
    Route Prediction from Trip Observations
    Tags:
    
    Destination prediction
    Microsoft Research
    This paper develops and tests algorithms for predicting the end-to- end route of a vehicle based on GPS observations of the vehicle’s past trips. We show that a large portion of a typical driver’s trips are repeated. Our algorithms exploit this fact for prediction by matching the first part of a driver’s current trip with o ne of the set of previously observed trips. Rather than predicting upcoming road segments, our focus is on making long term predictions of the route.
    
    2013-04-10 About
    Real-time query processing for billion node graphs - Microsoft Research
    Tags:
    
    Microsoft Research
    Satori
    2013-03-25 About
- Microsoft Teams
- Sharepoint
- VS Code
  - VS Code sucks
- Windows
  - Windows Copilot
  - Windows Media Player
  - Windows Vista
- Zune
Monsanto
Nike
Nissan
NLP: companies
- Cohere.ai
- Hugging Face
  - AutoNLP
  - Hugging Face: dev
  - huggingface / transformers
  - HuggingFace BigScience
Nokia
- Ora Lassila
Novartis
Nvidia
Pfizer
salesforce
Semantic web company
- French Semantic web company
  - Antidot
  - Mondeca
    - (LOV) Linked Open Vocabularies
  - TEMIS
- Ontoprise
- OpenLink Software
  - Kingsley Idehen
    - OpenLink Ajax Toolkit (OAT)
  - OpenLink Ajax Toolkit (OAT)
  - URIBurner
  - Virtuoso
    - Virtuoso:doc
    - Virtuoso: review
    - Virtuoso Open-Source Edition
    - Virtuoso Universal Server
  - Virtuoso Open-Source Edition
    OpenLink Structured Data Sniffer
    Tags:
    
    Data Sniffer
    OpenLink Software
    Structured data extraction
    2019-05-04 About
    openlink/rdf-editor: The OpenLink RDF Edito... - GitHub
    Tags:
    
    OpenLink Software
    RDF editor
    The OpenLink RDF Editor enables editing of RDF documents (in TURTLE notation) stored in a variety of HTTP accessible documents. Actual document access requires the target document is served from a system that supports at least one of the following open standards: Linked Data Platform (LDP), WebDAV, SPARQL 1.1 Update, or the SPARQL Graph Protocol.
    
    2016-01-15 About
    OpenLink Structured Data Sniffer now available in the Chrome Web Store As an...
    Tags:
    
    Data Sniffer
    Kingsley Idehen
    OpenLink Software
    2015-12-17 About
    OpenLink Data Explorer Extension
    Tags:
    
    OpenLink Software
    RDF browser
    2010-12-28 About
    Description of: http://demo.openlinksw.com/Northwind/Customer/ALFKI
    Tags:
    
    Kingsley Idehen
    Linked Data demo
    Linking Enterprise Data
    OpenLink Software
    Semantic web: evangelization
    2008-09-12 About
    OpenLink Data Explorer :: Firefox Add-ons
    Tags:
    
    Firefox
    OpenLink Software
    2008-09-02 About
- Talis
  - Nodalities
  - Talis platform
  - Talis RDF/JSON
- TopQuadrant
  - Dean Allemang
  - TopBraid
    - TopBraid/SPIN
      - SPIN functions
        
        Describing SPARQL Extension Functions « Lost Boy
        
        Tags:
        
        Leigh Dodds
        SPARQL Extension Functions
        SPIN functions
        
        2011-01-10 About
        
        Ascension Semantica: A New SPIN Cycle
        
        Tags:
        
        SPIN functions
        
        2011-01-09 About
Sony
- Sony Hack
Sun Microsystems
- Java
  - Apple Java
  - Applet
    - TouchGraph
  - blojsom
  - Cocoon
  - Java 5
    - Java 1.5 Mac OS X
  - Java 7
  - Java 8
    - Java 8 lambdas
      - Spark (Java web framework)
        
        Use Elasticsearch in your Java applications
        
        Tags:
        
        ElasticSearch
        Spark (Java web framework)
        Tutorial
        
        2017-05-15 About
        
        Spark Framework: A tiny Java web framework
        
        Tags:
        
        Spark (Java web framework)
        
        2017-05-15 About
  - Java concurrency
  - Java dev
    - Apache Shiro
    - Java: JNI
    - Java profiling
      - JVisualVM
        
        Profiling With VisualVM, Part 2 (NetBeans Profiler)
        
        Tags:
        
        JVisualVM
        
        2015-02-09 About
    - Java web dev
      - Java microframeworks
        Spark (Java web framework)
        
        Jump into Java microframeworks, Part 1: Introduction | JavaWorld
        
        Tags:
        
        Java microframeworks
        
        2017-05-16 About
    - JCS - Java Caching System
    - JDBC
    - Jena
      - Elda
        
        Elda -- an implementation of the Linked Data API
        
        Tags:
        
        Elda
        
        2011-01-24 About
      - Jena : Introduction
        
        Introduction to Jena
        
        Tags:
        
        Jena : Introduction
        
        2005-10-28 About
      - Jena and database
        Jena TDB
        
        TDB/QuadFilter - Jena Wiki
        
        Tags:
        
        Jena TDB
        
        This page describes how to filter quads at the lowest level of TDB. It can be used to hide certain quads (tripes in named graphs) or triples.
        
        2011-03-15 About
        
        jena-dev : Message: Re: [jena-dev] Could TDB sort triples by objects?
        
        Tags:
        
        Jena dev
        Jena TDB
        
        2010-12-22 About
        
        SDB: A SPARQL Database for Jena
        
        SDB/Store Description - Jena wiki
        
        Tags:
        
        Jena: assembler
        SDB: A SPARQL Database for Jena
        
        2008-11-17 About
        
        SDB/Dataset Description - Jena wiki
        
        Tags:
        
        Jena: assembler
        SDB: A SPARQL Database for Jena
        
        2008-11-17 About
        
        SDB - Jena wiki
        
        Tags:
        
        SDB: A SPARQL Database for Jena
        
        a new database subsystem for Jena. It uses a conventional relational database, rewriting SPARQL to SQL. The relational database tools for load balancing, security, clustering, backup and administration can all be used to manage the installation. Currently it works with Oracle, MS SQL Server, PostgreSQL, MySQL, Apache Derby and HSQLDB databases.
        
        2007-07-02 About
        
        OWLIM | Ontotext
        
        Tags:
        
        Jena and database
        TripleStore
        
        OWLIM is a family of semantic repositories, or RDF database management systems, with the following characteristics: native RDF engines, implemented in Java delivering full performance through both Sesame and Jena robust support for the semantics of RDFS, OWL 2 RL and OWL 2 QL best scalability, loading and query evaluation performance
        
        2012-06-13 About
        
        jena-dev : Message: Joseki requirements
        
        Tags:
        
        Jena and database
        Jena dev
        Joseki
        
        2009-06-26 About
        
        Berlin SPARQL Benchmark V2 - Results for Sesame, Virtuoso, Jena TDB, D2R Server, and MySQL
        
        Tags:
        
        Benchmark
        Jena and database
        SPARQL
        
        2008-09-24 About
        
        Create Scalable Semantic Applications with Database-Backed RDF Stores
        
        Tags:
        
        Jena and database
        TripleStore
        
        2008-09-10 About
        
        Jena Relational Database backend
        
        Tags:
        
        Jena and database
        
        2008-09-10 About
      - Jena dev
        Implementing a Jena Graph
        
        ARQtick: Jena-Mulgara : example of implementing a Jena graph
        
        Tags:
        
        Andy Seaborne
        Implementing a Jena Graph
        
        2009-09-18 About
        
        Jena: assembler
        
        SDB/Store Description - Jena wiki
        
        Tags:
        
        Jena: assembler
        SDB: A SPARQL Database for Jena
        
        2008-11-17 About
        
        SDB/Dataset Description - Jena wiki
        
        Tags:
        
        Jena: assembler
        SDB: A SPARQL Database for Jena
        
        2008-11-17 About
        
        the Assembler howto
        
        Tags:
        
        Howto
        Jena: assembler
        
        2008-11-17 About
        
        Jena Assembler (D2RQ V0.5 - User Manual)
        
        Tags:
        
        D2RQ
        Jena: assembler
        
        2008-11-17 About
        
        castagna/jena-examples · GitHub
        
        Tags:
        
        GitHub project
        Jena dev
        
        2012-12-09 About
        
        Re: Dealing with expensive queries (jena users mailing list)
        
        Tags:
        
        Jena dev
        
        2012-12-01 About
        
        jena-dev : Message: Re: [jena-dev] Could TDB sort triples by objects?
        
        Tags:
        
        Jena dev
        Jena TDB
        
        2010-12-22 About
        
        jena-dev : Message: Joseki requirements
        
        Tags:
        
        Jena and database
        Jena dev
        Joseki
        
        2009-06-26 About
        
        SourceForge.net: Jena: SCM
        
        Tags:
        
        CVS
        Jena dev
        
        2009-05-28 About
        
        Binding Java Objects to RDF
        
        Tags:
        
        Jena dev
        Semantic Web and OOP
        
        2009-05-14 About
        
        jena-dev : Message: Creating custom model for legacy system (Newbie question)
        
        Tags:
        
        Converting data into RDF
        Jena dev
        
        How to present the data from a legacy system (not an RDB) as RDF and thought I'd use Jena as a facade?
        
        2008-11-20 About
        
        jena-dev : Message: Re: [jena-dev] Checking URI validity before adding it to a model
        
        Tags:
        
        Jena dev
        URI
        
        2007-04-25 About
      - Jena Fuseki
        
        Sparql Secrets In Jena-Fuseki - DataScienceCentral.com
        
        Tags:
        
        Jena Fuseki
        
        2022-07-26 About
        
        Thesaurus-augmented Search with Jena Text | ZBW Labs
        
        Tags:
        
        Jena
        Jena Fuseki
        Lucene
        Semanlink related
        SKOS
        Text Search
        Thesaurus
        
        2013-08-12 About
      - Jena GRDDL Reader
        
        Jena GRDDL Reader
        
        Tags:
        
        Jena GRDDL Reader
        
        2007-07-17 About
      - Jena rules
        
        Lost Boy: Fun with Jena Rules
        
        Tags:
        
        Jena rules
        Tagging
        
        Extracting richer metadata from tagging conventions
        
        2005-06-24 About
      - Jena TDB
      - Jena User Conference
        
        chillyinside.com » Blog Archive » 2006 Jena User Conference
        
        Tags:
        
        Jena User Conference
        LinkTo Semanlink
        
        2006-05-26 About
        
        Jena User Conference - some interesting UI related papers/presentations from Shabajee, Paul on 2006-05-17 (public-semweb-ui@w3.org from May 2006)
        
        Tags:
        
        Jena User Conference
        LinkTo Semanlink
        
        2006-05-25 About
        
        2006 Jena User Conference - proceedings
        
        Tags:
        
        Jena User Conference
        
        2006-05-17 About
        
        JUC day 1: photos
        
        Tags:
        
        Jena User Conference
        
        2006-05-13 About
        
        Lost Boy: 2006 Jena User Conference Programme
        
        Tags:
        
        Jena User Conference
        Leigh Dodds
        
        2006-04-13 About
        
        Jena User Conference - programme up
        
        Tags:
        
        Danny Ayers
        Jena User Conference
        
        2006-04-13 About
        
        Burningbird » ‘allo Jena
        
        Tags:
        
        Jena User Conference
        Shelley Powers
        
        2006-04-13 About
        
        2006 Jena User Conference - schedule
        
        Tags:
        
        Jena User Conference
        
        2006-04-13 About
      - Jsonld/Jena
        
        Java Code Example com.github.jsonldjava.core.JsonLdOptions
        
        Tags:
        
        Jsonld/Jena
        Jsonld-java
        Sample code
        
        2016-04-09 About
        
        JsonLdOptions
        
        Tags:
        
        Jsonld/Jena
        
        2016-04-09 About
        
        Serialization of JSON-LD using appropriate context · Issue #101 · jsonld-java/jsonld-java
        
        Tags:
        
        Jsonld/Jena
        
        2016-04-09 About
        
        Jena: controlling JSON-LD output
        
        Tags:
        
        Jsonld/Jena
        
        2016-04-09 About
      - NG4J
        
        NG4J - Named Graphs API for Jena
        
        Tags:
        
        NG4J
        
        2007-07-13 About
      - semblog
      - SPARQL AND Jena
        ARQ
        ARQ property functions
        
        Message: RE: [jena-dev] ARQ property functions: is it possible to use a blank node as argObject?
        
        Tags:
        
        Andy Seaborne
        ARQ property functions
        fps' post
        
        2008-12-12 About
        
        ARQ: a question about property functions
        
        Tags:
        
        Andy Seaborne
        ARQ property functions
        Dev tips
        fps' post
        
        when handling a describe query with ARQ, is it possible to add statements to the returned RDF, based on a "magic property"? (Yes, it is)
        
        2008-07-14 About
        
        Lost Boy: Writing an ARQ Extension Function
        
        Tags:
        
        ARQ property functions
        Leigh Dodds
        
        2008-04-01 About
        
        Tarql: SPARQL for Tables - cygri/tarql · GitHub
        
        Tags:
        
        ARQ
        CSV
        Richard Cyganiak
        SPARQL 1.1
        
        SPARQL for Tables Tarql is a command-line tool for converting CSV files to RDF using SPARQL 1.1 syntax. It's written in Java and based on Apache ARQ.
        
        2013-02-08 About
        
        castagna/SARQ · GitHub
        
        Tags:
        
        ARQ
        Paolo Castagna
        
        SARQ is a combination of ARQ and Solr. It gives ARQ the ability to perform free text searches using a remote Solr server. Lucene indexes in Solr are additional information for accessing the RDF graph, not storage for the graph itself.
        
        2012-05-04 About
        
        Trying SPARQL 1.1 new query features with ARQ - bobdc.blog
        
        Tags:
        
        ARQ
        Bob DuCharme
        SPARQL 1.1
        
        2010-08-20 About
        
        ARQ - Property Paths
        
        Tags:
        
        ARQ
        
        2010-06-10 About
        
        ARQ - Documentation and Resources
        
        Tags:
        
        ARQ
        
        2008-08-11 About
        
        Lost Boy: Parameterised Queries with SPARQL and ARQ
        
        Tags:
        
        ARQ
        Leigh Dodds
        SPARQL Tips
        
        2008-07-06 About
        
        ARQ - Querying Remote SPARQL Services
        
        Tags:
        
        ARQ
        Querying Remote SPARQL Services
        
        2008-01-04 About
        
        ARQ - A SPARQL Processor for Jena
        
        Tags:
        
        ARQ
        
        2005-10-10 About
        
        Jena TDB
        Joseki
        
        jena-dev : Message: Joseki requirements
        
        Tags:
        
        Jena and database
        Jena dev
        Joseki
        
        2009-06-26 About
        
        RDF Net API
        
        Tags:
        
        Andy Seaborne
        Joseki
        RDF Net API
        
        A simple network interface to remote RDF models. There are two sets of functionality: a basic access mechanism that provides query capability to a collection of RDF statements, and an update mechanism where, subject to security constraints (not discussed here), changes to the RDF model can be made.
        
        2008-01-09 About
        
        Joseki - A SPARQL Server for Jena
        
        Tags:
        
        Joseki
        
        2008-01-04 About
        
        SDB: A SPARQL Database for Jena
        
        Federated SPARQL queries - bobdc.blog
        
        Tags:
        
        Bob DuCharme
        SPARQL AND Jena
        
        2010-01-20 About
        
        ARQ - SPARQL Tutorial
        
        Tags:
        
        SPARQL AND Jena
        SPARQL Tutorial
        
        2007-07-07 About
        
        Search RDF data with SPARQL (and Jena)
        
        Tags:
        
        IBM developerWorks
        SPARQL AND Jena
        
        2007-04-20 About
        
        danbri’s foaf stories » SPARQLing Protégé-OWL Jena integration
        
        Tags:
        
        Dan Brickley
        OWL
        Protégé
        SPARQL AND Jena
        
        2005-08-18 About
    - jersey
      - jersey/RDF
        
        Sommer — Java.net
        
        Tags:
        
        Henry Story
        jersey/RDF
        
        Semantic Object (Medata) Mapper
        
        2012-03-29 About
        
        Serializing Apache Jena's RDF resources via JAXB into JSON docments in a JAX-RS context.
        
        Tags:
        
        Jena
        jersey/RDF
        RDF-in-JSON
        
        2012-03-19 About
        
        Serialising Java Objects to RDF with Jersey (The Sun BabelFish Blog)
        
        Tags:
        
        Henry Story
        jersey/RDF
        
        2011-08-19 About
      - Jersey Cache-Control
        
        http - EntityTag - Value, Caching, Comparison - how to in Jersey - Stack Overflow
        
        Tags:
        
        Jersey Cache-Control
        Stack Overflow
        
        2017-04-11 About
        
        rest - HTTP content negotiation conflicts in JAX-RS/Jersey? - Stack Overflow
        
        Tags:
        
        Jersey Cache-Control
        
        2016-04-01 About
        
        Implementing ETag Caching with Jersey
        
        Tags:
        
        Jersey Cache-Control
        
        2015-05-14 About
        
        HTTP Caching in Java with JAX-RS | Heroku Dev Center
        
        Tags:
        
        JAX-RS
        Jersey Cache-Control
        
        2015-05-14 About
        
        java - Jersey: Default Cache Control to no-cache - Stack Overflow
        
        Tags:
        
        Annotations (java)
        Jersey Cache-Control
        Stack Overflow
        
        Cache-Control Using Annotations With Jersey
        
        2015-05-14 About
        
        Cache-Control using annotations with Jersey - LexTech
        
        Tags:
        
        Jersey Cache-Control
        
        2015-02-11 About
        
        JAX-RS RESTEasy Cache control with ETag example - How To Do In Java
        
        Tags:
        
        Cache
        JAX-RS
        Jersey Cache-Control
        
        2015-02-11 About
    - JUnit
      - HttpUnit
        
        Create test cases for Web applications
        
        Tags:
        
        HttpUnit
        IBM developerWorks
        
        2008-11-18 About
        
        Test Web applications with HttpUnit - JavaWorld
        
        Tags:
        
        HttpUnit
        
        2008-11-18 About
    - JVisualVM
    - log4j
    - Servlet
      - ISP / Servlet Hosting
        
        DailyRazor Hosting - Advanced Java Hosting, JSP Hosting, Servlets, ASP.NET, ASP 3.0, PHP Hosting
        
        Tags:
        
        ISP / Servlet Hosting
        
        2005-11-05 About
      - Java Server Faces
        MyFaces
        
        Apache MyFaces
        
        Tags:
        
        MyFaces
        
        2005-09-22 About
      - Servlet 3.0
        
        Servlet 3.0, les 3 points marquants | Blog Xebia France
        
        Tags:
        
        Servlet 3.0
        
        2011-02-03 About
      - Tomcat
        Docker-Tomcat
        
        DevOps: Docker and Apache Tomcat
        
        Tags:
        
        Docker-Tomcat
        
        2016-04-06 About
        
        A Java geekA Java geek > From Vaadin to Docker, a novice's journey
        
        Tags:
        
        Docker-Mac
        Docker-Tomcat
        
        2016-04-06 About
        
        boot2docker - Docker add warfile to official Tomcat image - Stack Overflow
        
        Tags:
        
        Docker-Tomcat
        
        2016-04-06 About
        
        Integrating Tomcat with Apache
        
        How do I redirect from Apache to Tomcat?
        
        Tags:
        
        Integrating Tomcat with Apache
        
        2012-03-02 About
        
        Tomcat 7
        
        Top 7 Features in Tomcat 7: The New and the Improved - Developer.com
        
        Tags:
        
        Tomcat 7
        
        2012-04-10 About
        
        Tomcat tips
        
        Bits n Bobs » tomcat
        
        Tags:
        
        Eclipse tip
        Mémoire (informatique)
        Tomcat in Eclipse
        Tomcat tips
        
        including how to increase mem for tomcat in eclipse
        
        2009-04-27 About
        
        SolrTomcat - URI Charset Config
        
        Tags:
        
        Solr (not english only)
        Tomcat
        
        2012-11-26 About
        
        Build a RESTful Web service using Jersey and Apache Tomcat
        
        Tags:
        
        jersey
        Tomcat
        
        2011-07-29 About
        
        PHP/Java Bridge
        
        Tags:
        
        PHP
        Tomcat
        
        Running PHP applications in Tomcat 6
        
        2011-04-05 About
        
        REST: Jersey configuration on Tomcat - Surya Suravarapu's Blog
        
        Tags:
        
        jersey
        Tomcat
        
        2011-02-04 About
        
        java.net.BindException: Address already in use:8080 ... uhm, who's listening? - Java-Monitor Forum
        
        Tags:
        
        Dev tips
        Tomcat
        
        2010-02-11 About
        
        FAQ/CharacterEncoding - Tomcat Wiki
        
        Tags:
        
        FAQ
        Servlet
        SPARQL
        Tomcat
        URI encoding
        
        The fact is: there is no char encoding in a GET request. So an unmodified installation of Tomcat will parse the URI in a GET as ISO-8859-1 ("the Servlet spec requires it"). But a SPARQL request, for instance, requires it to be UTF-8 encoded. So it can't work!!!<br/> Tomcat people insist that their behavior is a feature. Maybe it's not a bug, but I doubt it is a feature. To get tomcat working otherwise, you have to set a parameter in the server.xml (connector, URIEncoding) What if you don't control the tomcat install? Maybe the best thing to do is to parse the parameters from request.getQueryString(). In my web apps until now, I double UTF-8 encode the params in a GET (for instance, the semanlink bookmarklet)
        
        2008-10-12 About
        
        Java UTF–8 international character support with Tomcat and Oracle, 26/03/07, Kieran's blog
        
        Tags:
        
        Diacritics in URI
        Tomcat
        
        2008-10-11 About
        
        Tomcat - Bug 23929 – request.setCharacterEncoding(String) doesn't work
        
        Tags:
        
        bug
        Tomcat
        URI encoding
        
        2008-10-11 About
        
        How to get international unicode characters from a a form input field/servlet parameter into a string?
        
        Tags:
        
        Diacritics in URI
        Servlet
        Tomcat
        
        2008-10-11 About
        
        EclipseZone - Problems publishing webapp to Tomcat ...
        
        Tags:
        
        Tomcat
        Tomcat in Eclipse
        
        2007-10-22 About
        
        Tomcat 5 - Documentation Index
        
        Tags:
        
        Developer documentation
        Tomcat
        
        2006-09-15 About
        
        Server Configuration Reference - The Context Container
        
        Tags:
        
        Developer documentation
        Tomcat
        
        2006-09-13 About
        
        Configuring Tomcat's URI encoding
        
        Tags:
        
        Developer documentation
        Tomcat
        URI encoding
        
        By default, Tomcat uses ISO-8859-1 character encoding when decoding URLs received from a browser.
        
        2006-09-12 About
        
        General purpose build script for web applications and web services
        
        Tags:
        
        Java dev
        Servlet
        Tomcat
        
        including enhanced support for deploying directly to a Tomcat 5 based server.
        
        2006-09-03 About
        
        The Glove Compartment » Tomcat 5.5, mod_jk, and OS X’s built-in Apache
        
        Tags:
        
        Apache web server
        Mac OS X Web serving
        Tomcat
        
        2006-01-22 About
        
        Sysdeo Eclipse Tomcat plugin
        
        Tags:
        
        Eclipse
        Tomcat
        
        2006-01-07 About
  - Javadoc
  - Java in python
  - Java library
  - JavaOne
  - Java Server Faces
  - Java tip
  - Java tool
    - Mallet
  - JAX-RS
    - jersey
      Jersey vs. RESTEasy: A JAX-RS Implementation Comparison - Genuitec
      
      Tags:
      
      JAX-RS
      jersey
      REST
      RESTEasy
      
      2018-08-05 About
      HTTP Caching in Java with JAX-RS | Heroku Dev Center
      
      Tags:
      
      JAX-RS
      Jersey Cache-Control
      
      2015-05-14 About
      JAX-RS RESTEasy Cache control with ETag example - How To Do In Java
      
      Tags:
      
      Cache
      JAX-RS
      Jersey Cache-Control
      
      2015-02-11 About
  - JPA
  - JSP
    - Struts
  - Scala
  - Servlet
  - Swing
Tech company
- Semantic web company
  Data Virtuality
  Tags:
  - Data Warehouse
  - Tech company
  2014-04-30 About
Thomson Reuters
- Calais
Toyota
TSMC
Walmart
Yahoo!
- Peter Mika
- SearchMonkey
- Yahoo - My Web 2.0

1 Document (Long List)

The World's 50 Most Innovative Companies Interactive Scoreboard

Tags:

2008-05-04 About

Properties

rdf:type : sl:Tag
skos:prefLabel : Entreprise@fr