]> 2021-05-21T12:00:38Z 2021-05-21 Un partenariat plantes - champignons à l’origine de la végétalisation terrestre | CNRS High-depth African genomes inform human migration and health | Nature (2020) 2021-05-08T14:02:59Z 2021-05-08 2021-05-26T12:13:33Z 2021-05-26 > One of the easiest ways to get started with neural networks is by loading pre-trained neural networks through the HuggingFace Transformers pipeline interface Transformers Pipelines.ipynb - Colaboratory 2021-05-21T12:09:43Z Making sense of raw input 2021-05-21 >... this way we are able to **jointly learn** how to perceive (**mapping raw sensory information to concepts**) and apperceive (**combining concepts into declarative rules**) cf. [Making sense of sensory input](doc:2021/04/1910_02227_making_sense_of_se) Sinong Wang 2021-04-29T22:52:26Z Large pre-trained language models (LMs) have demonstrated remarkable ability as few-shot learners. However, their success hinges largely on scaling model parameters to a degree that makes it challenging to train and serve. In this paper, we propose a new approach, named as EFL, that can turn small LMs into better few-shot learners. The key idea of this approach is to reformulate potential NLP task into an entailment one, and then fine-tune the model with as little as 8 examples. We further demonstrate our proposed method can be: (i) naturally combined with an unsupervised contrastive learning-based data augmentation method; (ii) easily extended to multilingual few-shot learning. A systematic evaluation on 18 standard NLP tasks demonstrates that this approach improves the various existing SOTA few-shot learning methods by 12\%, and yields competitive few-shot performance with 500 times larger models, such as GPT-3. > a new approach, named as EFL, that can turn small LMs into better few-shot learners. The key idea of this approach is to reformulate potential NLP task into an entailment one, and then fine-tune the model with as little as 8 examples > > For instance, we can reformulate a sentiment classification task as a textual entailment one with an input sentence S1 as xin = [CLS]S1[SEP]S2[EOS]; where S2 = This indicates positive user sentiment, and let the language modelMto determine the if input sentence S1 entails the label description S2 Han Fang 2104.14690 2021-05-03T23:05:39Z Hao Ma 2021-04-29T22:52:26Z [2104.14690] Entailment as Few-Shot Learner 2021-05-03 Entailment as Few-Shot Learner Hanzi Mao Sinong Wang Madian Khabsa 2021-05-25 2021-05-25T23:56:44Z An Introduction to Knowledge Graphs | SAIL Blog maps, timeline, bibliographies thématiques 2021-05-03 2021-05-03T00:42:29Z The History of West Africa at a Glance Jingyang Li Recent advances in Knowledge Graph Embedding (KGE) allow for representing entities and relations in continuous vector spaces. Some traditional KGE models leveraging additional type information can improve the representation of entities which however totally rely on the explicit types or neglect the diverse type representations specific to various relations. Besides, none of the existing methods is capable of inferring all the relation patterns of symmetry, inversion and composition as well as the complex properties of 1-N, N-1 and N-N relations, simultaneously. To explore the type information for any KG, we develop a novel KGE framework with Automated Entity TypE Representation (AutoETER), which learns the latent type embedding of each entity by regarding each relation as a translation operation between the types of two entities with a relation-aware projection mechanism. Particularly, our designed automated type representation learning mechanism is a pluggable module which can be easily incorporated with any KGE model. Besides, our approach could model and infer all the relation patterns and complex relations. Experiments on four datasets demonstrate the superior performance of our model compared to state-of-the-art baselines on link prediction tasks, and the visualization of type clustering provides clearly the explanation of type embeddings and verifies the effectiveness of our model. Bo Li 2021-05-17T16:47:20Z 2009.12030 2020-09-25T04:27:35Z Guanglin Niu Guanglin Niu Yongfei Zhang 2020-10-06T13:52:59Z AutoETER: Automated Entity Type Representation for Knowledge Graph Embedding head_type + relation = tail_type (Hum, mais pour une relation entre 2 entités de même type ?) Shiliang Pu 2021-05-17 [2009.12030] AutoETER: Automated Entity Type Representation for Knowledge Graph Embedding > unsupervised method to automatically extract keyphrases from a document, that **only requires the document itself** > > 1. We extract **candidate phrases from the text, based on part-of-speech sequences**. More precisely, we keep only those phrases that consist of zero or more adjectives followed by one or multiple nouns (Wan and Xiao, 2008). > 2. We use sentence embeddings **to embed both the candidate phrases and the document itself in the same high-dimensional vector space** > 3. We rank the candidate phrases to select the output keyphrases. In addition we show how to improve the ranking step, by providing a way to tune the diversity of the extracted keyphrases. Simple Unsupervised Keyphrase Extraction using Sentence Embeddings - ACL Anthology (2018) 2021-05-31T11:47:52Z 2021-05-31 Cites [Matching the Blanks: Distributional Similarity for Relation Learning](doc:2021/05/1906_03158_matching_the_blank) 2021-05-13 2021-05-13T00:29:13Z CTLR@WiC-TSV: Target Sense Verification using Marked Inputs and Pre-trained Models (2021) Nicholas FitzGerald General purpose relation extractors, which can model arbitrary relations, are a core aspiration in information extraction. Efforts have been made to build general purpose extractors that represent relations with their surface forms, or which jointly embed surface forms with relations from an existing knowledge graph. However, both of these approaches are limited in their ability to generalize. In this paper, we build on extensions of Harris' distributional hypothesis to relations, as well as recent advances in learning text representations (specifically, BERT), to build task agnostic relation representations solely from entity-linked text. We show that these representations significantly outperform previous work on exemplar based relation extraction (FewRel) even without using any of that task's training data. We also show that models initialized with our task agnostic representations, and then tuned on supervised relation extraction datasets, significantly outperform the previous methods on SemEval 2010 Task 8, KBP37, and TACRED. Tom Kwiatkowski 2019-06-07T15:26:50Z Livio Baldini Soares 2021-05-13T00:39:03Z Matching the Blanks: Distributional Similarity for Relation Learning 2021-05-13 Jeffrey Ling > a new method of learning relation representations directly from text > > First, we study the **ability of the Transformer neural network architecture (Vaswani et al., 2017) to encode relations between entity pairs**, and we identify a method of representation that outperforms previous work in supervised relation extraction. Then, we present a method of training this relation representation **without any supervision from a knowledge graph or human annotators** from widely available distant supervision in the form of entity linked text > > **we assume** access to a corpus of text in which entities have been linked to unique identifiers and we define a relation statement to be a block of text containing two marked entities. Livio Baldini Soares [1906.03158] Matching the Blanks: Distributional Similarity for Relation Learning 2019-06-07T15:26:50Z 1906.03158 [Refers to](doc:2021/05/ctlr_wic_tsv_target_sense_veri) Is Word Sense Disambiguation outdated? | by Anna Breit | May, 2021 | Medium 2021-05-13 2021-05-13T00:27:16Z Heinrich Barth and the Western Sudan 2021-05-05 2021-05-05T10:30:25Z Dejiao Zhang Dejiao Zhang 2021-05-20T16:55:29Z 2103.12953 Kathleen McKeown 2021-05-20 2021-03-24T03:05:17Z Bing Xiang Henghui Zhu > The method we propose, learns discriminative features from both an autoencoder and a sentence embedding ([SIF embeddings](tag:sif_embeddings)), then uses assignments from a clustering algorithm as supervision to update weights of the encoder network. 2021-05-20T16:42:46Z 2021-05-20 A Self-Training Approach for Short Text Clustering - (Hadifar 2019) 2021-03-24T03:05:17Z Unsupervised clustering aims at discovering the semantic categories of data according to some distance measured in the representation space. However, different categories often overlap with each other in the representation space at the beginning of the learning process, which poses a significant challenge for distance-based clustering in achieving good separation between different categories. To this end, we propose Supporting Clustering with Contrastive Learning (SCCL) -- a novel framework to leverage contrastive learning to promote better separation. We assess the performance of SCCL on short text clustering and show that SCCL significantly advances the state-of-the-art results on most benchmark datasets with 3%-11% improvement on Accuracy and 4%-15% improvement on Normalized Mutual Information. Furthermore, our quantitative analysis demonstrates the effectiveness of SCCL in leveraging the strengths of both bottom-up instance discrimination and top-down clustering to achieve better intra-cluster and inter-cluster distances when evaluated with the ground truth cluster labels leverages contrastive learning to promote better separation between clusters (refers to [Hadifar 2019](doc:2021/05/a_self_training_approach_for_sh)) [2103.12953] Supporting Clustering with Contrastive Learning Andrew Arnold Feng Nan Shangwen Li Supporting Clustering with Contrastive Learning Xiaokai Wei Ramesh Nallapati > Our aim is to classify short invoice descriptions, in such a way that each class reflects a different group of products or services > The inherent advantage of embeddings in dealing with out-of-vocabulary words presents, at the same time, the disadvantage of providing a text representation that does not focus on the importance of individual terms for the classification. > > a method that combines the advantages of word embeddings with conventional term extraction techniques > employs terms to create distinctive semantic concept clusters. These clusters are ranked using a semantic similarity function which in turn defines a semantic feature space that can be used for text classification 2021-05-26T14:20:11Z 2021-05-26 Term Based Semantic Clusters for Very Short Text Classification (2019) > unsupervised method for discovering inference rules from text, such as "X is author of Y ≈ X wrote Y", "X solved Y ≈ X found a solution to Y", and "X caused Y ≈ Y is triggered by X". > Our algorithm is based on an **extended version of Harris' Distributional Hypothesis**, which states that words that occurred in the same contexts tend to be similar. Instead of using this hypothesis on words, we apply it to paths in the dependency trees of a parsed corpus. [Cited by](doc:2021/05/1906_03158_matching_the_blank) 2021-05-13 2021-05-13T00:56:25Z DIRT Discovery of inference rules from text (2001) Provable Limitations of Acquiring Meaning from Ungrounded Form: What will Future Language Models Understand? 2021-04-22T01:00:17Z Yoav Goldberg 2021-05-23T01:20:07Z [2104.10809] Provable Limitations of Acquiring Meaning from Ungrounded Form: What will Future Language Models Understand? William Merrill William Merrill 2021-04-22T01:00:17Z 2104.10809 Language models trained on billions of tokens have recently led to unprecedented results on many NLP tasks. This success raises the question of whether, in principle, a system can ever "understand" raw text without access to some form of grounding. We formally investigate the abilities of ungrounded systems to acquire meaning. Our analysis focuses on the role of "assertions": contexts within raw text that provide indirect clues about underlying semantics. We study whether assertions enable a system to emulate representations preserving semantic relations like equivalence. We find that assertions enable semantic emulation if all expressions in the language are referentially transparent. However, if the language uses non-transparent patterns like variable binding, we show that emulation can become an uncomputable problem. Finally, we discuss differences between our formal model and natural language, exploring how our results generalize to a modal setting and other semantic relations. Together, our results suggest that assertions in code or language do not provide sufficient signal to fully emulate semantic representations. We formalize ways in which ungrounded language models appear to be fundamentally limited in their ability to "understand". 2021-05-23 Roy Schwartz Noah A. Smith How can synaptic plasticity lead to meaningful learning? 2021-05-14T10:08:29Z 2021-05-14 2021-05-27 2021-05-27T15:30:59Z Carrot2 search results clustering engine (online) > simultaneously cluster and summarize documents by making use of both the document-term and sentence-term matrices 2021-05-25T18:12:00Z 2021-05-25 Integrating Document Clustering and Multidocument Summarization Adventures in Zero-Shot Text Classification 2021-05-25 2021-05-25T16:02:20Z Eugene Ie Alessandro Presta > We show that it is feasible to perform **entity linking by training a dual encoder (two-tower) model that encodes mentions and entities in the same dense vector space**, where candidate entities are retrieved by approximate nearest neighbor search. Unlike prior work, **this setup does not rely on an alias table followed by a re-ranker, and is thus the first fully learned entity retrieval model**. Contributions: > - a dual encoder architecture for learning entity and mention encodings suitable for retrieval. A key feature of the architecture is that it employs a modular **hierarchy of sub-encoders that capture different aspects of mentions and entities** > - a simple, fully unsupervised **hard negative mining** strategy that produces massive gains in retrieval performance, compared to using only random negatives > - high quality candidate entities very efficiently using approximate nearest neighbor search > - outperforms discrete retrieval baselines like an alias table or BM25 > strong retrieval performance across all 5.7 million Wikipedia entities in around 3ms per mention > since we are using a two-tower or dual encoder architecture, **our model cannot use any kind of attention over both mentions and entities at once**, nor feature-wise comparisons as done by Francis-Landau et al. (2016). This is a fairly severe constraint – for example, **we cannot directly compare the mention span to the entity title** – but it permits retrieval with nearest neighbor search for the entire context against a single, all encompassing representation for each entity We show that it is feasible to perform entity linking by training a dual encoder (two-tower) model that encodes mentions and entities in the same dense vector space, where candidate entities are retrieved by approximate nearest neighbor search. Unlike prior work, this setup does not rely on an alias table followed by a re-ranker, and is thus the first fully learned entity retrieval model. We show that our dual encoder, trained using only anchor-text links in Wikipedia, outperforms discrete alias table and BM25 baselines, and is competitive with the best comparable results on the standard TACKBP-2010 dataset. In addition, it can retrieve candidates extremely fast, and generalizes well to a new dataset derived from Wikinews. On the modeling side, we demonstrate the dramatic value of an unsupervised negative mining algorithm for this task. 1909.10506 Sayali Kulkarni Daniel Gillick 2019-09-23T17:52:34Z Larry Lansing Diego Garcia-Olano [1909.10506] Learning Dense Representations for Entity Retrieval Learning Dense Representations for Entity Retrieval 2019-09-23T17:52:34Z 2021-05-01T09:11:15Z Jason Baldridge 2021-05-01 Daniel Gillick A review of conceptual clustering algorithms 2021-05-26 2021-05-26T00:55:55Z 2021-05-31T22:10:51Z 2021-05-31 'Apple is eating our lunch': Google employees admit in lawsuit that the company made it nearly impossible for users to keep their location private 2019-11-21T11:37:18Z Yongdong Zhang Models semantic hierarchies by mapping entities into the polar coordinate system > Specifically, the radial coordinate aims to model entities at different levels of the hierarchy... the angular coordinate aims to distinguish entities at the same level of the hierarchy, and these entities are expected to have roughly the same radii but different angles. 2021-05-17T15:11:47Z Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction 2019-12-25T12:31:40Z 2021-05-17 [1911.09419] Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction 1911.09419 Knowledge graph embedding, which aims to represent entities and relations as low dimensional vectors (or matrices, tensors, etc.), has been shown to be a powerful technique for predicting missing links in knowledge graphs. Existing knowledge graph embedding models mainly focus on modeling relation patterns such as symmetry/antisymmetry, inversion, and composition. However, many existing approaches fail to model semantic hierarchies, which are common in real-world applications. To address this challenge, we propose a novel knowledge graph embedding model---namely, Hierarchy-Aware Knowledge Graph Embedding (HAKE)---which maps entities into the polar coordinate system. HAKE is inspired by the fact that concentric circles in the polar coordinate system can naturally reflect the hierarchy. Specifically, the radial coordinate aims to model entities at different levels of the hierarchy, and entities with smaller radii are expected to be at higher levels; the angular coordinate aims to distinguish entities at the same level of the hierarchy, and these entities are expected to have roughly the same radii but different angles. Experiments demonstrate that HAKE can effectively model the semantic hierarchies in knowledge graphs, and significantly outperforms existing state-of-the-art methods on benchmark datasets for the link prediction task. Jie Wang Zhanqiu Zhang Zhanqiu Zhang Jianyu Cai Hyènes (film) 2021-05-29 2021-05-29T23:36:24Z Adaptation de "Der Besuch der alten Dame" de Dürrenmatt Sinequa : Enterprise Search Platform 2021-05-19 2021-05-19T01:58:27Z > Results indicate that keyphrase extraction is still an open research question, with state-of-the-art neural-based models still challenged by simple baselines on some datasets [Github](https://github.com/ygorg/JCDL_2020_KPE_Eval) 2021-05-31 2021-05-31T11:56:12Z Large-Scale Evaluation of Keyphrase Extraction Models (2020) 2021-05-04T23:23:44Z 2021-05-04 Inria Paris NLP (ALMAnaCH team) sur Twitter : "#PAGnol, a new, free, GPT-3-like generative LM for French « Le “biodiversité-scepticisme”, plus discret que celui contre le dérèglement climatique, est en un sens bien plus inquiétant » 2021-05-24T14:46:11Z 2021-05-24 The City-State in Five Cultures | Department of History | University of Washington 2021-05-29 2021-05-29T00:50:24Z 2021-05-19T14:12:58Z 2021-05-19 Enterprise Knowledge Graph Solutions | eccenca Yann LeCun sur Twitter : "Barlow Twins: a new super-simple self-supervised method to train joint-embedding architectures (aka Siamese nets) non contrastively. " 2021-05-09T23:49:08Z 2021-05-09 fastai v2 cheat sheets 2021-05-10 2021-05-10T08:19:14Z the issue of clustering small sets of very short texts. Eg. in organizing brain-storming seminars > In order to cope with polysemy we adapt the SenseSearcher algorithm (SnS), by Kozlowski and Rybinski. In addition, we test the possibilities of improving the quality of clustering ultra-short texts by means of enriching them semantically. We present two approaches, one based on neural-based distributional models, and the other based on external knowledge resources. > It was shown that **only text-oriented clustering methods (STC, [Lingo](tag:lingo) and SnSRC) give reasonable results for French ultra short texts**, whereas the clustering quality of Bisecting k-means in these experiments is very low > The experiments with the neural network based models (implemented by means of Word2vec) showed much better results than other semantic enrichment methods for both algorithms and for both data sets (Good related work section) Clustering of semantically enriched short texts (2018) 2021-05-26 2021-05-26T17:22:53Z Norbert Zeh Md Rashadul Hasan Rakib 2020-01-31T02:12:05Z Enhancement of Short Text Clustering by Iterative Classification Magdalena Jankowska > Given a clustering of short texts obtained using an arbitrary clustering algorithm, iterative classification applies outlier removal to obtain outlier-free clusters. Then it trains a classification algorithm using the non-outliers based on their cluster distributions. Using the trained classification model, iterative classification reclassifies the outliers to obtain a new set of clusters. Short text clustering is a challenging task due to the lack of signal contained in such short texts. In this work, we propose iterative classification as a method to b o ost the clustering quality (e.g., accuracy) of short texts. Given a clustering of short texts obtained using an arbitrary clustering algorithm, iterative classification applies outlier removal to obtain outlier-free clusters. Then it trains a classification algorithm using the non-outliers based on their cluster distributions. Using the trained classification model, iterative classification reclassifies the outliers to obtain a new set of clusters. By repeating this several times, we obtain a much improved clustering of texts. Our experimental results show that the proposed clustering enhancement method not only improves the clustering quality of different clustering methods (e.g., k-means, k-means--, and hierarchical clustering) but also outperforms the state-of-the-art short text clustering methods on several short text datasets by a statistically significant margin. 2021-05-20 Md Rashadul Hasan Rakib 2001.11631 [2001.11631] Enhancement of Short Text Clustering by Iterative Classification 2021-05-20T17:59:46Z Evangelos Milios 2020-01-31T02:12:05Z 2021-05-28T14:23:24Z 2021-05-28 La cité oubliée d’Ulug Dépé | CNRS Le journal 2021-05-10T23:30:43Z Alex Russell sur Twitter : "If you install Firefox on Windows, MacOS, Linux, ChromeOS, or Android you get *real* Firefox, complete with the Gecko engine. But not on iOS. Apple cripples engine competition in silent, deeply impactful ways." / Twitter 2021-05-10