]> WORKSHOP: BigNet @ WWW 2018 Workshop on Learning Representations for Big Networks 2018-01-27T15:13:16Z 2018-01-27 'The Character of Physical Law': Richard Feynman's Legendary Course Presented at Cornell, 1964 | Open Culture 2018-01-08T23:16:54Z 2018-01-08 Awesome Knowledge Graph Embedding Approaches 2018-01-03T16:41:48Z lists libraries and approaches for knowledge graph embeddings 2018-01-03 > In our approach, we adapt neural language models for RDF graph embeddings. Such approaches take advantage of the word order in text documents, explicitly modeling the assumption that closer words in the word sequence are statistically more dependent. In the case of RDF graphs, we consider entities and relations between entities instead of word sequences. Thus, in order to apply such approaches on RDF graph data, we first have to transform the graph data into sequences of entities, which can be considered as sentences. Using those sentences, we can train the same neural language models to represent each entity in the RDF graph as a vector of numerical values in a latent feature space. 2018-01-03 2018-01-03T16:54:19Z RDF2Vec: RDF Graph Embeddings for Data Mining - (2016) This work focuses on modeling multi-relational data from KBs (Wordnet and Freebase in this paper), with the goal of providing an efficient tool to complete them by automatically adding new facts, without requiring extra knowledge. **Embedding entities and relationships of multirelational data**: a method which **models relationships by interpreting them as translations** operating on the low-dimensional embeddings of the entities. Motivation: - hierarchical relationships are extremely common in KBs and translations are the natural transformations for representing them. - cf. word embeddings and the “capital of” relationship between countries and cities, which are (coincidentally rather than willingly) represented by the model as translations in the embedding space. This suggests that there may exist embedding spaces in which 1-to-1 relationships between entities of different types may, as well, be represented by translations. The intention of our model is to enforce such a structure of the embedding space. [Good blog post by PY Vandenbussche](http://pyvandenbussche.info/2017/translating-embeddings-transe/) 2018-01-05 Translating Embeddings for Modeling Multi-relational Data (2013) 2018-01-05T14:46:46Z 2018-01-03 RDF2Vec: RDF Graph Embeddings for Data Mining 2018-01-03T16:50:08Z 2018-01-03 2018-01-03T17:15:17Z Combining semantics and deep learning for intelligent information services 2018-01-27T15:18:02Z 2018-01-27 TUTORIAL: Representation Learning on Networks - TheWebConf 2018 2018-01-22 Capitalism’s new crisis: after Carillion, can the private sector ever be trusted? | Politics | The Guardian 2018-01-22T18:22:52Z > People learning new concepts can often generalize successfully from just a single example, yet machine learning algorithms typically require tens or hundreds of examples to perform with similar accuracy... We present a computational model that captures these human learning abilities for a large class of simple visual concepts: handwritten characters from the world’s alphabets Human-level concept learning through probabilistic program induction (2015) 2018-01-04T14:56:17Z 2018-01-04 2018-01-09T13:43:25Z 2018-01-09 MIT 6.S094: Deep Learning for Self-Driving Cars 2018-01-27 2018-01-27T15:36:02Z [CFP](https://www2018.thewebconf.org/call-for-papers/research-tracks-cfp/web-content-analysis/) > In previous years, ‘content analysis’ and ‘semantic and knowledge’ were in separate track. This year, we combined these tracks to emphasize the close relationship between these topics; **the use of content to curate knowledge and the use of knowledge to guide content analysis and intelligent usage**. Some of the accepted papers: ### [Large-Scale Hierarchical Text Classification with Recursively Regularized Deep Graph-CNN](https://doi.org/10.1145/3178876.3186005) [Hierarchical Text Classification](/tag/nlp_hierarchical_text_classification): Text classification to a hierarchical taxonomy of topics, using graph representation of text, and CNN over this graph Renvoie à ce qui a été vu dans le tutorial "Graph-based Text Representations" from the abstract: > a graph-CNN based deep learning model to first convert texts to graph-of-words, and then use graph convolution operations to convolve the word graph. Graph-of-words representation of texts has the advantage of capturing non-consecutive and long-distance semantics. CNN models have the advantage of learning different level of semantics. To further leverage the hierarchy of labels, we regularize the deep architecture with the dependency among labels Conversion of text to graph: potentially given a single document ### [Weakly-supervised Relation Extraction by Pattern-enhanced Embedding Learning](https://doi.org/10.1145/3178876.3186024 ) Extraction de relations de corpus de textes de façon semi-supervisée, dans un contexte où on a peu de données labellisées décrivant les relations. Par exemple, des données labellisées indique que le texte "Beijing, capital of China" correspond à la relation entre entités : ("Beijing", "Capital Of", "China), et on voudrait pouvoir extraire les entités et relations pertinentes à partir de texte tel que "Paris, France's capital,..." Le papier décrit une méthode qui combine deux modules, l'un basé sur l'extraction automatique de patterns (par ex "[Head], Capital Of [Tail]") et l'autre sur la "sémantique distributionnelle" (du type "word embeddings"). Ces deux modules collaborent, le premier permettant de créer des instances de relations augmentant la base de connaissance sur lequel entrainer le second, et le second aidant le premier à déterminer des patterns informatifs ("co-entrainement") ### [Scalable Instance Reconstruction in Knowledge Bases via Relatedness Affiliated Embedding](https://doi.org/10.1145/3178876.3186017) Knowledge base completion problem: usually, it is formulated as a link prediction problem, but not here. A novel knowledge embedding model ("Joint Modelling and Learning of Relatedness and Embedding") ### [Improving Word Embedding Compositionality using Lexicographic Definitions](https://doi.org/10.1145/3178876.3186007) comment obtenir les meilleures représentations de texte à partir de représentations de mots (word embeddings) ? L'auteur utilise des ressources lexicographiques (wordnet) pour ses tests : l'embedding obtenu pour la définition d'un mot est-il proche de celui du mot ? Le papier s'appuie sur une [thèse du même auteur](https://esc.fnwi.uva.nl/thesis/centraal/files/f1554608041.pdf), claire et bien écrite. ### [CESI: Canonicalizing Open Knowledge Bases using Embeddings and Side Information](https://doi.org/10.1145/3178876.3186030) Amélioration de l'extraction de triplets (nom phrase, property, nom phrase) à partir de texte en calculant des embeddings pour les "nom phrases" (~entités) ### [Short-Text Topic Modeling via Non-negative Matrix Factorization Enriched with Local Word-Context Correlations](https://doi.org/10.1145/3178876.3186009) Topic modeling for short texts, leveraging the word-context semantic correlations in the training ### [Towards Annotating Relational Data on the Web with Language Models](https://doi.org/10.1145/3178876.3186029) ### A paper by [David Blei](/tag/david_blei): (Dynamic Embeddings for Language Evolution) RESEARCH TRACK: Web Content Analysis, Semantics and Knowledge 2018-01-28 2018-01-28T17:19:03Z Evaluating the Impact of Word Embeddings on Similarity Scoring in Practical Information Retrieval (2017) > Transferring the success of word embeddings to Information Retrieval (IR) task is currently an active research topic. While embedding-based retrieval models could tackle the vocabulary mismatch problem by making use of the embedding’s inherent similarity between distinct words, most of them struggle to compete with the prevalent strong baselines such as TF-IDF and BM25. Considering a practical ad-hoc IR task composed of two steps, matching and scoring, compares the performance of several techniques that leverage word embeddings in the retrieval models to compute the similarity between the query and the documents (namely word centroid similarity, paragraph vectors, Word Mover’s distance, as well as a novel inverse document frequency (IDF) re-weighted word centroid similarity). > We confirm that word embeddings can be successfully employed in a practical information retrieval setting. The proposed cosine similarity of IDF re-weighted, aggregated word vectors is competitive to the TF-IDF baseline. Evolution Strategies as a Scalable Alternative to Reinforcement Learning 2018-01-06 2018-01-06T15:11:28Z 2018-01-05 2018-01-05T14:37:38Z François-Paul Servant - Citations Google Scholar > We won’t protect jobs. But we will protect workers 2018-01-04T01:35:20Z While U.S. Workers Fear Automation, Swedish Employees Welcome It - MIT Technology Review 2018-01-04 2018-01-03 In this paper, we study how to optimize the document representation by leveraging neural-based approaches to capture latent representations built upon both validated medical concepts specified in an external resource as well as the used words. **Document vectors are learned so they allow predicting concepts in their context** 2018-01-03T15:44:56Z Learning Concept-Driven Document Embeddings for Medical Information Search (2017) DNA seen through the eyes of a coder 2018-01-26T15:01:21Z 2018-01-26 Big data meets Big Brother as China moves to rate its citizens | WIRED UK 2018-01-03 2018-01-03T17:42:44Z 2018-01-23 2018-01-23T01:50:22Z Engineers design artificial synapse for “brain-on-a-chip” hardware | Robohub Oh, shit, git! 2018-01-06T14:46:32Z 2018-01-06 [1103.0398] Natural Language Processing (almost) from Scratch Jason Weston Michael Karlen Ronan Collobert Natural Language Processing (almost) from Scratch 2018-01-17T18:40:10Z 2018-01-17 Koray Kavukcuoglu 2011-03-02T11:34:50Z Ronan Collobert Pavel Kuksa We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including: part-of-speech tagging, chunking, named entity recognition, and semantic role labeling. This versatility is achieved by trying to avoid task-specific engineering and therefore disregarding a lot of prior knowledge. Instead of exploiting man-made input features carefully optimized for each task, our system learns internal representations on the basis of vast amounts of mostly unlabeled training data. This work is then used as a basis for building a freely available tagging system with good performance and minimal computational requirements. 2011-03-02T11:34:50Z Leon Bottou seminal work Abstract: > a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including: part-of-speech tagging, chunking, named entity recognition, and semantic role labeling. This versatility is achieved by trying to avoid task-specific engineering and therefore disregarding a lot of prior knowledge. Instead of exploiting man-made input features carefully optimized for each task, our system learns internal representations on the basis of vast amounts of mostly unlabeled training data. This work is then used as a basis for building a freely available tagging system with good performance and minimal computational requirements 1103.0398 Current work in lexical distributed representations maps each word to a point vector in low-dimensional space. Mapping instead to a density provides many interesting advantages, including better capturing uncertainty about a representation and its relationships, expressing asymmetries more naturally than dot product or cosine similarity, and enabling more expressive parameterization of decision boundaries. This paper advocates for density-based distributed embeddings and presents a method for learning representations in the space of Gaussian distributions. We compare performance on various word embedding benchmarks, investigate the ability of these embeddings to model entailment and other asymmetric relationships, and explore novel properties of the representation. [1412.6623] Word Representations via Gaussian Embedding 1412.6623 2015-05-01T10:14:58Z 2018-01-28 Luke Vilnis 2014-12-20T07:42:40Z > Current work in lexical distributed representations maps each word to a point vector in low-dimensional space. Mapping instead to a density provides many interesting advantages > Novel word embedding algorithms that embed words directly as Gaussian distributional potential functions in an infinite dimensional function space. This allows us to map word types not only to vectors but to soft regions in space, modeling uncertainty, inclusion, and entailment, as well as providing a rich geometry of the latent space. Word Representations via Gaussian Embedding 2018-01-28T17:27:24Z Andrew McCallum Luke Vilnis Gradient descent vs. neuroevolution 2018-01-06 2018-01-06T15:24:23Z 2018-01-03 2018-01-03T17:06:57Z Global RDF Vector Space Embeddings 2018-05-23T09:23:47Z Universal Language Model Fine-tuning for Text Classification 2018-01-19 [1801.06146] Universal Language Model Fine-tuning for Text Classification Jeremy Howard Sebastian Ruder Jeremy Howard 2018-01-19T11:31:32Z 1801.06146 Inductive transfer learning has greatly impacted computer vision, but existing approaches in NLP still require task-specific modifications and training from scratch. We propose Universal Language Model Fine-tuning (ULMFiT), an effective transfer learning method that can be applied to any task in NLP, and introduce techniques that are key for fine-tuning a language model. Our method significantly outperforms the state-of-the-art on six text classification tasks, reducing the error by 18-24% on the majority of datasets. Furthermore, with only 100 labeled examples, it matches the performance of training from scratch on 100x more data. We open-source our pretrained models and code. code is available in the fastai lib [blog post](http://nlp.fast.ai/classification/2018/05/15/introducting-ulmfit.html) [see also](/doc/?uri=https%3A%2F%2Fyashuseth.blog%2F2018%2F06%2F17%2Funderstanding-universal-language-model-fine-tuning-ulmfit%2F) 2018-01-18T17:54:52Z Differentiable Programming 2018-01-14T19:20:20Z 2018-01-14 2018-01-14 2018-01-14T19:19:51Z We May All Die Horribly 2018-01-28T17:02:23Z 2018-01-28 Imagine a smartphone that’s able to negotiate the best price on a new car for you AI can win at poker: but as computers get smarter, who keeps tabs on their ethics? | Technology | The Guardian La classe africaine 2018-01-21T19:41:14Z 2018-01-21 An Adversarial Review of “Adversarial Generation of Natural Language” 2018-01-01T12:39:30Z 2018-01-01 Last November, researchers from the University of Southern California announced the successful results achieved with brain implants to improve memory 2018-01-03 2018-01-03T00:55:29Z Brain Implants and the BRAIN Initiative: lights and Shadows - OpenMind > 'Rights are for the powerless. They’re for the minority. They’re for the different. They’re for the weak’ 2018-01-05 Q&A: Edward Snowden on rights, privacy, secrets and leaks in conversation with Jimmy Wales – Wikitribune 2018-01-05T21:07:14Z 2018-01-30 Knowledge Graph Embedding by Translating on Hyperplanes (2014) 2018-01-30T13:35:21Z > we start by analyzing the problems of TransE on reflexive/one-to-many/many-to-one/many-to-many relations. Accordingly we propose a method named translation on hyperplanes (TransH) which interprets a relation as a translating operation on a hyperplane 2018-01-23 2018-01-23T14:42:50Z LEARNING GRAPH EMBEDDINGS FOR NODE LABELING AND INFORMATION DIFFUSION IN SOCIAL NETWORKS (2017) 2018-01-17 2018-01-17T21:02:46Z 57 Summaries of Machine Learning and NLP Research - Marek Rei 2018-01-22 2018-01-22T11:16:36Z Stanford Seminar - "Can the brain do back-propagation?" - Geoffrey Hinton 2018-01-14 2018-01-14T19:18:13Z Daniel Ellsberg on ‘The Doomsday Machine’ Swagger imposes some constraints, like the lack of hypermedia... if you are using swagger, you are probably giving up one of the most powerful feature of RESTful APIs. You are giving up evolvability! The problems with Swagger - NovaTec Blog 2018-01-05 2018-01-05T21:22:22Z How Dirt Could Save Us From Antibiotic-Resistant Superbugs | WIRED 2018-01-15T10:26:08Z 2018-01-15 I am a time-traveler from the future, here to beg you to stop what you are doing. : Bitcoin 2018-01-21T23:37:53Z 2018-01-21 Knowledge Graph and Text Jointly Embedding (2014) 2018-01-05 method of **jointly embedding knowledge graphs and a text corpus** so that **entities and words/phrases are represented in the same vector space**. Promising improvement in the accuracy of predicting facts, compared to separately embedding knowledge graphs and text (in particular, enables the prediction of facts containing entities out of the knowledge graph) [cité par J. Moreno](/doc/?uri=https%3A%2F%2Fhal.archives-ouvertes.fr%2Fhal-01626196%2Fdocument) 2018-01-05T15:41:19Z 2018-01-27 2018-01-27T13:21:31Z Embedding entities and relations in the knowledge base to low dimensional vector representations and then predict the possible truth of additional facts to extend the knowledge base Knowledge base completion by learning pairwise-interaction differentiated embeddings | SpringerLink (2015) Letters from Iwo Jima 2018-01-15T23:03:20Z 2018-01-15 film de Clint Eastwood 2018-01-03 2018-01-03T11:33:53Z Although deep learning has historical roots going back decades, neither the term "deep learning" nor the approach was popular just over five years ago, when the field was reignited by papers such as Krizhevsky, Sutskever and Hinton's now classic (2012) deep network model of Imagenet. What has the field discovered in the five subsequent years? Against a background of considerable progress in areas such as speech recognition, image recognition, and game playing, and considerable enthusiasm in the popular press, I present ten concerns for deep learning, and suggest that deep learning must be supplemented by other techniques if we are to reach artificial general intelligence. 2018-01-02T12:49:35Z Gary Marcus Gary Marcus Deep Learning: A Critical Appraisal 1801.00631 2018-01-02T12:49:35Z [1801.00631] Deep Learning: A Critical Appraisal Arxiv Sanity Preserver 2018-01-22T18:07:05Z 2018-01-22 A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines David Charte 2018-01-09T14:05:31Z 1801.01586 María J. del Jesus Many of the existing machine learning algorithms, both supervised and unsupervised, depend on the quality of the input characteristics to generate a good model. The amount of these variables is also important, since performance tends to decline as the input dimensionality increases, hence the interest in using feature fusion techniques, able to produce feature sets that are more compact and higher level. A plethora of procedures to fuse original variables for producing new ones has been developed in the past decades. The most basic ones use linear combinations of the original variables, such as PCA (Principal Component Analysis) and LDA (Linear Discriminant Analysis), while others find manifold embeddings of lower dimensionality based on non-linear combinations, such as Isomap or LLE (Linear Locally Embedding) techniques. More recently, autoencoders (AEs) have emerged as an alternative to manifold learning for conducting nonlinear feature fusion. Dozens of AE models have been proposed lately, each with its own specific traits. Although many of them can be used to generate reduced feature sets through the fusion of the original ones, there also AEs designed with other applications in mind. The goal of this paper is to provide the reader with a broad view of what an AE is, how they are used for feature fusion, a taxonomy gathering a broad range of models, and how they relate to other classical techniques. In addition, a set of didactic guidelines on how to choose the proper AE for a given task is supplied, together with a discussion of the software tools available. Finally, two case studies illustrate the usage of AEs with datasets of handwritten digits and breast cancer. Salvador García 2018-01-09 [1801.01586] A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines 2018-01-04T23:51:05Z Francisco Herrera Francisco Charte 2018-01-04T23:51:05Z David Charte 2018-01-01 2018-01-01T12:41:36Z AI and Deep Learning in 2017 – A Year in Review – WildML 2018-01-23T18:20:31Z 2018-01-23 Vocabulaires dans le web de données : quels outils open-source ? - Sparna Blog 2018-01-02T19:19:30Z To Sate China’s Demand, African Donkeys Are Stolen and Skinned - The New York Times 2018-01-02 2018-01-23T13:44:20Z L’histoire du fémur de Toumaï | Dans les pas des archéologues 2018-01-23