litlas
ログインアカウント作成

Linguistic Regularities in Continuous Space Word Representations

著者: Tomás Mikolov, Wen-tau Yih, Geoffrey Zweig - Conference of the North American Chapter of the Association for Computational Linguistics, NAACL 2013 被引用: 3,965

Continuous space language models have recently demonstrated outstanding results across a variety of tasks. In this paper, we examine the vector-space word representations that are implicitly learned by the input-layer weights. We find that these representations are surprisingly good at capturing syntactic and semantic regularities in language, and that each relationship is characterized by a relation-specific vector offset. This allows vector-oriented reasoning based on the offsets between words. For example, the male/female relationship is automatically learned, and with the induced vector representations, “King-Man + Woman ” results in a vector very close to “Queen. ” We demonstrate that the word vectors capture syntactic regularities by means of syntactic analogy questions (provided with this paper), and are able to correctly answer almost 40 % of the questions. We demonstrate that the word vectors capture semantic regularities by using the vector offset method to answer SemEval-2012 Task 2 questions. Remarkably, this method outperforms the best previous systems. 1

Indexing by LatentSemantic AnalysisIndexing by Latent Semantic Analysis10.1162/15324430332253322310.1162/153244303322533223HierarchicalProbabilistic Neural…Hierarchical Probabilistic Neural Network Language ModelNeural ProbabilisticLanguage ModelsNeural Probabilistic Language ModelsReducing theDimensionality of Data…Reducing the Dimensionality of Data with Neural NetworksContinuous spacelanguage modelsContinuous space language modelsA Scalable HierarchicalDistributed Language…A Scalable Hierarchical Distributed Language ModelA unified architecturefor natural language…A unified architecture for natural language processing: deep neural networks with multitask learningWord Representations: ASimple and General…Word Representations: A Simple and General Method for Semi-Supervised LearningRecurrent neural networkbased language modelRecurrent neural network based language modelStrategies for traininglarge scale neural…Strategies for training large scale neural network language modelsStructured Output Layerneural network language…Structured Output Layer neural network language modelDependency-Based WordEmbeddingsDependency-Based Word EmbeddingsLearning SemanticHierarchies via Word…Learning Semantic Hierarchies via Word EmbeddingsLearning ContinuousPhrase Representations…Learning Continuous Phrase Representations for Translation ModelingNormalized WordEmbedding and Orthogona…Normalized Word Embedding and Orthogonal Transform for Bilingual Word TranslationLearning LexicalEmbeddings with…Learning Lexical Embeddings with Syntactic and Lexicographic KnowledgeMulti-GranularityChinese Word EmbeddingMulti-Granularity Chinese Word EmbeddingBlack Holes and WhiteRabbits: Metaphor…Black Holes and White Rabbits: Metaphor Identification with Visual FeaturesBusiness PerceptionBased on Sentiment…Business Perception Based on Sentiment Analysis Through Deep Neuronal Networks for Natural Language ProcessingDynamic Word Embeddingsvia Skip-Gram FilteringDynamic Word Embeddings via Skip-Gram FilteringPredictive UncertaintyEstimation via Prior…Predictive Uncertainty Estimation via Prior NetworksAnalogical Reasoning onChinese Morphological…Analogical Reasoning on Chinese Morphological and Semantic RelationsTopic recommendationusing Doc2VecTopic recommendation using Doc2VecLinguistic Regularitiesin Continuous Space Wor…Linguistic Regularities in Continuous Space Word Representations過去の参考文献中心の論文この論文を引用する論文古い新しい

ノードをクリックするとフォーカスを固定、空白をクリックすると本論文に戻ります。ホバーで一時的にプレビューできます。各ノードのページはタイトルから開けます。