English
Overview
Word2Vec introduced efficient methods for learning distributed representations of words from large text corpora. The resulting word vectors capture semantic and syntactic relationships, enabling famous analogies like “king - man + woman = queen”.
Key Contributions
- Skip-gram Model: Predicts context words given a target word
- CBOW (Continuous Bag of Words): Predicts target word from context
- Efficiency: Can train on billions of words in hours
- Semantic Relationships: Vectors capture meaningful word relationships
Architecture
Two main architectures：
- Skip-gram: Given word w, predict surrounding words
- CBOW: Given surrounding words, predict center word w
Both use shallow neural networks with a single hidden layer.
Impact
Word2Vec revolutionized NLP：
- Foundation for all modern word embeddings
- Inspired GloVe, FastText, and eventually contextual embeddings
- Demonstrated that simple models at scale can capture complex semantics
Paper Link
.3781中文
概述
Word2Vec引入了从大规模文本语料库学习词的分布式表示的高效方法。生成的词向量捕捉语义和句法关系，实现了著名的类比如”国王 - 男人 + 女人 = 女王”。
主要贡献
- Skip-gram模型：给定目标词预测上下文词
- CBOW（连续词袋）：从上下文预测目标词
- 高效性：可在数小时内训练数十亿词
- 语义关系：向量捕捉有意义的词关系
架构
两种主要架构：
- Skip-gram：给定词w，预测周围的词
- CBOW：给定周围的词，预测中心词w
两者都使用只有一个隐藏层的浅层神经网络。
影响
Word2Vec彻底改变了NLP：
- 所有现代词嵌入的基础
- 启发了GloVe、FastText，最终发展到上下文嵌入
- 证明了大规模简单模型可以捕捉复杂语义