English

Overview

Google’s Neural Machine Translation (GNMT) system demonstrated how to deploy neural machine translation at scale. Using an 8-layer LSTM with residual connections and attention, GNMT significantly reduced translation errors compared to phrase-based systems.

Key Contributions

  1. Deep Architecture: 8-layer encoder and decoder with residual connections
  2. Attention Mechanism: Improved alignment between source and target
  3. Wordpiece Model: Subword tokenization for handling rare words
  4. Quantization: 8-bit inference for production deployment

Results

  • 60% reduction in translation errors on many language pairs
  • Bridged the gap between human and machine translation
  • Deployed to Google Translate serving billions of translations

Impact

GNMT showed that neural approaches could replace traditional statistical MT：

  • Sparked industry-wide adoption of neural MT
  • Demonstrated importance of engineering for production ML
  • Wordpiece tokenization became standard (used in BERT, GPT)

arXiv

.08144

中文

概述

谷歌神经机器翻译（GNMT）系统展示了如何大规模部署神经机器翻译。使用带有残差连接和注意力的8层LSTM，GNMT与基于短语的系统相比显著减少了翻译错误。

主要贡献

  1. 深层架构：带残差连接的8层编码器和解码器
  2. 注意力机制：改进源语言和目标语言之间的对齐
  3. Wordpiece模型：子词分词处理罕见词
  4. 量化：8位推理用于生产部署

结果

  • 许多语言对的翻译错误减少60%
  • 缩小了人工翻译和机器翻译之间的差距
  • 部署到谷歌翻译，服务数十亿次翻译

影响

GNMT表明神经方法可以取代传统统计机器翻译：

  • 引发全行业采用神经机器翻译
  • 展示了生产级机器学习工程的重要性
  • Wordpiece分词成为标准（用于BERT、GPT）

论文链接

arXiv

.08144