## Phrase-Based & Neural Unsupervised Machine Translation

- 1. carefully initialize the MT system with an inferred bilingual dictionary. 通过双语字典对MT模型进行初始化。
- 2. leverage strong language models, via training the sequence-to-sequence system as a denoising autoencoder. 通过训练seq2eq模型来利用强大的语言模型作为降噪自编码。
- 3. turn the unsupervised problem into a supervised one by automatic generation of sentence pairs via back-translation.把无监督问题转换为有监督的问题，也就是通过back-translation自动生成语言对。

- B. 初始化 - C. 语言模型
- D. 迭代反向翻译。

### Initialization

- bilingual dictionary
- dictionaries inferred in an unsupervised way. Lample et al. (2018) and Artetxe et al. (2018)

- join the monolingual corpora
- apply BPE tokenization on the resulting corpus
- learn token embeddings (Mikolov et al., 2013) on the same corpus

### Language Modeling

C is a noise model with some words dropped and swapped. $P_{s→s}$ and $P_{t→t}$ are the composition of encoder and decoder both operating on the source and target sides, respectively. Back-translation:

### Iterative Back-translation

Dual Learning for Machine Translation $fr \rightarrow \hat{en} \rightarrow fr$ fr 是 target language. en 是 source language.

$u^{* }(y)=argmaxP_{t→s}(u|y)$, $u^{* }(y)$ 是 pesudo source sentence.

$v^{* }(x)=argmaxP_{s→t}(v|x)$, $v^{* }(x)$ 是 pesudo target sentence.

While sharing the encoder is critical to get the model to work, shar- ing the decoder simply induces useful regularization.