IPM Data Science Meetup #7
▪️Neural Machine Translation
▫️By: Hamid Reza Ghader, University of Amsterdam
🕙 Thursday 10 Esfand 1396, 13:00 AM
What does Attention in Neural Machine Translation Pay Attention to?
Neural machine translation models have taken over as the state-of-the-art from the long-standing phrase-based machine translation models in the recent years. This is because Neural models have shown better performance in translating multiple language pairs and a great potential to improve over others. Among the different neural machine translation models, attentional models are the most popular due to their capability to efficiently encode long sentences. Attention in neural machine translation provides the possibility to encode relevant parts of the source sentence at each translation step. As a result, attention is considered to be an alignment model as well. However, there is no work that specifically studies attention and provides analysis of what is being learned by attention models. Thus, the question still remains that how attention is similar or different from the traditional alignment. In this talk, we provide detailed analysis of attention and compare it to traditional alignment. We answer the question of whether attention is only capable of modelling translational equivalent or it captures more information. We show that attention is different from alignment in some cases and is capturing useful information other than alignments.
Hamid is a PhD candidate at the University of Amsterdam in the Information and Language Processing Systems (ILPS) group. He is working on Statistical Machine Translation with Dr. Christof Monz and Prof. Maarten de Rijke. Before starting his PhD, he was working as a research assistant in Natural Language and Text Processing Laboratory at University of Tehran. There, he was part of Faraazin machine translation development team.
Hamid has done his masters in AI at Iran University of Science and Technology working on grammar induction and his bachelor in software engineering at University of Tehran.