Proceedings of the First Workshop on Neural Machine Translation

Authors: Raphael Shu, Hideki Nakayama, Jan Niehues, Eunah Cho, Thanh-Le Ha, Alex Waibel, Michael Denkowski, Graham Neubig, Chris Dyer, Yoav Goldberg, Austin Matthews, Waleed Ammar, Antonios Anastasopou- Los, Miguel Ballesteros, David Chiang, Daniel Clothiaux, Trevor Cohn, Kevin Duh, Manaal Faruqui, Cynthia Gan, Dan Garrette, Yangfeng Ji, Lingpeng Kong, Adhiguna Kuncoro, Gaurav Ku- Mar, Chaitanya Malaviya, Paul Michel, Yusuke Oda, Matthew Richardson, Philipp Koehn, Rebecca Knowles, Marine Carpuat, Yogarshi Vyas, Xing Niu, Markus Freitag, Hideki Nakayama, Yusuke Oda, S. Raphael, Koichiro Yoshino, Foster, George, Markus Freitag, Goto, Koehn, Larkin, Samuel, Morishita, Makoto, Nakamura, Satoshi, Nakayama Hideki, Neubig, Graham, Niu, Oda Yusuke, Shu Raphael, Sudoh, Katsuhito, Tanaka, Vyas, Yoshino Koichiro, Ondrej Bojar, Rajen Chatterjee, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, Cettolo, S Niehues, L Stker, M Bentivogli, Federico, Eunah Cho, Jan Niehues, Thanh-Le Le, Matthias Sperber, Mohammed Mediani, Alex Waibel, Kyunghyun Cho, Bart Van Merrinboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, Yoshua Bengio, D William, Lewis Christian Federmann, Jonathan Clark, Chris Dyer, Alon Lavie, Noah Smith, Philip Arthur, Graham Neubig, Satoshi Nakamura, Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio, Ondej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Aurelie Neveol, Mariana Neves, Martin Popel, Matt Post, Raphael Rubino, Carolina Scarton, Lucia Specia, Marco Turchi, Karin Verspoor, Marcos Zampieri, Mauro Cettolo, Christian Girardi, Marcello Federico, Wenhu Chen, Evgeny Matusov, Shahram Khadivi, Jan-Thorsten Peter, Jonathan Clark, Chris Dyer, Alon Lavie, Noah Smith, Chris Dyer, Victor Chahuneau, Noah Smith, Rajen Amin Farajian, Costanza Chatterjee, Shahab Conforti, Vevake Jalalvand, Mattia A Di Balaraman, Duygu Gangi, Marco Ataman, Matteo Turchi, Marcello Negri, Federico, Yarin Gal, Zoubin Ghahramani, Sbastien Jean, Kyunghyun Cho, Roland Memisevic, Yoshua Bengio, Yoon Kim, Alexander Rush, P Diederik, Jimmy Kingma, Ba, Adhiguna Kuncoro, Miguel Ballesteros, Lingpeng Kong, Chris Dyer, Noah Smith, Thang Luong, Ilya Sutskever, Quoc Le, Oriol Vinyals, Wojciech Zaremba ; Barry, Alexandra Haddow, Birch, Rico Sennrich, Barry Haddow, Alexandra Birch, Ilya Sutskever, Oriol Vinyals, Quoc V Le, Philip Arthur, Graham Neubig, Satoshi Nakamura, Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio, Luisa Bentivogli, Arianna Bisazza, Mauro Cettolo, Marcello Federico, Ondej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Aurelie Neveol, Mariana Neves, Martin Popel, Matt Post, Raphael Rubino, Carolina Scarton, Lucia Specia, Marco Turchi, Karin Verspoor, Marcos Zampieri, David Chiang, Kyunghyun Cho, Bart Van Merrienboer, Dzmitry Bahdanau, Yoshua Bengio, Shuoyang Ding, Kevin Duh, Huda Khayrallah, Philipp Koehn, Matt Post, Shuoyang Ding, Kevin Duh, Huda Khayrallah, Philipp Koehn, Matt Post, Chris Dyer, Victor Chahuneau, Noah Smith, Michel Galley, Jonathan Graehl, Kevin Knight, Daniel Marcu, Steve Deneefe, Wei Wang, Ignacio Thayer, Michel Galley, Mark Hopkins, Kevin Knight, Daniel Marcu, Ann Irvine, Chris Callison-Burch, Marcin Junczys-Dowmunt, Tomasz Dwojak, Hieu Hoang, Nal Kalchbrenner, Phil Blunsom, Philipp Koehn, Barry Haddow, Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Christopher Dyer, Ondej Bojar, Alexandra Constantin, Evan Herbst, Lemao Liu, Masao Utiyama, Andrew Finch, Eiichiro Sumita, Minh-Thang Luong, Christopher Manning, Thang Luong, Ilya Sutskever, Quoc Le, Oriol Vinyals, Wojciech Zaremba, Rico Sennrich, Orhan Firat, Kyunghyun Cho, Alexandra Birch, Barry Haddow, Julian Hitschler, Marcin Junczys-Dowmunt, Samuel Lubli, Antonio Valerio Miceli, Jozef Barone, Maria Mokry, Nadejde, Rico Sennrich, Barry Haddow, Alexandra Birch, Rico Sennrich, Barry Haddow, Alexandra Birch, Jrg Tiedemann, Antonio Toral, M Vctor, Snchez-Cartagena, Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, Hang Li, Marco Turchi, Tijl De Bie, Nello Cristianini, Philip Williams, Rico Sennrich, Maria Nadejde, Matthias Huck, Barry Haddow, Ondej Bojar, Amittai Axelrod, Xiaodong He, Jianfeng Gao, Nicola Bertoldi, Marcello Federico, Marine Carpuat, Hal Daume, Iii, Katharine Henry, Ann Irvine, Jagadeesh Jagarlamudi, Rachel Rudinger, Boxing Chen, Fei Huang, Boxing Chen, Roland Kuhn, George Foster, Kevin Duh, Graham Neubig, Katsuhito Sudoh, Hajime Tsukada, Vladimir Eidelman, Jordan Boyd-Graber, Philip Resnik, George Foster, Roland Kuhn, George Foster, Cyril Goutte, Roland Kuhn, Z Gong, M Zhang, G Zhou, Eva Hasler, Barry Haddow, Philipp Koehn, Sanjika Hewavitharana, Dennis Mehay, Sankaranarayanan Ananthakrishnan, Prem Natarajan, Geoffrey Hinton, N Srivastava, Kevin Swersky, Philipp Koehn, Minh-Thang Luong, Christopher Manning, Spyros Matsoukas, I Antti-Veikko, Bing Rosti, Zhang, Robert Moore, William Lewis, Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu, Holger Schwenk, Rico Sennrich, Barry Haddow, Alexandra Birch, Rico Sennrich, Barry Haddow, Alexandra Birch, Rico Sennrich, Yik-Cheung Tam, Ian Lane, Tanja Schultz, Jg Tiedemann, Nicola Ueffing, Hermann Ney, Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio, David Chiang, Junyoung Chung, C aglar Glc ehre, Kyunghyun Cho, Yoshua Bengio, Fabien Cromieres, Isao Goto, Bin Lu, Ka Chow, Eiichiro Sumita, Benjamin Tsou, Isao Goto, Ka Chow, Bin Lu, Eiichiro Sumita, Benjamin Tsou, Sepp Hochreiter, Jrgen Schmidhuber, Philipp Koehn, Franz Och, Daniel Marcu, Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, Evan Herbst, Philipp Koehn, Haitao Mi, Zhiguo Baskaran Sankaran, Abe Wang, Ittycheriah, Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu, Ilya Sutskever, Oriol Vinyals, Quoc V Le, Seiya Tokui, Kenta Oono, Shohei Hido, Justin Clayton, Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, Hang Li, Nicolas Boulanger-Lewandowski, Yoshua Bengio, Pascal Vincent, Xiaoguang Hu, Wei Li, Xiang Lan, Hua Wu, Haifeng Wang, Sbastien Jean, Kyunghyun Cho, Roland Memisevic, Yoshua Bengio, Nal Kalchbrenner, Phil Blunsom, Thang Luong, Ilya Sutskever, Quoc Le, Oriol Vinyals, Wojciech Zaremba, Ilya Sutskever, Oriol Vinyals, Quoc Le, Richard Zens, Daisy Stanton, Peng Xu, Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio, Ondej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Aurelie Neveol, Mariana Neves, Martin Popel, Matt Post, Raphael Rubino, Carolina Scarton, Lucia Specia, Marco Turchi, Karin Verspoor, Marcos Zampieri, Fabin Cromieres, Diederik Kingma, Jimmy Ba, Minh-Thang Luong, Hieu Pham, Christopher Manning, Toshiaki Nakazawa, Manabu Yaguchi, Kiyotaka Uchimoto, Masao Utiyama, Eiichiro Sumita, Sadao Kurohashi, Hitoshi Isahara, Graham Neubig, Yosuke Nakata, Shinsuke Mori, Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu, Ilya Sutskever, Oriol Vinyals, Quoc Le, Seiya Tokui, Kenta Oono, Shohei Hido, Justin Clayton, Sadaf Abdui, -Rauf, Holger Schwenk, Eneko Agirre, Carmen Banea, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Rada Mihalcea, German Rigau, Janyce Wiebe, Amittai Axelrod, Xiaodong He, Jianfeng Gao, Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio, Yoshua Bengio, Jrme Louradour, Ronan Collobert, Jason Weston, Ondrej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, Others, Mauro Cettolo, Christian Girardi, Marcello Federico, Boxing Chen, Roland Kuhn, George Foster, Colin Cherry, Fei Huang, David Chiang, Trevor Cohn, Cong Duy Vu, Ekaterina Hoang, Kaisheng Vymolova, Chris Yao, Gholamreza Dyer, Haffari, Michael Denkowski, Greg Hanneman, Alon Lavie, Bonnie Dorr, Bonnie Dorr, Lisa Pearl, Rebecca Hwa, Nizar Habash, Kevin Duh, Ching-Man Au Yeung, Tomoharu Iwata, Masaaki Nagata, Nadir Durrani, Fahim Dalvi, Hassan Sajjad, Stephan Vogel, M Amin Farajian, Rajen Chatterjee, Costanza Conforti, Shahab Jalalvand, Vevake Balaraman, Mattia Di, Duygu Gangi, Marco Ataman, Matteo Turchi, Marcello Negri, Federico, Christian Federmann, William Lewis, Victoria Fossum, Kevin Knight, Steven Abney, George Foster, Roland Kuhn, Pascale Fung, Percy Cheung, Kuzman Ganchev, Dipanjan Das, Cyril Goutte, Marine Carpuat, George Foster, Nizar Habash, Bonnie Dorr, Silja Hildebrand, Stephan Vogel, Jie Jiang, Andy Way, Julie Carson-Berndsen, Sergio Jimenez, Claudia Becerra, Alexander Gelbukh, Alon Lavie, William Lewis, Christian Federmann, Ying Xin, Pierre Lison, Jrg Tiedemann, Austin Matthews, Waleed Ammar, Archna Bhatia, Weston Feely, Greg Hanneman, Eva Schlinger, Swabha Swayamdipta, Yulia Tsvetkov, Alon Lavie, Chris Dyer, Yashar Mehdad, Matteo Negri, Marcello Federico, Haitao Mi, Zhiguo Wang, Abe Ittycheriah, Stefan Dragos, Daniel Munteanu, Marcu, Stefan Dragos, Daniel Munteanu, Marcu, Matteo Negri, Alessandro Marchetti, Yashar Mehdad, Luisa Bentivogli, Danilo Giampiccolo, Matteo Negri, Alessandro Marchetti, Yashar Mehdad, Luisa Bentivogli, Danilo Giampiccolo, Matteo Negri, Yashar Mehdad, Tsuyoshi Okita, K Sudip, Andy Naskar, Way, Jason Riesa, Daniel Marcu, Rico Sennrich, Orhan Firat, Kyunghyun Cho, Alexandra Birch, Barry Haddow, Julian Hitschler, Marcin Junczys-Dowmunt, Samuel Lubli, Antonio Valerio Miceli, Jozef Barone, Maria Mokry, Nadejde, Rico Sennrich, Barry Haddow, Alexandra Birch, Jason Smith, Chris Quirk, Kristina Toutanova, Lucia Specia, Dhwaj Raj, Marco Turchi, Jrg Tiedemann, Dekai Wu, Towski, Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, Oriol Vinyals, Bing Zhao, Stephan Vogel, Jiang Zhao, Man Lan, Zheng-Yu Niu - 2017 cited by 28

Recently, the attention mechanism plays a key role to achieve high performance for Neural Machine Translation models. However, as it computes a score function for the encoder states in all positions at each decoding step, the attention model greatly increases the computational complexity. In this paper, we investigate the adequate vision span of attention models in the context of machine translation, by proposing a novel attention framework that is capable of reducing redundant score computation dynamically. The term "vision span" means a window of the encoder states considered by the attention model in one step. In our experiments, we found that the average window size of vision span can be reduced by over 50% with modest loss in accuracy on English-Japanese and German-English translation tasks.