Proceedings of the Eighteenth Conference on Computational Natural Language Learning

Authors: Roser Morante, Wen-Tau Yih, Wen-Tau Scott, Yih, Steven Abney, Kai-Wei Chang, Shay Cohen, Trevor Cohn, Aron Culotta, Michael Gamon, Andr Martins, Sebastian Riedel, Nathan Schneider, Sabine Schulte, Sima Khalil, An, Mark Steedman, Ivan Titov, Peter Turney, Anders Sgaard, Anders Johannsen, Barbara Plank, Dirk Hovy, Hector Martinez, Rebecca Mason, Eugene Charniak, Tamara Berg, Alexander Berg, Jonathan Shih, David Blei, Michael Jordan, James Clarke, Mirella Lapata, D Lowe, F Andr, Noah Martins, Smith, R Mason, E Charniak, Rebecca Mason, David Mimno, Hanna Wallach, Jason Naradowsky, David Smith, Andrew Mccallum, Margaret Mitchell, Jesse Dodge, Amit Goyal, Kota Yamaguchi, Karl Stratos, Xufeng Han, Alyssa Mensch, Alexander Berg, Tamara Berg, Hal Daum, Iii, Aude Oliva, Antonio Torralba, V Ordonez, G Kulkarni, T Berg, Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu, G Al-Gaphari, M Al-Yadoumi, Yaser Al-Onaizan, Kevin Knight, Tim Buckwalter, Achraf Chalabi, Hany Gerges, Ahmed, Kholy, Nizar Habash, Ramy Eskander, Nizar Habash, Owen Rambow, Nadi Tomeh, Noura Farra, Nadi Tomeh, Alla Rozovskaya, Nizar Habash, Andrew Freeman, Sherri Condon, Christopher Ackerman, Nizar Habash, Abdelhadi Soudi, Tim Buckwalter, Nizar Habash, Mona Diab, Owen Rabmow, Nizar Habash, Ramy Eskander, Abdelati Hawwari, Nizar Habash, Ryan Roth, Owen Rambow, Ramy Eskander, Nadi Tomeh, Nizar Habash, Kevin Knight, Jonathan Graehl, Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, Evan Herbst, Franz Joseph, Och, Hermann Ney, Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, Ryan Roth, Wael Salloum, Nizar Habash, Wael Salloum, Nizar Habash, Hassan Sawaf, Khaled Shaalan, Hitham Abo Bakr, Ibrahim Ziedan, Andreas Stolcke, Ines Zribi, Rahma Boujelbane, Abir Masmoudi, Mariem Ellouze, Lamia Belguith, Nizar Habash, Elif Aktolga, James Allan, David Smith, L Allison, T I Dix, Daniel Br, Torsten Zesch, Iryna Gurevych, Chris Biemann, Michael Collins, Nigel Duffy, Marco Ernandes, Giovanni Angelini, Marco Gori, Marco Ernandes, Giovanni Angelini, Marco Gori, David Ferrucci, Adam Lally, David Ferrucci, Eric Brown, Jennifer Chu-Carroll, James Fan, David Gondek, Aditya Kalyanpur, Adam Lally, J Murdock, Eric Nyberg, John Prager, Nico Schlaefer, Chris Welty, David Ferrucci, Eric Brown, Jennifer Chu-Carroll, James Fan, David Gondek, Aditya Kalyanpur, Adam Lally, J Murdock, Eric Nyberg, John Prager, Nico Schlaefer, Christopher Welty, Evgeniy Gabrilovich, Shaul Markovitch, James F Allen, Marco Baroni, Robert Zamparelli, Yoshua Bengio, Rjean Ducharme, Pascal Vincent, Jonathan Berant, Ido Dagan, Jacob Goldberger, Nathanael Chambers, Dan Jurafsky, Nathanael Chambers, Daniel Jurafsky, Jackie Chi, Kit Cheung, Hoifung Poon, Lucy Vanderwende, Timothy Chklovski, Patrick Pantel, R Collobert, J Weston, L Bottou, M Karlen, K Kavukcuoglu, P Kuksa, Marie-Catherine De Marneffe, Bill Maccartney, Christopher Manning, Charles Fillmore, Lea Frermann, Ivan Titov, Manfred Pinkal, Martin Charles, Golumbic, Ron Shamir, Andrew Gordon, Rakesh Gupta, Mykel Kochenderfer, Ashutosh Modi, Ivan Titov, Alexandre Klementiev, Robert Parker, David Graff, Junbo Kong, Ke Chen, Kazuaki Maeda, Michaela Regneri, Alexander Koller, Manfred Pinkal, Sebastian Riedel, Limin Yao, Andrew Mccallum, Benjamin Marlin, Richard Socher, Brody Huval, Christopher Manning, Andrew Ng, Joseph Turian, Lev Ratinov, Yoshua Bengio, Naushad Uzzaman, Hector Llorens, Leon Derczynski, James Allen, Marc Verhagen, James Pustejovsky, Miles Anne H Anderson, Ellen Bader, Elizabeth Bard, Gwyneth Boyle, Simon Doherty, Stephen Garrod, Jacqueline Isard, Jan Kowtko, Jim Mcallister, Miller, Yoav Artzi, Luke Zettlemoyer, Michele Basseville, Igor Nikiforov, Harr Srk Branavan, Luke Chen, Regina Zettlemoyer, Barzilay, Elia Bruni, Nam Khanh Tran, Marco Baroni, L David, Raymond Chen, Mooney, Dave Golland, Percy Liang, Dan Klein, Christopher Kennedy, Louise Mcnally, Shimon Kogan, Dimitry Levin, Jacob Bryan R Routledge, Noah Sagi, Smith, Thomas Kollar, Stefanie Tellex, Deb Roy, Nicholas Roy, Jayant Krishnamurthy, Thomas Kollar, Percy Liang, Michael Jordan, Dan Klein, C Dong, Jorge Liu, Nocedal, Slav Petrov, Dan Klein, Stefanie Tellex, Thomas Kollar, Steven Dickerson, Matthew Walter, Ashis Gopal Banerjee, Seth Teller, Nicholas Roy, Robert Tibshirani, Adam Vogel, Dan Jurafsky, Haonan Yu, Jeffrey Mark, Siskind, Jin Yu, Ehud Reiter, Jim Hunter, Chris Mellish, M Jeffrey, Zacks, M Khena, Swallow, M John, Raymond Zelle, Mooney, Luke Zettlemoyer, Michael Collins, E istein, Julien Andersen, Edward Nioche, John Briscoe, Carroll, Marco Baroni, Alessandro Lenci, Marco Baroni, Raffaella Bernardi, Ngoc-Quynh Do, Chung-Chieh Shan, Chris Biemann, Gilles Bisson, Claire Ndellec, Dolores Caamero, Ted Briscoe, John Carroll, Rebecca Watson, Sharon Caraballo, Philipp Cimiano, Steffen Staab, Daoud Clarke, Ido Dagan, Lillian Lee, Fernando Pereira, Marti Hearst, Lili Kotlerman, Ido Dagan, Idan Szpektor, Maayan Zhitomirsky-Geffet, Dekang Lin, Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean, George Miller, Andriy Mnih, Geoffrey Hinton, Patrick Pantel, Deepak Ravichandran, Rion Snow, Daniel Jurafsky, Andrew Ng, Idan Szpektor, Ido Dagan, Joseph Turian, Lev Ratinov, Y Bengio, D Peter, Turney, Akira Ushioda, Andreas Wagner, Julie Weeds, David Weir, Julie Weeds, David Weir, Diana Mccarthy, Maayan Zhitomirsky, -Geffet, Ido Dagan, Rie Kubota, Ando, Tong Zhang, Yoshua Bengio, Rejean Ducharme, Pascal Vincent, Yoshua Bengio, Peter Peter F Brown, Robert Desouza, Vincent Mercer, Pietra Della, Jenifer Lai, Ronan Collobert, Jason Weston, Paramveer Dhillon, Dean Foster, Lyle Ungar, John Duchi, Elad Hazan, Yoram Singer, A David, Huffman, Yiming David D Lewis, Tony Yang, Fan Rose, Li, Dekang Lin, Xiaoyun Wu, Tomas Mikolov, Martin Karafit, Lukas Burget, Jan Cernock, Sanjeev Khudanpur, Scott Miller, Jethran Guinness, Alex Zamanian, Andriy Mnih, Geoffrey Hinton, Frederic Morin, Yoshua Bengio, Fernando Pereira, Naftali Tishby, Lillian Lee, Lev Ratinov, Dan Roth, Jun Suzuki, Hideki Isozaki, Erik, Tjong Kim, Sang, Fien De, Meulder, Joseph Turian, Lev Ratinov, Yoshua Bengio, Tong Zhang, David Johnson, Meni Adler, Jonathan Berant, Ido Dagan, Michele Banko, Michael Cafarella, Stephen Soderland, Matthew Broadhead, Oren Etzioni, Jonathan Berant, Ido Dagan, Jacob Goldberger, Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, Jamie Taylor, Kenneth Church, Patrick Hanks, Ido Dagan, Dan Roth, Mark Sammons, Fabio Zanzotto, Arthur Dempster, Nan Laird, Donald Rubin, Oren Etzioni, Michele Banko, Stephen Soderland, Daniel Weld, Anthony Fader, Stephen Soderland, Oren Etzioni, Yoav Goldberg, Jon Orwant, Lili Kotlerman, Ido Dagan, Idan Szpektor, Maayan Zhitomirsky-Geffet, Vladimir Levenshtein, Dekang Lin, Michael Mausam, Stephen Schmitz, Robert Soderland, Oren Bart, Etzioni, Quinn Mcnemar, Yashar Mehdad, Giuseppe Carenini, Raymond Ng, Shafiq Joty, Oren Melamud, Jonathan Berant, Ido Dagan, Jacob Goldberger, Idan Szpektor, Sebastian Riedel, Limin Yao, Andrew Mccallum, Benjamin Marlin, Rion Snow, Dan Jurafsky, Andrew Ng, Julie Weeds, David Weir, F Peter, Peter Brown, Robert Desouza, Vincent Mercer, Jenifer Della Pietra, Lai, Mary, Elaine Califf, Raymond Mooney, Andrew Carlson, Justin Betteridge, Richard Wang, Estevam Hruschka, Jr, Tom Mitchell, Laura Chiticariu, Yunyao Li, Frederick Reiss, Fabio Ciravegna, William Cohen, Yoram Singer, Michael Collins, Yoram Singer, J Curran, T Murphy, B Scholz, D Downey, O Etzioni, S Soderland, D Weld, Oren Etzioni, Michael Cafarella, Doug Downey, Ana-Maria Popescu, Tal Shaked, Stephen Soderland, Daniel Weld, Alexander Yates, Anthony Fader, Stephen Soderland, Oren Etzioni, Dayne Freitag, Sonal Gupta, Christopher Manning, A Marti, Hearst, Winston Lin, Roman Yangarber, Ralph Grishman, Michael Mausam, Stephen Schmitz, Robert Soderland, Oren Bart, Etzioni, Tara Mcintosh, James Curran, Ramesh Nallapati, Christopher Manning, Patrick Pantel, Deepak Ravichandran, Hoifung Poon, Pedro Domingos, Ellen Riloff, Alan Ritter, Luke Zettlemoyer, Oren Mausam, Etzioni, E Agichtein, L Gravano, Javier Artiles, Qi Li, Taylor Cassidy, Suzanne Tamang, Heng Ji, Steven Bethard, James Martin, Chris Burges, Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, R Estevam, Tom Hruschka, Mitchell, Nathanael Chambers, Shan Wang, Dan Jurafsky, X Angel, Christopher Chang, Manning, Silviu Cucerzan, Silviu Cucerzan, R Fikes, N Nilsson, Jerome H Friedman, Guillermo Garrido, Bernardo Cabaleiro, Anselmo Penas, Alvaro Rodrigo, Damiano Spina, Guillermo Garrido, Anselmo Penas, Bernardo Cabaleiro, Alvaro Rodrigo, Ji Heng, Ralph Grishman, Hoa Dang, Ji Heng, Taylor Cassidy, Qi Li, Suzanne Tamang, Ekaterina Kozlova, Manicheva Maria, Petrova Elena, Tatiana Popova, Mike Mintz, Steven Bills, Rion Snow, Daniel Jurafsky, James Pustejovsky, Marc Verhagen, James Pustejovsky, M Jos, Robert Castano, Roser Ingria, Robert Sauri, Andrea Gaizauskas, Graham Setzer, Katz, Dragomir R Radev, Avirup Sil, Alexander Yates, Avirup Sil, Alexander Yates, Avirup Sil, Fei Huang, Alexander Yates, Jannik Strtgen, Michael Gertz, Gjergji Fabian M Suchanek, Gerhard Kasneci, Weikum, Mihai Surdeanu, Sonal Gupta, John Bauer, David Mc-Closky, X Angel, Valentin Chang, Christopher Spitkovsky, Manning, Partha Pratim Talukdar, Derry Wijaya, Tom Mitchell, Partha Pratim Talukdar, Derry Wijaya, Tom Mitchell, Marc Verhagen, Roser Sauri, Tommaso Caselli, James Pustejovsky, Yafang Wang, Mingjie Zhu, Lizhen Qu, Marc Spaniol, Gerhard Weikum, Yafang Wang, Bin Yang, Lizhen Qu, Marc Spaniol, Gerhard Weikum, Daniel Weld, Raphael Hoffmann, Fei Wu, Yiping Zhou, Lan Nie, Omid Rouhani-Kalleh, Flavian Vasile, Scott Gaffney, L Bottou, S Buchholz, E Marsi, D Burkett, S Petrov, J Blitzer, D Klein, W Che, M Wang, C Manning, T Liu, R Collobert, J Weston, R Collobert, J Weston, L Bottou, M Karlen, K Kavukcuoglu, P Kuksa, D Das, S Petrov, G Durrett, A Pauls, D Klein, K Ganchev, J Gillenwater, B Taskar, W Guo, M Diab, J Henderson, R Hwa, P Resnik, A Weinberg, C Cabezas, O Kolak, W Jiang, Q Liu, Y L, M Khapra, S Sohoney, A Kulkarni, P Bhattacharyya, S Kim, G Lee, S Kim, M Jeong, J Lee, G Lee, S Kim, K Toutanova, H Yu, A Klementiev, I Titov, B Bhattarai, M Kozhevnikov, I Titov, K Liu, Y L, W Jiang, Q Liu, R Mcdonald, K Crammer, F Pereira ; A, R Mcdonald, F Pereira, K Ribarov, J Haji, R Mcdonald, S Petrov, K Hall, T Naseem, H Chen, R Barzilay, M Johnson, T Naseem, R Barzilay, A Globerson, J Nivre, J Hall, S Kbler, R Mcdonald, J Nilsson, S Riedel, D Yuret, S Petrov, D Das, R Mcdonald, eljko Agi, Danijela Merkler, Daa Berovi, Miguel Ballesteros, Joakim Nivre, Sabine Buchholz, Erwin Marsi, Marie-Catherine De Marneffe, Bill Maccartney, Christopher Manning, Greg Durrett, Adam Pauls, Dan Klein, Kuzman Ganchev, Dipanjan Das, Johan Hall, Jens Nilsson, Joakim Nivre, Glsen Eryigit, Beta Megyesi, Mattias Nilsson, Markus Saers, Kenneth Heafield, Ivan Pouzyrevsky, Jonathan Clark, Philipp Koehn, Rebecca Hwa, Philip Resnik, Amy Weinberg, Clara Cabezas, Okan Kolak, Philipp Koehn, Franz, Josef Och, Daniel Marcu, Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Christopher Dyer, Ondej Bojar, Alexandra Constantin, Evan Herbst, Philipp Koehn, Ryan Mcdonald, Slav Petrov, Keith Hall, Ryan Mcdonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith Hall, Slav Petrov, Hao Zhang, Oscar Tckstrm, Claudia Bedini, Nria Bertomeu Castell, Jungmee Lee, Tahira Naseem, Regina Barzilay, Amir Globerson, Joakim Nivre, Johan Hall, Jens Nilsson, Joakim Nivre, Johan Hall, Jens Nilsson, Glsen Eryigit, Svetoslav Marinov, Joakim Nivre, Johan Hall, Sandra Kbler, Ryan Mc-Donald, Jens Nilsson, Sebastian Riedel, Deniz Yuret, Joakim Nivre, Johan Hall, Jens Nilsson, Atanas Chanev, Sandra Gls en Eryiit, Svetoslav Kbler, Erwin Marinov, Marsi, Josef Franz, Hermann Och, Ney, Josef Franz, Och, Slav Petrov, Dipanjan Das, Ryan Mcdonald, Frank Smadja, Vasileios Hatzivassiloglou, Kathleen Mckeown, Oscar Tckstrm, Ryan Mcdonald, Jakob Uszkoreit, Oscar Tckstrm, Dipanjan Das, Slav Petrov, Ryan Mcdonald, Joakim Nivre, Oscar Tckstrm, Ryan Mcdonald, Joakim Nivre, David Yarowsky, Grace Ngai, Richard Wicentowski, Daniel Zeman, Philip Resnik, Yue Zhang, Joakim Nivre, Hai Zhao, Yan Song, Chunyu Kit, Guodong Zhou, Jason Baldridge, Srinivas Bangalore, Aravind Joshi, Matthew Beal, Zoubin Ghahramani, Carl Rasmussen, Taylor Berg-Kirkpatrick, Alexandre Bouchard-Ct, John Denero, Dan Klein, Yonatan Bisk, Julia Hockenmaier, Phil Blunsom, Trevor Cohn, Johan Bos, Cristina Bosco, Alessandro Mazzei, Cristina Bosco, Vincenzo Lombardo, Daniela Vassallo, Leonardo Lesmo, Christopher Carter, Robert Kohn, Chi Zhiyi, Dipanjan Das, Slav Petrov, Dan Garrette, Jason Baldridge, Dan Garrette, Jason Baldridge, Dan Garrette, Jason Mielens, Jason Baldridge, Yoav Goldberg, Meni Adler, Michael Elhadad, Sharon Goldwater, Thomas Griffiths, Julia Hockenmaier, Mark Steedman, Mark Johnson, Sharon Goldwater, Mark Johnson, Julian Kupiec, Yoong Keok, Lee, Aria Haghighi, Regina Barzilay, Mitchell Marcus, Beatrice Santorini, Mary Marcinkiewicz, Bernard Merialdo, Sujith Ravi, Kevin Knight, Sujith Ravi, Jason Baldridge, Kevin Knight, Noah Smith, Jason Eisner, Mark Steedman, Jason Baldridge, Daniel Tse, James Curran, Jurgen Van Gael, Yunus Saatci, Yee Teh, Zoubin Ghahramani, Jurgen Van Gael, Andreas Vlachos, Zoubin Ghahramani, Jonathan Weese, Chris Callison-Burch, Adam Lopez, Nianwen Xue, Fei Xia, Fu-Dong Chiou, Martha Palmer, Luke Zettlemoyer, Michael Collins, Jeff Bilmes, Katrin Kirchhoff, Stanley Chen, Joshua Goodman, David Chiang, Jonathan Clark, Chris Dyer, Alon Lavie, Noah Smith, Maria Josep, Franc Crego, Jos ois Yvon, Mario, Nadir Durrani, Helmut Schmid, Alexander Fraser, Nadir Durrani, Alexander Fraser, Helmut Schmid, Yang Feng, Trevor Cohn, Heidi Fox, Michel Galley, Christopher Manning, Reinhard Kneser, Hermann Ney, Philipp Koehn, Franz Och, Daniel Marcu, Philipp Koehn, Hieu Hoang, Alexandra Mayne, Christopher Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, Evan Herbst, Hai-Son Le, Alexandre Allauzen, Franc ois Yvon, Frans Och, Hermann Ney, J Frans, Och, Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu, Andreas Stolcke, Chris Callison-Burch, Philipp Koehn, Christof Monz, Kay Peterson, Mark Przybocki, Omar Zaidan, Colin Cherry, George Foster, Jonathan Clark, Chris Dyer, Alon Lavie, Noah Smith, Hal Daum, , Iii, Jagadeesh Jagarlamudi, Yonggang Deng, Jia Xu, Yuqing Gao, Qing Dou, Kevin Knight, Qing Dou, Kevin Knight, Nikesh Garera, David Yarowsky, Aria Haghighi, Percy Liang, Taylor Berg-Kirkpatrick, Dan Klein, Zhongjun He, Yao Meng, Hao Yu, Ann Irvine, Chris Callison-Burch, Ann Irvine, Chris Callison-Burch, Howard Johnson, Joel Martin, George Foster, Roland Kuhn, Alex Klementiev, Ann Irvine, Chris Callison-Burch, David Yarowsky, Philipp Koehn, Kevin Knight, Philipp Koehn, Franz Joseph Och, Daniel Marcu, Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, Evan Herbst, Philipp Koehn, Prasanth Kolachina, Nicola Cancedda, Marc Dymetman, Sriram Venkatapathy, Dragos Munteanu, Daniel Marcu, Josef Franz, Hermann Och, Ney, Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu, Matt Post, Chris Callison-Burch, Miles Osborne, Reinhard Rapp, Sujith Ravi, Kevin Knight, Sujith Ravi, Maryam Majid Razmara, Reza Siahbani, Anoop Haffari, Sarkar, Charles Schafer, David Yarowsky, Jason Smith, Chris Quirk, Kristina Toutanova, Richard Zens, Daisy Stanton, Peng Xu, Jiajun Zhang, Chengqing Zong, Rami Al-Rfou, Bryan Perozzi, Steven Skiena, Marco Baroni, Alessandro Lenci, Marco Baroni, Alessandro Lenci, Marco Baroni, Georgiana Dinu, Germn Kruszewski, Yoshua Bengio, Rjean Ducharme, Pascal Vincent, Christian Jauvin, William Blacoe, Mirella Lapata, John Bullinaria, Joseph Levy, Kenneth Church, Patrick Hanks, Ronan Collobert, Jason Weston, Ronan Collobert, Jason Weston, Lon Bottou, Michael Karlen, Koray Kavukcuoglu, Pavel Kuksa, Ido Dagan, Fernando Pereira, Lillian Lee, David Jurgens, Peter Turney, Saif M Mohammad, Keith Holyoak, Lili Kotlerman, Ido Dagan, Idan Szpektor, Maayan Zhitomirsky-Geffet, Omer Levy, Yoav Goldberg, Dekang Lin, Patrick Pantel, Dekang Lin, Tomas Mikolov, Stefan Kombrink, Lukas Burget, Sanjeev Cernocky, Khudanpur, Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory Corrado, Jeffrey Dean, Tomas Mikolov, Wen-Tau Yih, Geoffrey Zweig, Jeff Mitchell, Mirella Lapata, Andriy Mnih, Geoffrey Hinton, Yoshiki Niwa, Yoshihiko Nitta, Sebastian Pad, Mirella Lapata, Fernando Pereira, Naftali Tishby, Lillian Lee, Richard Socher, Jeffrey Pennington, Eric Huang, Andrew Ng, Christopher Manning, Joseph Turian, Lev Ratinov, Yoshua Bengio, D Peter, Michael Turney, Littman, D Peter, Patrick Turney, Pantel, D Peter, Turney, D Peter, Turney, D Peter, Turney, Alisa Zhila, Wen-Tau Yih, Christopher Meek, Geoffrey Zweig, Tomas Mikolov, Eneko Agirre, Enrique Alfonseca, Keith Hall, Jana Kravalova, Marius Pas, Aitor Soroa, Yoshua Bengio, Rjean Ducharme, Pascal Vincent, Christian Jauvin, F Peter, Peter Brown, Robert Desouza, Vincent Mercer, Jenifer Della Pietra, Lai, Elia Bruni, Gemma Boleda, Marco Baroni, Nam-Khanh Tran, Alexander Budanitsky, Graeme Hirst, John Bullinaria, Joseph Levy, Kenneth Church, Patrick Hanks, Ronan Collobert, Jason Weston, Georgiana Dinu, Mirella Lapata, Lev Finkelstein, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, Eytan Ruppin, James Gorman, James Curran, Reinhard Kneser, Hermann Ney, Dekang Lin, Kevin Lund, Curt Burgess, Oren Melamud, Jonathan Berant, Ido Dagan, Jacob Goldberger, Idan Szpektor, Tomas Mikolov, Martin Karafit, Lukas Burget, Jan Cernock, Sanjeev Khudanpur, Jeff Mitchell, Mirella Lapata, Adam Pauls, Dan Klein, Adam Pauls, Dan Klein, Martin Riedl, Chris Biemann, Tony Rose, Mark Stevenson, Miles Whitehead, Herbert Rubenstein, John Goodenough, Maria Ruiz-Casado, Enrique Alfonseca, Pablo Castells, Ming Tan, Wenli Zhou, Lei Zheng, Shaojun Wang, D Peter, Patrick Turney, Pantel, D Peter, Turney, Dongqiang Yang, David Powers, Dongqiang Yang, David Powers, Mehmet Ali Yatbaz, Enis Sert, Deniz Yuret, Elika Bergelson, Daniel Swingley, Luc Boruta, Sharon Peperkamp, Benot Crabb, Emmanuel Dupoux, Luc Boruta, M Brent, N Feldman, T Griffiths, S Goldwater, J Morgan, N Feldman, B Myers, K White, T Griffiths, J Morgan, Abdellah Fourtassi, Emmanuel Dupoux, Abdellah Fourtassi, Benjamin Brschinger, Mark Johnson, Emmanuel Dupoux, Abdellah Fourtassi, Ewan Dunbar, Emmanuel Dupoux, Abdellah Fourtassi, Thomas Schatz, Balakrishnan Varadarajan, Emmanuel Dupoux, Stella Frank, Naomi Feldman, Sharon Goldwater, Judit Gervain, Jacques Mehler, Sharon Goldwater, Thomas Griffiths, Mark Johnson, Zellig Harris, Valerie Hazan, Sarah Barrett, James Hillenbrand, Laura Getty, Michael Clark, Kimberlee Wheeler, M Huijbregts, M Mclaren, D Van Leeuwen, A Jansen, K Church, Aren Jansen, Emmanuel Dupoux, Sharon Goldwater, Mark Johnson, Sanjeev Khudanpur, Kenneth Church, Naomi Feldman, Hynek Hermansky, Florian Metze, Richard Rose, Mike Seltzer, Pascal Clark, Ian Mcgraw, Balakrishnan Varadarajan, Erin Bennett, Benjamin Borschinger, Justin Chiu, Ewan Dunbar, Abdallah Fourtassi, David Harwath, Chia Ying Lee, Keith Levin, Atta Norouzian, Vijay Peddinti, Rachel Richardson, Thomas Schatz, Samuel Thomas, Mark Johnson, Thomas Griffiths, Sharon Goldwater, W Peter, Richard Jusczyk, Aslin, K Thomas, Susan Landauer, Dumais, J Lany, J Saffran, D Lawrence, C Lee, J Glass, Bogdan Ludusan, Maarten Versteegh, Aren Jansen, Guillaume Gravier, Xuan-Nga Cao, Mark Johnson, Emmanuel Dupoux, Kikuo Maekawa, Hanae Koiso, Sadaoki Furui, Hitoshi Isahara, D Mandel, P Jusczyk, D Pisoni, Andrew Martin, Sharon Peperkamp, Emmanuel Dupoux, J Maye, J Werker, L Gerken, C Ngon, A Martin, E Dupoux, D Cabrol, M Duthat, S Peperkamp, S Nittrouer, J Pierrehumbert, J Pitman, M Yor, Shukla, White, Aslin, G Vallabha, J Mcclelland, F Pons, J Werker, S Amano, Sanjeev Balakrishnan Varadarajan, Emmanuel Khudanpur, Dupoux, Sandra Waxman, Susan Gelman, J Werker, S Curtin, Janet Werker, Richard Tees, H Yeung, J Werker - 2014 cited by 40

In NLP, we need to document that our proposed methods perform significantly better with respect to standard metrics than previous approaches, typically by reporting p-values obtained by rank-or randomization-based tests. We show that significance results following current research standards are unreliable and, in addition, very sensitive to sample size, covariates such as sentence length, as well as to the existence of multiple metrics. We estimate that under the assumption of perfect metrics and unbiased data, we need a significance cut-off at 0.0025 to reduce the risk of false positive results to <5%. Since in practice we often have considerable selection bias and poor metrics, this, however, will not do alone.