A challenge set approach to evaluating machine translation

Download
  1. (PDF, 421 KB)
  2. Get@NRC: A challenge set approach to evaluating machine translation (Opens in a new window)
AuthorSearch for: ; Search for: ; Search for:
TypeArticle
Journal titleComputer Science
Article numberarXiv:1704.07431
Pages# of pages: 27
AbstractNeural machine translation represents an exciting leap forward in translation quality. But what longstanding weaknesses does it resolve, and which remain? We address these questions with a challenge set approach to translation evaluation and error analysis. A challenge set consists of a small set of sentences, each hand-designed to probe a system's capacity to bridge a particular structural divergence between languages. To exemplify this approach, we present an English-French challenge set, and use it to analyze phrase-based and neural systems. The resulting analysis provides not only a more fine-grained picture of the strengths of neural systems, but also insight into which linguistic phenomena remain out of reach.
Publication date
PublisherCornell University Library
Linkhttps://arxiv.org/abs/1704.07431
LanguageEnglish
AffiliationInformation and Communication Technologies; National Research Council Canada
Peer reviewedNo
NPARC number23002238
Export citationExport as RIS
Report a correctionReport a correction
Record identifierd03ee995-bcbb-4464-83d9-68f874d84e6c
Record created2017-09-12
Record modified2017-09-12
Bookmark and share
  • Share this page with Facebook (Opens in a new window)
  • Share this page with Twitter (Opens in a new window)
  • Share this page with Google+ (Opens in a new window)
  • Share this page with Delicious (Opens in a new window)