< Back to previous page

Publication

Character-word LSTM language models

Book Contribution - Book Chapter Conference Contribution

© 2017 Association for Computational Linguistics. We present a Character-Word Long Short- Term Memory Language Model which both reduces the perplexity with respect to a baseline word-level language model and reduces the number of parameters of the model. Character information can reveal structural (dis)similarities between words and can even be used when a word is out-of-vocabulary, thus improving the modeling of infrequent and unknownwords. By concatenating word and character embeddings, we achieve up to 2.77% relative improvement on English compared to a baseline model with a similar amount of parameters and 4.57% on Dutch. Moreover, we also outperform baseline word-level models with a larger number of parameters.
Book: Proceedings EACL 2017
Pages: 417 - 427
ISBN:9781510838604
Publication year:2017
BOF-keylabel:yes
IOF-keylabel:yes
Authors from:Higher Education
Accessibility:Open