Variable Word Rate N-grams
| dc.creator | Gotoh, Yoshihiko | |
| dc.creator | Renals, Steve | |
| dc.date | 2000-03-29 | |
| dc.date.accessioned | 2026-07-07T03:16:08Z | |
| dc.date.available | 2026-07-07T03:16:08Z | |
| dc.description | The rate of occurrence of words is not uniform but varies from document to document. Despite this observation, parameters for conventional n-gram language models are usually derived using the assumption of a constant word rate. In this paper we investigate the use of variable word rate assumption, modelled by a Poisson distribution or a continuous mixture of Poissons. We present an approach to estimating the relative frequencies of words or n-grams taking prior information of their occurrences into account. Discounting and smoothing schemes are also considered. Using the Broadcast News task, the approach demonstrates a reduction of perplexity up to 10%. | |
| dc.description | 4 pages, 4 figures, ICASSP-2000 | |
| dc.identifier | https://arxiv.org/abs/cs/0003081 | |
| dc.identifier | http://arxiv.org/abs/cs/0003081 | |
| dc.identifier.uri | http://salesiana.dossiersoluciones.com/handle/123456789/30238 | |
| dc.subject | Computation and Language | |
| dc.subject | I.2.7 | |
| dc.title | Variable Word Rate N-grams | |
| dc.type | text |