Experiments on predictability of word in context and information rate in natural language

Simple item page
dc.creatorManin, Dmitrii
dc.date2006-12-27
dc.date.accessioned2026-07-07T08:18:05Z
dc.date.available2026-07-07T08:18:05Z
dc.descriptionBased on data from a large-scale experiment with human subjects, we conclude that the logarithm of probability to guess a word in context (unpredictability) depends linearly on the word length. This result holds both for poetry and prose, even though with prose, the subjects don't know the length of the omitted word. We hypothesize that this effect reflects a tendency of natural language to have an even information rate.
dc.description14 pages, 3 figures
dc.identifierhttps://arxiv.org/abs/cs/0612136
dc.identifierhttp://arxiv.org/abs/cs/0612136
dc.identifierManin, D.Yu. 2006. Experiments on predictability of word in context and information rate in natural language. J. Information Processes (electronic publication, http://www.jip.ru), 6 (3), 229-236
dc.identifier.urihttp://salesiana.dossiersoluciones.com/handle/123456789/134311
dc.subjectInformation Theory
dc.titleExperiments on predictability of word in context and information rate in natural language
dc.typetext

Files

Collections

arXiv