John Goldsmith (University of Chicago), "Information Theory for Linguists"
Abstract: Information came on the scene in the late 1940s, and something about it seemed to speak to linguists at the time, but not much came of that interest for quite a while (despite the enthusiasm of Jakobson, Harris, and Hockett--though Chapter 7 of Trubetzkoy's Grundzüge was prescient in this regard). In the last 30 years, however, things have changed a great deal. There is much more that linguists can learn and employ from information theory these days. The principal reason for the change is that in its original form, information theory was devoted to averages over large ensembles, and this averaging had the unfortunate effect of washing out what was of greatest interest to linguists. I will illustrate some of the ways that linguists now look at some of the central ideas of information theory (entropy, mutual information, ideal compressed length, e.g.) with considerable interest. I'll choose my examples from phonology and morphology, but it is not hard to apply the same ideas to syntax as well. The specific focus will be on obtaining methods that induce grammatical structure from data.