Schedule of Topics

A really important paper by Ionnidis about problems with statistical hypothesis testing is Why Most Published Research Findings Are False; for a very readable discussion see Trouble at the Lab, The Economist, Oct 19, 2013 and the really great accompanying video. (Show that one to your friends and family!) For an interesting response, see Most Published Research Findings Are False--But a Little Replication Goes a Long Way.

Kilgarriff (2005) is a fun and contrarian read regarding the use of hypothesis testing methodology specifically in language research.

Named entities represent another form of lexical association. Named entity recognition is introduced in Jurafsky and Martin, Ch 22 and Ch 7 of the NLTK book.

Piantadosi et al. (2011), Word lengths are optimized for efficient communication

Some other interesting work on proposed information theoretic explanations of (or at least connections to) to how language works: Jaeger (2010), Redundancy and reduction: Speakers manage syntactic information density; Maurits et al. (2010), Why are some word orders more common than others? A uniform information density account; Han, Jurafsky, and Futrell (2020), Universals of word order reflect optimization of grammars for efficient communication. Although it's a bit older, also see the syllabus for a 2009 seminar taught by Dan Jurafsky and Michael Ramscar, Information-Theoretic Models of Language and Cognition, which looks as if it was awesome.

Roger Levy provides a formal proof that uniform information density minimizes the "difficulty" of interpreting utterances. The proof assumes that, for any given word i in an utterance, the difficulty of processing it is some power k of its surprisal with k > 1.