the mit press probabilistic linguistics apr 2003

[...]... developing the concept of probabilistic grammar, which lies at the heart of probabilistic linguistics Since many di¤erent probabilistic grammars have been 12 Bod proposed in the literature, there is a need for a theory that creates some order among them, just as Formal Language Theory creates order among nonprobabilistic grammars While I will only scratch the surface of a Formal Stochastic Language Theory,... selecting the rule the rule the rule the rule the rule the rule the rule S ! NP VP from among the rules starting with an S, NP ! Mary from among the rules starting with an NP, VP ! V NP from among the rules starting with a VP, V ! likes from among the rules starting with a V, NP ! V NP from among the rules starting with an NP, V ! buzzing from among the rules starting with a V, NP ! bees from among the rules... each tree in the treebank corresponds to the structure as it was perceived for that sentence by some hypothetical natural language user (Some subcategorizations are omitted to keep the example simple.) Note that the only di¤erence between the two structures (apart from the words) is the syntactic label covering the last two words of the sentences, which is VP in the first sentence and NP in the second... somewhat di¤erent for linguistics Since probability theory does not form part of a traditional linguistics curriculum, probabilistic linguistics may not be as accessible as some other areas This is further reinforced by the disciplinary gap between probabilistic and categorical approaches, the first being dominant in psycholinguistics and natural language processing, the second in generative linguistics One... of the derivation corresponding to the tree in figure 2.3 is the joint probability of selecting the rules in table 2.2 The probability of (1) can be computed by dividing the number of occurrences of rule S ! NP VP by the number of occurrences of all rules that start with an S There are two rules S ! NP VP in the treebank, and the total number of S-rules is also two (in fact, they coincide); thus, the. .. on the other hand, describes the probability that both A and B occur as a joint event in an experiment where events can have more than one outcome; and this probability is equal to the product of the probabilities of A and B (or in the general case, to the product of the probability of A and the conditional probability of B given A) 2.4 Probabilistic Grammars With these concepts from probability theory... Elementary Probability Theory 23 Table 2.3 The probabilities of the various rules in table 2.2 Event (1) selecting the with an S (2) selecting the with an NP (3) selecting the with a VP (4) selecting the with a V (5) selecting the with an NP (6) selecting the with a V (7) selecting the with an NP Probability rule S ! NP VP from among the rules starting 1 rule NP ! Mary from among the rules starting 1/5... We then select the next rule among the rules that can be combined with the previous rule (i.e., that start with the same category as the leftmost category on the right-hand side of the previous rule), and we repeat this process until only words remain Note that this experiment is well defined only if each rule can indeed be combined with the previous rule and if the first rule starts with an S Thus, the. .. illustrate that one need not discard the many insights of modern linguistics in order to insightfully model this middle ground On the contrary, a probabilistic approach can push the boundaries of linguistic theory forward, by substantially enriching the current state of knowledge Probabilistic linguistics 2 Bod, Hay, and Jannedy increases the range of data for which a theory can account, and for which... what is on the lefthand side of the bar in (8) Most textbooks on probability theory first define the concept of conditional probability and then, from that, the formula for joint probability For the current exposition, it seemed more intuitive to do this the other way round From (8), Bayes’ rule can be derived First, we will rename the variables of (8): PðH jEÞ ¼ PðE; HÞ ; PðEÞ ð9Þ where, in the context . the contrary, a probabilistic approach can push the boundaries of linguistic theory forward, by substan- tially enriching the current state of knowledge. Probabilistic linguistics increases the. specialist. The probabilistic approach to the study of language may seem opposed to the categorical approach, which has dominated linguistics for over 40 years. Yet one thesis of this book is that the. researchers to change the level of magnification when exploring theoretical and practical problems in linguistics. Another was the sense that a handbook on probabilistic linguistics, providing