Sunday, August 16, 2009

Rasch theoretical analysis of timed reading tests

If the probability of an event occurring in a very short interval of time is θ, the probability that the event occurs a times in n intervals may be estimated by the Poisson distribution:

 p{a|n} ≈ n θae-n θ /a! (10)

Suppose n intervals may be aggregated into a time period of interest. For a telephone exchange, this might be a period of peak demand. For a reading test it might be five or ten minutes, however long the test lasts. Now imagine another time period used to define frequency of events λ. Rasch (op. cit. page 34) uses a 10 second interval, I prefer a minute, most physicists use a second, but as mentioned in the previous blog, it make no difference to the essence of the argument.

Note, however, in just a few lines we have referred to three distinct time intervals. First there is the very short time interval, used to define θ, and during which the event will only ever occur once. Rasch (op. cit. page 35) uses one hundredth of second for this, but conceptually it could be a lot smaller. Second, in order of size, is the frequency defining interval, such that λ is the number of times the event occurs (or is expected to occur) in this interval. Third is the experimental period, or period of interest made up by n of the very short intervals, and which could also be expressed as t of the frequency defining intervals (seconds, minutes or whatever), such that the expected number of events in the period could be expressed as either the left or right hand side of the equation below:

 nθ = λt (30)

The probability of of a specified number of events a occurring in the experimental period time period t then becomes:

 p{a|t} ≈ λtae-λt /a! (31)

Rasch then makes an observation which I skirted over on my first reading of the book, but which I used implicitly in my last blog: "the event that the number of words a read in a given time T exceeds a given number n is identical with the event that the time t used for reading n words is less than T.

 p{a ≥ N|T} = p{t ≤ T|N} (32)

The left hand side is the sum of all the probabilities that a is N, N+1, N+2 ... :

 p{a ≥ N|T} ≈ e-λt (λtN/N! + λtN+1/(N+1)! + λtN+2/(N+2)! ...) (33a) p{t ≤ T|N} ≈ e-λt (λtN/N! + λtN+1/(N+1)! + λtN+2/(N+2)! ...) (33b)

Rasch then throws in special case, which seems intuitively obvious, but I am sure he has good reason. In 33b he sets N to zero, so he is calculating the probability that within a certain time at least zero events have occurred:

 p{t ≤ T|0} ≈ e-λt (λt0/0! + λt1/(1)! + λt2/(2)! ...) 1 = e-λt (λ + λt + λt2/2! ...) (34) eλt = 1 + λt + λt2/2! ... λt = ln(1 + λt + λt2/2! ...)

All of which is supposed to add up to 1. I can't see it myself, but perhaps we'll use the expression later. A second special case is when N is 1, which is the probability that either zero or 1 events take place in time T:

 p{t ≤ T|1} ≈ e-λt (λt1/1! + λt2/(2)! + λt3/(3)! ...) = e-λt (λt + λt2/2! + λt3/3! ...) = e-λt λt (1 + λt/2! + λt2 /3! ...) = 1 - e-λt (35)

Again I can't see how Rasch gets to my 35 (his 6.4 (op. cit. page 39)) but I've included it for completeness, in case it makes sense later. And from here here jumps to:

 p{t|1} = λe-λt (36)

Where p{t|1}is the probability distribution for reading time of the first and any subsequent word. Rasch goes on to show a similar distribution for the reading times of N words, but I shall skip that, because the essential difference between a speeded test and my use of scoring rates is that I focus on the scoring "rate" for individual items, by recording the time take on each item, and dividing that into my unit time for computing rates. So for me, equation 35 is quite interesting, but anything on multiple words (or items) is not so.

The next section (op. cit. page 40) is also interesting to me, because Rasch talks of two students, A and B, who read a different rates. In fact he says student A reads twice as fast as student B:

 λA = 2λB (37)

Rasch then explains how to estimate the relative difficulty of texts, based on observed reading speeds. Each pupil reads a series of texts numbered 1 to k, and for each:

 λA1 = 2 λB1 λAi = 2 λBi

Dividing:

 λAi/λA1 = λBi/ λB1 (38)

So the ratio between the expected reading speeds for text i and another test (such as text 1) is constant for all pupils, regardless of the ratio of the expected reading speeds for the pupils. Rasch generalises this for student v:

 λvi/ λv1 = εi (39)

Rearranging:

 λvi = λv1εi (40)

Rasch calls λv1 a person factor and ε a text factor for reading speed (op. cit. page 40) And consistently with his preference for difficulty as a term over easiness he defines difficulty in relation to reading speed as:

 δi = 1/εi (41)

Redefining λv1 as ζv, Rasch can now express the expected reading speed for any student/text combination as:

 λvi = ζv/ δi (42)

Regardless of the contortions required to express expected reading speed in the same way as the probability of a correct reading, Rasch emphasises that accuracy and speed are not the same things, although they may be related, and addressing any such relationship remains an interesting possible topic for empirical research.