Thursday, August 6, 2009

Substitution and Manipulation

I have decided that I really don't like the Rasch personal and item parameters, because they seem counter intuitive, and I don't like his use of the Poisson Distribution, because the assumptions required by it are unrealistic, certainly if you broaden the discussion beyond a very easy reading test. But I have resolved to continue anyway. I've been thinking about this for years, and at last I've bought the book, so I don't have to worry about library or other loan returns. I can take as long as I like, and I have resolved to understand every equation in the book, if only to disagree with it, or at least with it's use.

I mentioned at the end of my last blog that Rasch refers to the "probability of misreading" (op. cit. page 16) in a test, but he is a bit vague as to whether this a single error or several. The vagueness continues into the next section (op. cit. page 17), when he refers back to his equation 2.1, which is my equation 12:

θvi = δiv (12)

When this equation is introduced, θvi is defined as "the probability of making a mistake in a test" (op. cit. page 16, his italics, but my emphasis, because he italicised the whole statement). That to me means looks like a single mistake in a whole test. But on page 17, Rasch starts talking about the Poisson Distribution, and the "number of words" (op. cit. page 17) in a test. Suddenly θvi has become (without any explicit acknowledgement of the change) the probability of making a mistake on a specific word in a test, and l, the expected frequency of errors in a text comprising n words is being defined as:

lvi = niδiv (18)

Next the Greek letter τ is being introduced as "the impediment of the text" (op. cit. page 17 my italics), and whereas previously δi had been the "test factor" (op. cit. page 17 my italics), now we are being told:

τi = niδi (19)

which certainly seems to confirm that δi has been relegated to represent the difficulty of an individual word, although this is not explained explicitly, and it still carries the subscript i for the whole test, and it is not at all clear whether all words carry the same probability of error, or as would seem more likely, different ones. Nevertheless Rasch proceeds to substitute 19 into 18 thus:

lvi = τiv (20)

Rasch verbalises this by saying lvi is the quotient of test impediment and ability. He calls the lvi "the parameter ... of the Poisson law - the mean number of misreadings". I would have called it the expected number of misreadings, but the fact remains that Rasch has put his parameters into something which can be estimated with reasonable accuracy (if the reading test is reasonably long). I wonder if I can produce anything similar with my bean model? Let's pull out my equation 15:

θij = (ai + aj)/2n (15)

In this equation, the n is not the same as the n in the Rasch equation. It is just an arbitrary number of beans in an imaginary box. To avoid confusion I shall stipulate that the probabilities ai/n and aj/n are expressed as percentages, so equation 15 becomes:

θij = (ai + aj)/2 (21)

We can now use n as Rasch does and imagine a child sitting a test comprising n items of identical difficulty. The expected score (or as Rasch puts it, mean of many sittings) will then be:

lij = n(ai + aj)/2 (22)

I'm not sure what this does for me, but is corresponds to Rasch equation 20. Both equations are currently unsolvable, because there are two unknowns, but I think I know what is coming next.

Rasch considers a specific person, who records avi misreadings. There is plenty of scope for confusion here. Rasch uses v to indicate a person; I use a j. And I use ai and aj for my item and person parameters, and Rasch uses avi for the number of misreadings. That is fine, as long as the reader watches the subscripts closely - the letter a is meaningless here without it's subscript.

I am turning the page now and Rasch compounds the confusion by referring to a second test with the letter j. I can't do subscripts of subscripts so I shall just refer to any two tests as i1 and i2. So when Rasch has two texts read by the same person, I shall call the number of misreadings in the two texts avi1 and avi2. Now from Rasch equation 20:

ζv τi1/ avi1 (23)

ζv τi2/ avi2

Now we have 2 equations with three unknowns, which is not very helpful, so in my next blog I shall return to the text.

No comments: