Rasch-based estimation

The essence of Rasch-based estimation is that observed scores are adjusted to take into account other observations. So for example item scores may be adjusted to take into account the observed abilities of the candidates (who in my research are usually children). And the candidate or child scores may be adjusted to take into account the observed difficulties of the items. To explain this process I shall refer to the Winsteps documentation because the Winsteps software is the only computer based estimation tool I have used, and because its creators are recognised academically as being at the core of the objective measurement movement (to the extent that such a thing exists).

The formula given by them for adjusting item scores is:

d_i = m_i - root(1 + s_i²/2.9)ln(r_i/(n_i - r_i))

where d_i is the revised item difficulty, m_i is the mean ability of the children attempting item i and s_i is the standard deviation of those abilities, r_i is the observed raw score on item i, and n_i is the number of children.

In an earlier blog I used a few lines of Java to simulate the results from a test comprising 64 items sat by 64 children. Item difficulty and child ability were set by parameters in the model, and in the first pass I graduated item difficulty evenly over the 64 items and I set child ability to zero on the Rasch scale or a neutral probability of 50%. In the real world nobody will ever know or be able to measure the underlying ability of any child. The advantage of using a model like this is that we know both item difficulty and child ability in advance, and we can use this knowledge to appraise the performance a measurement tool or estimation methodology.

From my spreadsheet and yesterday's charts, m_i is -0.057 and the variance of the observed abilities (s_i²) is 0.034. The calculations here are made easier than envisaged in the formula because every child addressed every item, so the whole of the first section of the equation is a constant and the final bracket has already been calculated and charted. The middle section of the equation turns out to be:

root(1 + 0.03662/2.9) = root(1.01159) = 1.00578

This coefficient is applied to the final segment of the equation, which from the earlier blogs is the "transformed" raw score for the item. So we have:

d_i' = m_i - 1.00578d_i

where d_i' is the revised item difficulty and d_i is the raw item difficulty. That doesn't look right to me. Is there a bracket missing? Let's think about it for a minute. Let's ignore the coefficient for a minute, which is so close to unity it make very little difference. That leaves:

d_i' = m_i - d_i

Now lets think about a special case where the observed item probability is 50-50, such that the "transformed" item score is zero logits. Then:

d_i' = m_i

I guess I'm not a real mathematician because I need concrete examples to help me understand things. Harking back to the discussion in my blog The Meaning of Rasch, in I think the tenth paragraph on that page I said:

"Suppose a child box has been combined with an item box, and repetitive sampling produces red and blue beans in equal proportions. If the item is assumed to have neutral difficulty, so the red beans represent 50% of the total, it might be deduced from this that the child has neutral ability, and that the child box also contains 50% red beans. But if the item box is known to have 75% blue beans, and the combined box sampling indicated a 50-50 combination ratio, one might deduce that the child box contains a higher proportion of red beans (75% red if both boxes contain the same number of beans)."

This fits the special case equation above quite well, where the observed item difficulty d_i is zero, and the adjusted item difficulty d_i' equates with the mean ability m_i of the children addressing the item. In my beans model ability equates with the probability of pulling out a red bean from a child box and difficulty equates with the probability of pulling out a blue bean from the item box. If the child ability is 75%, there are 48 red beans in the child box, and if this equates with item difficulty, there are 48 blue beans in the item box. Combine these boxes together and you have 48 + 16, or 64 red beans and 16 + 48, or 64 blue beans, and this fits nicely with the observed results.

In conclusion therefore, although at first the formula looked a bit odd to me, I am now happy that I have read it correctly, and that there is indeed no missing bracket.

In my next blog I shall apply this formula to the data.