Wednesday, July 22, 2009

Reading Rasch

My blog will be (more) shambolic and disorganised (than usual) for the next few posts. I am reading a classic text to clarify my understanding of a topic which has been rattling around in my brain for years. I find it helps to make notes when I read a technical text, and the more technical the text, the more detailed the notes have to be.

I mentioned in my previous blog the sleight of hand by which the Rasch parameters (individual ability and item difficulty) can be estimated independently of one another. I have seen the formulae and I have read the text, but it does not sit comfortably with me. You cannot begin the estimation process for item difficulty without individuals, and vice versa.

I think some confusion arises because in much of the subsequent literature the term "logits" is used, whereas, in the original book Rasch emphasises that his parameters are pure numbers - they don't have units. If you try to create units (for whatever reason), in my opinion, you lose objectivity. It is only when you leave the parameters as pure numbers that you preserve objectivity.

So if you apply a set of 3 items to a population of students, the Rasch difficulty parameters (RDP) for those items may turn out to be 1, 2 and 5. If you apply the same items to a different population, I am not sure whether the RDP will be exactly the same, or in the same ratio. My gut feeling is the latter, but I'll firm up that view (or not) as I go through the original argument.

I'm on page 4 at the moment, and reading the claims, rather than the argument which proves them. A problem I have when reading something written fifty years ago, especially written by someone born a hundred years ago, is that the author sometimes finds exciting stuff which, in the context of today, I find less exciting. Exacerbating this, in an original text, which has subsequently been written about a lot, the author, making the case for the first time, emphasises stuff which the later reader takes for granted, or at least has read many times before.

For me, having enjoyed the movie Jurassic Park, what makes Rasch interesting is his use of a probabilistic model, as distinct from traditional deterministic models. Among my friends I describe Rasch Theory as Chaos Theory for Social Scientists. Rasch, on the other hand, in his introduction, seems most excited about the distinction between the "aggregates" approach of traditional statistical analysis and the (individual) parametric approach of his method. He is also fairly (and not unreasonably) excited about the elimination of one or other of his parameters from his equations.

On page 5, Rasch introduces his work with reading tests. He emphasises the importance of having access to results from two tests applied close together in time. He also collected data from two distinct student populations - a so called "normal" population, and a second dataset from students requiring remedial tuition in native language skills.

Still on page 5, Rasch introduces charts shown on the ensuing pages. I remember reading this for the first time and straining my brain for something new and interesting in the charts. But there was not. Contrary to the claim by Rasch on page 3, paragraph 3 (of the whole book) "statistical tools ... such as correlation coefficients ... have found no place in out investigations", these charts are essentially those shown by modern software when calculating repeated measures reliability coefficients. These are essentially correlation coefficients between the scores in two similar tests taken one after the other. Granted, that Rasch protests on page 5 that "we are not going to consider this as a case for correlation analysis [but rather] glance at the figures ... and look for the possibility of translating uniformly from the results in [the first test] to results in [the second test], but methinks that's protesting too much. Where would the harm be in calculating the correlation coefficient and then proceeding with the original analysis?

On page 10, Rasch explicitly discusses the merits and demerits of deterministic versus probabilistic models. I had forgotten that was in there and I am glad it is, because as I said above, it is what for me makes Rasch interesting. He gives Newton's laws as a classic example of the deterministic model, and the kinetic theory of gases and the rules of radioactive decay as examples of a probabilistic approach in physics. He then describes human behaviour as essentially random, and suggests probabilistic modelling is better suited to psychometrics than any deterministic approach.

I am also delighted to observe that Rasch illustrates his argument with a "ball-drawing game", randomly pulling red and white balls from a bag. I had completely forgotten that he did this, and reading it now makes me feel much more confident about continuing with the models I described in some of my earlier blogs. Admittedly he then jumps from balls to mice (perhaps because mice, being alive, have more in common with children), but I shall continue to use balls, because the theory works just as well with them.

No comments: