Tuesday, October 20, 2009


I am crap at marketing. In a perfect world, I'd never have to sell anything. I'd just be. But the world is not perfect. And why waste time creating computer software if you can't sell it?

One of the reasons I began the quest to learn Java was that the version of my software written in VB6 is sitting on 600 CD's, which now gather dust in my living room. I spent a year writing to, calling and visiting schools, but the revenue generated was barely compensation for the installation time, let alone marketing, let alone development.

Out of 221 schools on the target list (essentially primary schools in the metropolitan area of Perth, Western Australia), 61 agreed to a meeting and a trial on at least one computer, and 16 actually paid for a full site license. The official price was $660, but I let some have it for half that, and I always threw in a day's worth of installation and training. I didn't charge for the two or three presale visits, and for the three to six month evaluation and decision making period, I had no income. After a year of this, and a negative income after operating costs, I pulled up stumps and let the software rot.

Was the product crap? Of course I am biased, but I don't think so. A group of schools in the remote Pilbara region used Federal Government grants to fly me out to install the software and train staff. A number of schools paid me to address "PD" staff training sessions. Two regional radio stations interviewed me on the software and what it was trying to do.

What went wrong? Essentially, I ran out of steam. For one person to develop, market, and support a product is to much to ask.

Successful "education" software is mainly produced by games companies, who have an existing marketing and support infrastructure. Even the software is produced mainly by games developers, and the curriculum content is traditionally minimal.

Latterly, some software has begun to appear on the web, with a slightly higher quality content. Mathletics is one. My daughters use it at school, and for a while they were keen to use it for homework. But their interest waned after a few weeks. Now they use my software at home just as often, perhaps to please me.

The bottom line is that designing software, which covers the curriculum, records progress, and is sufficiently interesting to captivate the attention of children for more than a few hours, is a task so enormous, so difficult, and so expensive, that neither the public nor the private sector has yet attempted to do it properly.

I freely admit that both my CD software and my web-based software only scratches at the surface of the primary school math curriculum. But I'm not going to sit at home and dedicate the rest of my life to reinventing the wheel if I can't get people to use what I have done already.

Selling my web-based software really shouldn't be that hard. It's free! But so far, offering it to just two schools to try out has been like trying to push shit up hill.

The first was the local school in the country town, where I now live. I have lived here, repairing computers, for five years, I have four children, and I sit on the local Council, so getting an appointment was not a problem. And the meeting went very well. I had been told that the new deputy principals, a husband and wife team, were dead keen on computers, and on their application in education. And sure enough, they understood what I was saying and were enthusiastic about what they saw. They said they would address the next staff meeting on the subject and get back to me.

Silence followed.

Many weeks later the same team were invited into a Council function. A few days before the function, I put in a phone call to ask what had happened, but the call was not returned. At the function itself, where I was in the role of host, rather than salesman, I sought them out again to ask what had happened. Unable to run away, or avoid me, I sensed awkwardness, even fear. After much vacillation and skirting around the subject, all I got out of them was that I would need to talk to the principal. So far I have not bothered. The application is not permanently hosted yet, and it would be premature to make a fuss. I called on them, because I had been told they would be interested. They were, but some invisible barrier rose up and prevented further progress.

The second was my daughters' primary school. Again, as a parent, I had easy access to an audience with the school principal, and just as with the first school, she purported to like what she saw. She even later wrote a polite note extolling the virtues of the software, but enclosed the CD, which she was returning to me. So I tried the guy who emails the school newsletter, and he passed me to the lady who runs the computers. I got the CD back to her, but after a couple of failed attempts to meet (my contact with the girls is infrequent and erratic), she was also wanting to return the CD, saying the school had access to "plenty of resources". I'll remember that the next time they write to parents asking for money.

Prior to contacting these schools, I had assumed they would be friendly, and suitable candidates for a pre-release trial of the software, tolerant of glitches, and willing to try subsequent editions. They have made it clear that they are not. And while ten years have passed since I was last knocking on doors, and awareness of technology has improved, teachers are as conservative as ever, entrenched in their daily routine, and deeply suspicious of anything unfamiliar.

Sunday, October 18, 2009

More on Scoring Rates

Following on from my last blog, the table below shows the raw scoring rates on two additional items, H and K. Item H is twice as hard as item I, and is only addressed by Student B. Item K is twice as easy as Item J and is only addressed by Student A. This scenario synthesises one which might be generated by a computer based adaptive arithmetic test, which presents more difficult items to more able students and easier items to less able students.

Raw Rates Item H Item I Item J Item K Session Mean
Student A
4 8 16 9.33
Student B 4 8 16
It Mean 4 6 12 16 9.33

From the table, the effect of the adaptive component of the computer based arithmetic test has been similar to that of a very good handicapper in a horse race. By presenting more difficult items to the more able student and easier items to the less able student, it has produced a dead heat in the result. An examiner looking at the raw rates might be misled into thinking that both students had the same ability. Hence the need to adjust the results to take into account the difficulty of the items presented to each student.

Similarly the adaptive component of the test has distorted the item mean scores of those items presented to only one student. Take Item H. The item mean scoring rate is shown as 4. However, had the item been presented to Student A, from the stated assumptions of the example, one might have expected the scoring rate to have been 2 capm, and the item mean scoring rate would then have been 3, not 4. In the case of Item K, the item mean scoring rate is shown as 16. From the stated assumptions of the example, had this item been presented to Student B, one might have expected the scoring rate to have been 32 capm, and the item mean scoring rate would then have been 24. Item H has been made to look relatively easier than it is, because it was only presented to the more able student, and Item K has been made to look relatively harder than it is, because it was only presented to the less able student.

The scoring rate quotients calculate out as follows:

Quotients Item H Item I Item J Item K Session Mean
Student A
0.43 0.86 1.71 1.00
Student B 0.43 0.86 1.71
It Mean 0.43 0.64 1.29 1.71

The session rates can then be adjusted, using the item quotients to calculate the adjusted rate. I am reversing the order here from my previous blog. In the previous blog I adjusted the item rates first, but in this example, it is the session rates which most clearly "need" adjusting, and the item rates in fact cannot be adjusted.

Adjusted Rates Item H Item I Item J Item K Session Mean
Student A
6.22 6.22 9.33 7.26
Student B 9.33 12.44 12.44
It Mean 9.33 9.33 9.33 9.33 9.33

The adjusted mean scoring rate for Student B is now higher than that for Student A, but not by a factor of 2. Just looking at the numbers for this example, it very clear that the data set is "incomplete". The "missing" data from Student A on item H and from Student B on Item K is distorting the results. The session quotient method of transforming the data offsets the distortion partially, but not completely.

And in this example, adjusting the item rates, using the session mean quotients, is not very useful, as the session means were identical, and the session mean quotients were all unity. It follows that iterations would not achieve much, because no matter how many times you divide by one, you move no further forward.

Friday, October 9, 2009

A closer look at scoring rates

In my blog of 25 August, I described some iterative transformations on scoring rate data from a computer based arithmetic test. I said I would report the results of further iterations, if I liked them, and from the time that has past it should be obvious to anyone reading this that I didn't.

The transformations were based on what I called the scoring rate quotient (SRQ). Essentially I divided the scoring rate for every item in every test session by the mean of all scoring rates for every item in every test session to produce the SRQ for individual session-item combinations and to calculate the mean SRQ for every session and for every item.

To illustrate, imagine two students, A and B, addressing two items, I and J. Imagine in this case that Student B scores at twice the rate of Student A and that Item I is twice as difficult as Item J. The raw scoring rates might look as follows:

Raw Rates Item I Item J Session Mean
Student A 4 8 6
Student B 8 16 12
Item Mean 6 12 9

The scoring rate quotients would then be as follows:

Quotients Item I Item J Session Mean
Student A 0.44 0.89 0.67
Student B 0.89 1.78 1.33
Item Mean 0.67 1.33 1.00

The session quotients can then be used to recalculate the item rates.

Adjusted Rates Item I Item J Session Mean
Student A 6 12 9
Student B 6 12 9
Item Mean 6 12 9

Or the item quotients can then be used to recalculate the session rates.

Adjusted Rates Item I Item J Session Mean
Student A 6 6 6
Student B 12 12 12
Item Mean 9 9 9

Expressing this algebraically, the means are calculated as follows:

Session Mean A = (RIA + RJA)/2 (1)

Session Mean B = (RIB + RJB)/2

Item Mean I = (RIA + RIB)/2

Item Mean J = (RJA + RJB)/2

Grand Mean A = (RIA + RJA + RIB + RJB)/4 (2)

The scoring rate quotients are then calculated by dividing the raw scoring rates by the grand mean:

SRQ IA = 4RIA/(RIA + RJA + RIB + RJB) (3)




The session mean quotients are then:

Session A Mean Quotient = 4(RIA + RJA)/2(RIA + RJA + RIB + RJB) (4)

= 2(RIA + RJA)/(RIA + RJA + RIB + RJB)

Session B Mean Quotient = 2(RIB + RJB)/(RIA + RJA + RIB + RJB)

The item mean quotients are:

Item I Mean Quotient = 2(RIA + RIB)/(RIA + RJA + RIB + RJB)

Item J Mean Quotient = 2(RJA + RJB)/(RIA + RJA + RIB + RJB)

And the grand mean of the quotients is:

Grand Mean SRQ = 4(RIA + RJA + RIB + RJB)/4(RIA + RJA + RIB + RJB) (5)

= 1

The adjusted item rates are calculated by dividing the raw item rates by the session mean quotients.

Adj Item IA = RIA(RIA + RJA + RIB + RJB)/2(RIA + RJA) (6)

Adj Item JA = RJA(RIA + RJA + RIB + RJB)/2(RIA + RJA)

Adj Item IB = RIB(RIA + RJA + RIB + RJB)/2(RIB + RJB)

Adj Item IB = RIB(RIA + RJA + RIB + RJB)/2(RIB + RJB)

now we have stipulated that item I is twice as hard as item J so:

RJA = 2RIA (7)
and RJB = 2RIB

So we can re-write expression 6 as:

Adj Item IA = RIA(RIA + 2RIA + RIB + 2RIB)/2(RIA + 2RIA)


= (3RIA + 3RIB)/6

= (RIA + RIB)/2 (8)

Adj Item IB = RIB(RIA + 2RIA + RIB + 2RIB)/2(RIB + 2RIB)


= (RIA + RIB)/2
so Adj Item IA = Adj Item IB

Thus the adjusted item rate for item I is identical for both sessions, and also equal to the item mean for item I. The same is true for item J.

Of course this is the special case envisaged by Rasch, where all items are completed by all students. It was nice to work through this special case, because, in my mind at least, it indicates that a single pass transformation is sufficient, and that there is no need for multiple iterations.

In my next blog, I shall have a closer look at the more general case where not all items are completed by all students.

Thursday, October 8, 2009

UK Car Hire - caveat emptor

The Web is a great boon to travellers. Where once you had to sit like a prune on a travel agents chair while they fiddled about for hours on a computer, now you can tailor make your own holiday from your own living room.

As well as the vendor sites, there are these web sites, which purport to search for and sort whatever you are looking for. At the top of the list are what purport to be the best bargains, and the unwary might simply click on these and look no further.

If you type UK car hire into Google, an array of sites like this one appear high on the list. It invites you to enter dates, a pickup location, and a car type, and then runs a search "of up to 40 companies" for you.

A couple of things should be born in mind when interpreting these results. One is that only 4 companies actually have representation at the airport. From memory these are Avis, Budget, Europcar, and Hertz. The others are scattered through West London, take ages to pick you up, and are hellish difficult to find when you need to drop the car off.

Another is that the "search" website quotes a very bare hire price. When I ran a search a couple of weeks ago, Thrifty quoted £322 for 9 days, which is approximately £ 36 a day - and very reasonable it seemed. They told me in advance that 2 child seats would add £5 a day each, bringing the rental to £46 a day - still quite reasonable. What they did not tell me that collision damage waiver, which used to be about £5 a day, was now an extra £18 a day. That, and a few other extras, brought the total price for the nine days to £649, which was more than double the original quote at nearly £76 a day. They also offered to sting me for even more if I wanted automatic transmission.

So while the "search" websites might be a useful first step to identify suitable car models, the wary traveller should then go to the four airport based companies and get detailed quotes including all extras, collision damage waiver, and automatic transmission, if required.