3. A Large-Scale Classroom Experiment

The on-going evaluation of PAT and the PUMP curriculum is a kind of "design experiment" (Brown, 1992) on the effect of both instructional innovations in the unforgiving setting of real schools. Evolving versions of PAT have been tested in laboratory experiments following the cognitive tutoring design methodology (Koedinger & Anderson, 1996; Anderson, et al., 1995). However, the urban classroom situation is unlike the refined and controlled environment of the lab and laboratory standards cannot realistically be applied. As such, we have begun by addressing the practical question of whether the whole package, PUMP curriculum and PAT, is effective by comparing it against a traditional curriculum without PAT. By laboratory standards, this experiment confounds two variables, a change in curriculum and the use of PAT. However, our strategy is first to establish the success of the whole package and then, if indeed it is successful, to examine the effect of the curriculum and intelligent tutoring components independently.