Evaluation Overview

Formative Evaluation Strategies

1. Exploratory learning environment

  • Fractions Lab developed by iTalk2Learn including exploratory tasks and task-dependent support. Evaluation focused on:
    • User interface.
    • Impact of Fractions Lab on students conceptual understanding.

Evaluation strategies

  • Several iterations of evaluation studies.
  • Close collaboration with students and teachers.
Evaluation status

  • Four evaluation studies with a total of 155 students completed in the UK.
  • One study with 13 students completed in Germany.

2. Speech recognition for children

Evaluation strategies

  • Lab evaluation studies.
Evaluation status

  • First speech recognition system for English trained.
  • First speech recognition system for German is currently being trained.
  • Further studies planned after further training of the model.

3. Sequencing of structured tasks

  • Whizz and Fractions Tutor integrated in iTalk2Learn platform.
  • Sequencing of Whizz/Fractions Tutor tasks based on performance prediction.

Evaluation strategies

  • Lab experiments with Whizz dataset: analysis and development of a machine learning based sequencer.
  • Machine learning lab studies.
  • Field trial.
Evaluation status

  • Data collection for further sequencer development completed with different sequences of Fractions Tutor tasks with 113 students in Germany.
  • Field trial with the developed sequencer completed with 98 students using Whizz.

4. Amelioration of performance prediction by speech indicators


Evaluation strategies

  • Machine learning lab studies.
Evaluation status

  • Data features have been identified and feature analysis showed that the developed features can be used for affect recognition.
  • Preliminary model has been trained that can detect student affect.
  • Field trial with the iTalk2Learn platform planned for winter 2014/15.

5. Task-dependent and task-independent support

Evaluation strategies

Evaluation status

  • Five WoZ studies with a total of 38 students completed in the UK. In two of these studies the task-dependent support was already delivered automatically for parts of the tasks.
  • One WoZ study with 13 students study completed in Germany.
  • Further trials with automatic support planned for fall 2014 in the UK and in Germany.

6. Switching between exploratory tasks and structured tasks

  • Adaptive switching and sequencing algorithm developed by iTalk2Learn (intervention model).

Evaluation strategies

  • WoZ studies.
Evaluation status

  • First switching has taken place during the WoZ studies.
  • Further studies planned for fall 2014 in Germany and in the UK.

Want to know more?

Go back to project aims

Take a look at the iTalk2Learn work packages