WP5: Data Collection and Evaluation
This work package has two main objectives:
- Formative evaluation activities aimed at informing the project about the current state of the platform.
- Summative evaluation activities (including two field trials) aimed at evaluating the effectiveness of the project.
More specifically, the formative evaluation goals are to:
- Compare the performance of intelligence-based statistical machine learning models to that of existing (rule-based) models of artificial intelligence in real-world applications.
- Seek and incorporate feedback from practitioners (i.e. teachers and schools) using expert evaluation methods such as cognitive walk-throughs.
- Work with Work Package 1, Work Package 2 and Work Package 3 to evaluate the following in an iterative design and test process:
- The cognitive models for elementary education and the pedagogical interventions (Work Package 1)
- The recommender systems for adaptive teaching and guidance mechanisms (Work Package 2)
- The infant speech recognition (Work Package 3).
The summative evaluation goals are to:
- Define appropriate testing methodologies (e.g. define appropriate test and control groups) and a common set of metrics (e.g. define dependent measures) to thoroughly assess the new technology in the context of two trials, using quantitative measures. The testing methodologies chosen will safeguard a methodologically and statistically sound evaluation of the project.
- Assess the intelligent learning support platform and its modules from Task 4.3 (T4.3) for problem selection, pedagogical intervention, exploratory vs. structured learning activity selection, and human-computer interaction. This will be done by running two distinct field trials. These two trials take place in two different countries, with two different languages and in two different learning environments. This is so that the validity of the developed cognitive models and the applicability of the adaptive tutoring achieved by our platform will be evaluated in depth and with a view towards exploitation.