Testing the iTalk2Learn platform
“Life is like a box of chocolates: You never know what you’re gonna get.”
This widely-loved phrase from the Forest Gump movie feels fitting in relation to testing a learning programme such as the iTalk2Learn platform. You never know which problem will occur at this stage of development, where all components are integrated into the platform for the first time. Thus, testing the iTalk2Learn platform is currently the task for the consortium.
Apart from intensive testing in recent weeks, we also scheduled scenario testing during our 9th general project meeting in Bochum.
In this scenario, all project partners, plus RUB’s student research assistants, played the role of students that interact with the iTalk2Learn platform. To keep the scenario as similar to the reality of school classes, RUB provided the same thirty laptops which will later be used for the summative evaluation studies in Germany. We additionally set up a local network with a server that was brought from the UK by our colleagues at BBK just for this occasion. By using a local network, we can work independently from the individual schools’ computer infrastructure and internet access.
We were all really excited about the second day of the general project meeting as it was the big day of testing. So each of us were sitting in front of a laptop, headphones on, talking with the iTalk2Learn programme and learning about fractions… when surprise, surprise, after 20 minutes of interaction with the system, it crashed!
However, thanks to our competent technical team we were able to identify the problem, resolve and fix it. Would it not have been a pity if we had not been able to finish our lesson about equivalent fractions? These are the usual hiccoughs one can expect when implementing such an innovative platform like iTalk2Learn for the first time. We have two months set aside just to test the system until we are confident that everything will run smoothly in schools.
Presentation at 9th General Meeting in Bochum
Advisory meeting with Prof. Aleven
The second highlight of our general meeting was an advisory meeting that we had been able to schedule with Prof. Vincent Aleven from Carnegie Mellon University Pittsburgh. He is a leading expert in the field of tutor development and thus has quite a wide range of experience in how to fine-tune our preparations for the upcoming summative evaluation studies. To leverage his expertise, we presented our intervention model and our plans for the summative evaluation.
Overall, Prof. Aleven sees a central innovation in the way we combine affect data with cognitive measures to adapt instruction to individual students. For example, we detect affect from students’ verbal and non-verbal utterances and use it as a basis for delivering emotional boosts and appropriately challenging next tasks. But this is not a one-way street: we also expect the instruction to have an impact on students’ affect.
Prof. Aleven pointed us at additional ways to measure how learning with the platform may enhance students’ affect and motivation. With room for exploration and affect support provided by the platform, this seems particularly interesting.
One further outcome of the advisory interaction between the iTalk2Learn team and Prof. Aleven was his advice about potential log data analyses. With these log data, we will be able to investigate what an individual student was doing at a certain point during his learning journey. He underlined the (future) need to be able to reinstate a past state of the system and replay it by making use of the logged data. We learned that this can even be used to generate new data that had not been logged before!
Finally, Prof. Aleven is very much looking forward to the finished platform which we will demonstrate publically at AIED in Madrid in June. We were very grateful for his precious feedback and look forward to any further advisory interaction with him!
