Research
Evidence-based research remains limited as the field of adaptive learning is still evolving.
One of the largest studies on commercially available adaptive courseware products was conducted by the Bill & Melinda Gates Foundation between 2013 - 2015. They started the Adaptive Learning Market Acceleration Program (ALMAP) to provide grants to higher education institutions to adopt adaptive learning technologies into their courses.
Researchers then looked at the effects on student outcomes and gathered data on instructor and student satisfaction. There were mixed reviews.
Student Outcomes
4 out of 15 courseware implementations resulted in slightly higher average course grades, but the majority had no discernible impact on grades.
Overall, in the 16 grantee-provided data sets provided for this study, the odds of successfully completing a course were not affected by the use of adaptive courseware.
Instructor and Student Satisfaction
The major concern expressed by instructors was getting students to use the adaptive courseware frequently enough.
Instructional practice data indicated only a modest reduction in the amount of time devoted to lecture/presentation in the blended adaptive course sections. The mean number of hours spent on instruction using blended adaptive learning was 6.9 hours per week, while the mean number of hours spent on face-to-face instruction was 7.3 hours per week. These results suggest that the use of adaptive learning was not as transformative in terms of instructional practice as it may have stated.
Instructors generally reported more satisfaction with the adaptive courseware than students did, and students gave mixed reports of how engaging they found the adaptive courseware, but generally they reported positive impacts on learning.
In another study, Liu et. al (2017) examined students’ learning in four content areas (biology, chemistry, math, and information literacy), their experience using the adaptive system, and compared student characteristics of those who engaged and did not engage in the intervention.
The adaptive courseware used in the study was programmed to include pre-determined learning pathways students would take depending on their diagnostic test. Researchers hypothesized that those who completed the learning paths of a content module would show a higher gain in their content knowledge than those who did not complete a learning path.
Example of part of diagnostic test from Brightspace LeaP adaptive system:
Pre- and Post-Tests
Researchers found that while the effect of the adaptive learning intervention on students’ learning in chemistry showed a statistically significant difference between students that completed at least one learning path compared to those that did not complete a learning path, there was no statistically significant difference in learning in math, biology or information literacy.
Student perceptions of their own learning
Results showed that certain design aspects of the modules had students feeling confused and frustrated. Several students were doubtful about the ability of the learning paths to effectively direct them to only the content they needed to review. Students also expressed frustration with not being clear on what content they missed or what content they should cover.
CONSIDERATIONS
Future research into blended learning technology implementation and efficacy in higher education is badly needed in a field awash in marketing claims. The success of an adaptive learning intervention depends on its design, and also the consideration of the whole student (both cognitive and non-cognitive abilities). Many pre-packaged adaptive learning technologies are one-off builds that don’t allow for content authoring. Therefore, user testing of a system is very important before committing to a platform.
Factors in student learning
.jpg)
While adaptive learning technologies are not meant to replace a teacher in the classroom, there are some important factors to consider for student learning:
Learner profiling: Most adaptive systems do not have robust learner profiling capabilities, especially for a broad base of learner profiles and demographics.
Content authoring: The content is pre-determined by the vendor. Teachers would like the ability to occasionally override the system’s recommendations, as they may know, for example, that a student can breeze through practice questions but then not be able to apply skills learned to a project.
Non-Cognitive Attributes: Learning as a process is more complicated than algorithmic reading or multiple choice responses. Students' non-cognitive skills are related to motivation, integrity, and interpersonal interaction, and are important in learning. As an example, the opportunities to engage in interpersonal interaction when operating within an adaptive technology platform (ATP) are rare (depending if the ATP is used as part of a blended learning program).
Use of the Mangahigh platform in the study of mathematics was researched by Tenorio et. al (2015) in Brazil. Mangahigh offers games of differing levels of difficulty, which should be used according to the cognitive abilities of each student. One of the aims of the software is to spark curiosity and engagement in the student when it comes to learning math, so they (and other game-based adaptive learning platforms) also attempt to take into consideration a student’s non-cognitive abilities such as motivation.
The study attempted to find a correlation between traditional learning methods and the use of Mangahigh games in teaching and an evaluation. They found that statistically, there was no correlation between the traditional learning assessment grades and score performance in the game or quiz.