Recently, work on the Southampton University Virtual Learning Environment Review has focussed on analysing general trends in the use of VLEs, and in how these trends have come about, by evaluating reviews undertaken by other universities in recent years.
These trends and statistics will provide an insight into how the market is changing, and may allow a simple satisfaction rating to be determined. Reviews at other institutions will help to inform decisions made in this review, and may guide some of the steps.
The investigations uncover opposing trends in the sector, and opposing approaches to reviews. Some show universities adapting and selecting their VLEs for learning, while others decide that the best approach is to get more out of the current system, learning to adapt to the software and use it as it was designed.
Usage Trends
A spreadsheet was compiled of the Virtual Learning Environment used at a range of different institutions. The VLE used in 2010, 2012, and 2015 was listed, so that the change over time could be assessed. Once the data was compiled, a level of statistical analysis and data representation was applied. The full report is here.
Firstly, out of the 65 universities surveyed, the proportions using each different make of VLE over the period was represented, along with the corresponding trends in usage of each system.
The graphs show a general trend away from Blackboard, with a growing market share for Moodle. Other systems hold generally constant, with only a slight trend away to Moodle. Over the five year period, Moodle is the only VLE to have grown in popularity. This shows a tendency for universities to pick a system designed for a slightly different form of learning. It is also a general trend towards open source software which is more flexible – institutions are keen to adapt the system to tailor their learning.
The general trend also sees more universities moving away from Blackboard than Moodle – in fact, no universities migrated away from Moodle at all. This is interesting, and a statistical significance test demonstrates that there are fewer reasons to move away from Moodle than Blackboard.
Seven of the 39 universities using Blackboard in 2010 migrated, in addition to three of eight not using Blackboard. Using this, we can construct a null hypothesis that there is a 21% (10/47) chance of a university changing its VLE in the space of five years.
Now applying this hypothesis to the universities which have maintained Moodle throughout – 16 in total – the chance that none have migrated away from Moodle in the five years is 2%. There is therefore evidence at the 5% level that the null hypothesis can be rejected, in favour of a hypothesis which suggests there are fewer reasons to move away from Moodle than from other providers.
If we were to assume that these figures are exactly on the 5% level of significance, to allow comparison between satisfaction ratings, it emerges that in a five year period, it is equally likely for 30% of universities to migrate from Blackboard as it is for 17% to move away from Moodle. It could therefore be suggested that Blackboard is more likely to be replaced as a VLE than Moodle.
Trends due to Situations
The number of students at a university could impact the choice of VLE. In particular, some systems may be better designed to handle large numbers of courses and deal with large numbers of students submitting and storing work online. To test this theory, the mean and variance in the number of students at universities using each VLE was calculated. Calculating these parameters allows normal distributions to be generated, showing the likelihood that a university of a given number of students uses a given VLE. In effect, this shows the most popular VLE given a university of a specified size.
The graph demonstrates that for a wide range of attendances, Blackboard is preferred. Moodle is third-most popular for most of the distribution, as it also lags behind bespoke systems, which are typically implemented at small institutions. Canvas is also considered here, with data from the USA being used to form the distribution.
The reason for this trend could be a general tendency for larger universities to favour a managed, supported system in Blackboard and Canvas, while smaller universities are able to tailor their requirements more effectively using an open source and customisable system.
It is also conceivable that fewer large universities have migrated away from Blackboard, because of the technical challenge required to move all of the courses onto a new system. More students means more courses, and this takes extra time and therefore cost to set up on a new system. Smaller universities are able to adapt the system for their learning, but larger universities must learn to adapt to the proprietary software.
The trend due to university group was also investigated. It was suggested that Russel Group universities may tend towards a different system to University Alliance group members. This was evaluated using a grouped chart:
No signifiant trend was seen in this grouping, although this gave further weight to the theory that university size had an effect. A large bias towards Moodle is seen in the 1994 group, composed of the smallest universities. The largest universities are seen in the University Alliance and Russel Groups, which both favour Blackboard. Canvas, only seen here in the USA, is a middle ground.
Other Reviews
A range of other reviews were collected, summarised, and analysed, in order to determine the way that other universities had gone about choosing their VLE. Each university studied had approached the problem in a slightly different way. This drew some important points to consider when conducting the Southampton Review.
There are a range of parameters that need to be considered in conducting a review, and the way in which the review is conducted will be influenced by the specification and scenario. In particular, reviews can be conducted by setting up either a specification or test cases to evaluate the effectiveness of a system. The systems can then be piloted, or user satisfaction surveyed, in order to respond to these specification points and gather sufficient data to rate the effectiveness of each system against the requirements.
Also important to consider are the costs and the hidden costs. For example, the cost of upgrading the system and the cost of maintaining the new system, in addition to any infrastructure changes that this may require. The cost of training new users of the system, and as a result losing time and momentum on existing projects, is not negligible.
Finally, it may be possible to improve user satisfaction simply by redesigning the content provided on the same VLE. The design of the content and the actual content provided has an impact on the ease of use and overall satisfaction in addition to the design and layout of the software used.
Summary
The outlook in the VLE market is changing. Blackboard is becoming less popular, in favour of Moodle, and Canvas is also breaking into the UK. To arrest the decline, Blackboard has released a major overhaul of its interface, focussing more strongly on the user satisfaction while using the system.
However, the fact remains – these systems are used for learning. Often, user satisfaction comes from being able to use the systems in a familiar way, without having to undergo extra training. It is perhaps the case that an element of ‘learning to adapt’ needs to be seen in the VLEs to help improve digital skills and allow students to get the most out of their experience, instead of an ‘adapt for learning’ philosophy where optimum content may not be delivered in an effective and efficient way.
There is much to consider as the review progresses over the next few weeks.
Alex McCormick 