Surveys, Specifications, and Statistics

Much of the information that will be used to determine the outcome of the VLE review will come directly from students. Data will be collected through surveys and direct interviews with focus groups, and will then be analysed to determine the general student feeling towards the VLE. The same process will be undertaken with academic staff as they are likely to have different views.

Collecting data via a survey is not as simple as just coming up with questions. In order to get effective responses to the survey and to draw significant conclusions, the survey needs to be designed with the intended responses in mind. It is also difficult to avoid inducing bias with the phrasing of the questions, and there is a balance to be had between being thorough and covering lots of information, and being brief enough to get plenty of responses.

Principle of the Review

The specification and surveys are used to put together a full picture of the current VLE status and inform the final outcome.

It has been determined that the review will focus on two main objectives, to make the outcome as clear as possible. Firstly, the review will determine criteria for a successful VLE, marking out the properties that the VLE should have in order to be most effective for learning and for teaching. Secondly, a range of possible VLEs will be evaluated against these criteria, including the current VLE, Blackboard. The degree to which Blackboard meets the specification is the benchmark; if other VLEs can beat this benchmark then a change can be considered.

Using this two-part structure allows at least some form of quantitative comparison between the alternatives, which will allow the systems to be ranked mathematically. This is important for removing bias – although human input is used to determine the ranking, the ranking is not specified by humans. This subtle difference is the fine line between a subjective decision and an objective decision. If the decision can be made objectively, it is more likely to be widely accepted, as opposed to a subjective decision where the reviewer can influence the result. An objective evaluation is more likely to determine the system that is the best for the most number of people.

Creating a Specification

The specification, or list of criteria, is part one of the review. Determining the specification points, and therefore coming up with a complete list of points on which the VLE will be judged, is something that can be done manually and subjectively. However, it needs to be sure to cover all of the possible points, because the importance of these points can then be established using the questionnaires.

The specification will include points such as the cost, which is easily measured and very objective, but also the ease of use, which is much harder to measure and is much more subjective. The importance of this point will therefore have to be evaluated in the questionnaire.

The draft specification is the initial set of criteria, and is created by the review team. Once the draft specification has been completed, it can be used as the basis of the questionnaire, which will aim to determine how important the subjective points are, and how well existing systems match these points.

Writing the Questions

An example of one of the survey questions in iSurvey

Writing effective questions means writing questions which have meaningful answers. In fact, it makes more sense to play a Jeopardy style game, where a relevant and objective statistic is given as the answer, and the question is created from this answer. The value of the statistic will later be determined from the range of responses. Creating questions in this way guarantees that every question can be used to produce an objective parameter.

For example, one of the specification points is that the VLE should be accessible in some form on mobile devices. To establish whether this is really important to the student group, we wish to be able to state that “x% of students would use the VLE on mobile devices, if the functionality was available”. To fill in the gap requires a set of answers: yes or no. So this leads to the question being written as something with a yes/no answer: and it becomes something along the lines of “Would you like to use the VLE on a mobile device?” From this, we can establish how important it is that a VLE is available on a mobile device, and present an objective, numerical answer.

This is one type of question. There is a second type of question that must also be considered – and this involves evaluating a VLE against a certain measure. It may be desirable to assess how useful a particular feature of a VLE is; the corresponding statistic would be “y% of students felt that this feature benefitted their studies, while z% had not used the feature”. It is important to include the second clause here, because if a feature has not been used it is not possible to tell if it is useful or not.

The answers to the question must therefore include at least ‘useful, not useful, never used’, and it would provide clearer definition if a range of usefulness was provided (‘very useful, useful, no benefit, unhelpful, very unhelpful, never used’). The question becomes: “How helpful did you find this feature to understanding your course?” Note that this is not “How useful did you find this feature”, or “How much did this benefit your studies”, as they are subjective questions and do not rank the feature clearly on how effective it is at improving learning in particular.

Analysing the Results

Once the questions have been written in the above way, with the intended responses linked to parameters that we wish to know, calculating the value of the parameter is left to the realm of statistics.

The survey will return a statistic based on the responses of a sample. Because we are not interviewing every single student at the university (known as the ‘population’), our results are only relevant to a ‘sample’, which is a small group of the population. The parameter that we wish to establish is a value which relates to the population, but because we only have data from the sample all that is known is a ‘statistic’. The statistic therefore needs to be converted into a parameter, and this requires knowledge of the sample, and some neat mathematics.

The sample will have returned a statistic, such as ‘72% of the sample would use the VLE on mobile devices’. In very simple terms, we could use this to estimate that 72% of the whole population would also use the VLE on mobile devices. However, it is possible that the sample consisted mostly of History students, who for some reason do not use the VLE on mobile devices. It does not matter what this reason is, or what the biased group is, but it can be corrected for.

Sampling

Stratified sampling: one sampling unit is taken from each stratum to form the sample, which contains proportional numbers from each group.

This means using a technique called stratified sampling. If History students make up 10% of the population, then even if they made up 70% of the statistic, their responses are only given a 10% weight in the final parameter. Other responses are given a greater weight, so that every faculty group has their relevant weight across the overall population. This makes the scaling process more accurate.

There will also be uncertainty in the value of the parameter, due to random fluctuations in the responses that cannot be ironed out. It is correct therefore to establish an uncertainty in the value, which is normally defined as 1 divided by the square root of n, the number of samples. In the case of 100 samples, this uncertainty is 10%. This is valid for the yes/no questions, where the response is binary and the result is a simple fraction.

However, for more complex questions, such as how useful the system is, the responses will follow a distribution. The nature of the distribution means it will have a mean and a variance – an average point, and an amount by which the value varies according to chance. The sample mean and variance both have these uncertainties in when scaled up to the population size, and as such stating an absolute value for the population can only be done with caution – the variance could be large.

Using the Data

Once the population parameters have been found, they can be used to fine-tune the VLE specification. In addition, the results will be used to evaluate the current VLE – Blackboard – against this specification and therefore set a benchmark that other VLEs can be compared against.

The methods described are essential in forming comparisons between the different systems in a way that is fair and unbiased to all. Measuring each against a well-defined, objectively written specification ensures that the review is rigorous and complete, and allows future re-evaluation if the review is conducted again in the future.

Adapting for Learning, Learning to Adapt

Recently, work on the Southampton University Virtual Learning Environment Review has focussed on analysing general trends in the use of VLEs, and in how these trends have come about, by evaluating reviews undertaken by other universities in recent years.

These trends and statistics will provide an insight into how the market is changing, and may allow a simple satisfaction rating to be determined. Reviews at other institutions will help to inform decisions made in this review, and may guide some of the steps.

The investigations uncover opposing trends in the sector, and opposing approaches to reviews. Some show universities adapting and selecting their VLEs for learning, while others decide that the best approach is to get more out of the current system, learning to adapt to the software and use it as it was designed.

Usage Trends

A spreadsheet was compiled of the Virtual Learning Environment used at a range of different institutions. The VLE used in 2010, 2012, and 2015 was listed, so that the change over time could be assessed. Once the data was compiled, a level of statistical analysis and data representation was applied. The full report is here.

Firstly, out of the 65 universities surveyed, the proportions using each different make of VLE over the period was represented, along with the corresponding trends in usage of each system.

Trends in VLE usage from 2010 to 2015

Number of universities using each system from 2010-15

The graphs show a general trend away from Blackboard, with a growing market share for Moodle. Other systems hold generally constant, with only a slight trend away to Moodle. Over the five year period, Moodle is the only VLE to have grown in popularity. This shows a tendency for universities to pick a system designed for a slightly different form of learning. It is also a general trend towards open source software which is more flexible – institutions are keen to adapt the system to tailor their learning.

The general trend also sees more universities moving away from Blackboard than Moodle – in fact, no universities migrated away from Moodle at all. This is interesting, and a statistical significance test demonstrates that there are fewer reasons to move away from Moodle than Blackboard.

Seven of the 39 universities using Blackboard in 2010 migrated, in addition to three of eight not using Blackboard. Using this, we can construct a null hypothesis that there is a 21% (10/47) chance of a university changing its VLE in the space of five years.

Now applying this hypothesis to the universities which have maintained Moodle throughout – 16 in total – the chance that none have migrated away from Moodle in the five years is 2%. There is therefore evidence at the 5% level that the null hypothesis can be rejected, in favour of a hypothesis which suggests there are fewer reasons to move away from Moodle than from other providers.

If we were to assume that these figures are exactly on the 5% level of significance, to allow comparison between satisfaction ratings, it emerges that in a five year period, it is equally likely for 30% of universities to migrate from Blackboard as it is for 17% to move away from Moodle. It could therefore be suggested that Blackboard is more likely to be replaced as a VLE than Moodle.

Trends due to Situations

The number of students at a university could impact the choice of VLE. In particular, some systems may be better designed to handle large numbers of courses and deal with large numbers of students submitting and storing work online. To test this theory, the mean and variance in the number of students at universities using each VLE was calculated. Calculating these parameters allows normal distributions to be generated, showing the likelihood that a university of a given number of students uses a given VLE. In effect, this shows the most popular VLE given a university of a specified size.

Popularity of VLEs against number of students at university

The graph demonstrates that for a wide range of attendances, Blackboard is preferred. Moodle is third-most popular for most of the distribution, as it also lags behind bespoke systems, which are typically implemented at small institutions. Canvas is also considered here, with data from the USA being used to form the distribution.

The reason for this trend could be a general tendency for larger universities to favour a managed, supported system in Blackboard and Canvas, while smaller universities are able to tailor their requirements more effectively using an open source and customisable system.

It is also conceivable that fewer large universities have migrated away from Blackboard, because of the technical challenge required to move all of the courses onto a new system. More students means more courses, and this takes extra time and therefore cost to set up on a new system. Smaller universities are able to adapt the system for their learning, but larger universities must learn to adapt to the proprietary software.

The trend due to university group was also investigated. It was suggested that Russel Group universities may tend towards a different system to University Alliance group members. This was evaluated using a grouped chart:

University VLE usage, grouped by University group

No signifiant trend was seen in this grouping, although this gave further weight to the theory that university size had an effect. A large bias towards Moodle is seen in the 1994 group, composed of the smallest universities. The largest universities are seen in the University Alliance and Russel Groups, which both favour Blackboard. Canvas, only seen here in the USA, is a middle ground.

Other Reviews

A range of other reviews were collected, summarised, and analysed, in order to determine the way that other universities had gone about choosing their VLE. Each university studied had approached the problem in a slightly different way. This drew some important points to consider when conducting the Southampton Review.

There are a range of parameters that need to be considered in conducting a review, and the way in which the review is conducted will be influenced by the specification and scenario. In particular, reviews can be conducted by setting up either a specification or test cases to evaluate the effectiveness of a system. The systems can then be piloted, or user satisfaction surveyed, in order to respond to these specification points and gather sufficient data to rate the effectiveness of each system against the requirements.

Also important to consider are the costs and the hidden costs. For example, the cost of upgrading the system and the cost of maintaining the new system, in addition to any infrastructure changes that this may require. The cost of training new users of the system, and as a result losing time and momentum on existing projects, is not negligible.

Finally, it may be possible to improve user satisfaction simply by redesigning the content provided on the same VLE. The design of the content and the actual content provided has an impact on the ease of use and overall satisfaction in addition to the design and layout of the software used.

Summary

The outlook in the VLE market is changing. Blackboard is becoming less popular, in favour of Moodle, and Canvas is also breaking into the UK. To arrest the decline, Blackboard has released a major overhaul of its interface, focussing more strongly on the user satisfaction while using the system.

However, the fact remains – these systems are used for learning. Often, user satisfaction comes from being able to use the systems in a familiar way, without having to undergo extra training. It is perhaps the case that an element of ‘learning to adapt’ needs to be seen in the VLEs to help improve digital skills and allow students to get the most out of their experience, instead of an ‘adapt for learning’ philosophy where optimum content may not be delivered in an effective and efficient way.

There is much to consider as the review progresses over the next few weeks.

Presenting Challenges Presents Challenges

Following a meeting with a group of other interns within the Iliad at Southampton University, where we discussed a range of different presentation tools, it’s time for another reflection on how to present the outcome of this review.

It’s not easy to make an effective presentation, despite the range of tools available. The software isn’t everything, and the challenge of engaging an audience remains. When presenting the challenges overcome for a particular project, or detailing the targets for the next one, the presentation of information to an audience is key.

What’s Out There?

The interactive session, delivered by Fiona Harvey (@FionaJHarvey) covers all sorts of ways that presentations, discussions, and announcements can be made far more interesting through the use of software. Starting with some pretty basic tools, and then covering increasingly complex and capable pieces of software, and supplemented with demonstrations, seven different presentation methods were covered.

• ThingLink – a resource for making images interactive. Hotspots are added to any old background image, which when hovered over, display links and text to add further context. It’s a great way of adding extra detail to an image without covering parts of it up, and a quick and easy way of labelling a picture without delving into complex editing software.
• Sway – a new Microsoft Office app which tells a story. It’s a bit like PowerPoint, but it’s not slide based. Instead, images flash across the screen, accompanied by text, embedded tweets, videos, and interactive charts. Being part of the new Office suite, it’s cloud based and can be shared simply by copying a link, but it can also be embedded within pages.
• Adobe Slate – the adobe offering to compete with Sway. Based as an IoS app, it allows a gallery to be accompanied by captions, comments, and animations to make the transition smoother. It can be used as the background to an on-stage presentation, but is far more effective at demonstrating things to small groups on a mobile device.
• Emaze – purpose-build presentation software. It’s slide based, but shows the slides in a totally different way to PowerPoint. Instead of sliding through a screen, they are displayed on walls in a gallery to be viewed as the audience walks through, or passed down by the Hands of God. Something novel, and fresh, but with all of the content formatting tools necessary to present information.
• Prezi – everyone’s favourite alternative to PowerPoint. The classic open canvas, with slides of all sizes and orientations swept out by the flying camera, is ideal for infographics and presentations with a bit of structure. ‘Seeing the bigger picture’ comes to mind, which is probably exactly what it was concieved for.
• HaikuDeck – a tool for creating dynamic, animated presentations very quickly. It has a resource of stock images, which accompany any text added to the slides. Aside from helping you spot the unfortunate double-entendres, this gives the presentation a professional backdrop. If only the animations, transitions, and themes gave the same feel.
• VideoScribe – this draws the presentation on-screen in real time. Like all of those marketing campaigns where the on-scree text is written out by a hand as it goes, images and text are coloured in to produce a video to accompany this presentation. It’s slow to present the content, but does make it memorable. A resource of images makes the presentation very quick to put together. It is designed to tell a story, so the open canvas has a flow to it – something not obvious at first but very subtle and powerfull if used correctly.
• PowToon – another video creator… but this time it’s free. Stock animations, pictures, and characters accompany customised text to get the point across with added emphasis. In addition, music accompanies the video, generating an immersive presentation experience. The best stuff is available in the paid version, but it is easy enough to get started for free.

Making Effective Presentations

It’s not all about the software, unfortunately. The content is still key, and it still has to be interesting and engaging otherwise no amount of furious animated drawing is going to convince an audience. In some cases the software can paper over cracks, but in others it is makes them glaringly obvious.

The presentation software also needs to be used appropriately. In a formal setting, it needs to be formal. In an upbeat setting, the music can’t be from the Titanic. And so it goes on – choosing the right tool is vital.

This means there is no clear best option. When a presentation has been concieved, the information collected, and the structure decided upon, the best tool for the job can be chosen. The trouble with the software available is that it tends to force the structure of the presentation too much, getting in the way of a clear, well presented delivery. If the presentation is designed to have distinct sections with different themes, but they merge together, there is a lack of clarity in the content being presented, which makes the presentation less effective than it should be.

Once the structure has been determined, there is almost certainly something available which can accentuate the structure, bring it out, and make the content the focus, while maintaining interest and keeping the audience engaged. This is the key to a good presentation. In summary, make sure the tool isn’t driving the presentation.

What’s Changed?

A few years ago, even animating the text entry on a PowerPoint made it look unprofessional. Viewers would tut quietly as a title came in letter by letter – but that’s exactly the effect generated in many of the above options.

Nowadays, PowerPoint is seen as the old-age approach. In the days of Web 2.0, software as a service is a popular, and growing, concept. The explosion of available software has also provided many different alternatives, and in a generation where standing out from the crowd is so important, PowerPoint has had its day. To an extent, anyway.

There are some times where the data just has to be presented, and PowerPoint is still number one for interacting with other office products, which makes it invaluable for linking data to spreadsheets and showing statistics. However, increasingly, making something seem a bit different, new, and otherwise unusual is a great way to make it memorable.

The animations, transitions, and new themes available in these different presentation media covers that exact specification. New, different, and a little bit quirky – all designed to grab an audience.

Impress.js: a html presentation tool.

But if PowerPoint got boring, so might these. There will be a time when everyone has seen a PowToon, everyone has made a Sway, and they are no longer new, or different. Perhaps things will take another step forward, maybe to HTML based presentations like impress.js. Or perhaps PowerPoint will come to the fore again, seeming new and different because nobody has seen one in the last five years.

It’s a tricky one to stay on top of, keeping up to date and relevant, choosing the right tool for the job, and ensuring that something new, fresh, and previously unseen comes in to everything. It’s well worth it though, because presenting information is the interface through which people know you, and know your brand, and your business. Getting the information across is critical to ensuring the success of whatever is being presented.

Information – Present and Correct

In recent years, the development of the Web 2.0 principle has led to an explosion of software available online to do just about anything you could possibly want it to. Simple websites providing simple functions hosted entirely in the cloud, normally completely free – what’s not to like?

An example of Web 2.0: Google.

Having the functions available is all very well, but knowing what to use for any specific task is a skill which makes the difference between a very poor result and an exceptional one. Recently, I have been looking into ways of presenting information (not data*) which has been processed from the results of our VLE surveys, and finding a range of tools on the web to do it.

What is to like?

A basic web search for creating presentation tools brings up eight online services in the top 10 search results, and all of them have at least a basic version available free. We all like free stuff, and we like finding it quickly – getting started on a presentation online takes less than a minute.

With a huge range of tools out there, there must also be something which does everything you want it to, and which can create a presentation exactly how you want it to look – whether you want to export it into HTML, download an image, or combine the two and make an infographic in Canva to use as the background for a Prezi. Whoa – fancy!

HaikuDeck

These tools are great to use, too. The market is so competitive that they have to be comfortable to use – familiar from the outset, fast, and designed with the user in mind. There are none of the old standards which come with desktop programs, gradually migrating from one version to another and trying to keep an archaic interface ticking over.

The ease of having this functionality available online, free, 24/7 means that some of the traditional boundaries to collaboration are broken down – compatibility and access. One of the principles behind web 2.0 is ‘Software as a Service’ – the software is provided as a service online, which is accessed whenever it is required. When not required, it goes away, unlike desktop programs and apps which stay behind on your computer. It can be updated whenever the owner wants, because that just means pushing new code to a server somewhere on the Internet – much easier than trying to get several million users to download an update to the program they currently use locally on their computer. Everyone uses the same version of the software, and that means there are no compatibility issues, as you might get with PowerPoint 2007 and 2013.

If someone has access to the World Wide Web (not the Internet*), regardless of where they are in the world, they can use the software. This means collaboration can take place across the world, and it doesn’t need any specialist software installed on machines so internet cafés do the job just as well as personal laptops. Having information stored in the cloud also means that the latest version is always the one being worked on – no more trying to merge changes when you mistakenly open the copy you saved last week.

So what’s not to like?

Each of these tools is excellent at what it does. They all know exactly what they are trying to achieve, and execute it very well. The problem is, this is usually of limited scope. For example, if you want to animate the arrival of text on a page, you’ll find that isn’t covered by Haiku Deck, or Canva, and pretty soon you’re going to be wishing you’d just fired up PowerPoint anyway.

If you’re like me and loathe to give away any sort of passwords or personal information, these sites will not be for you. Each one will require a login and a password, and since you only found them through a search and spent all of half a second scanning the description, giving away an e-mail address and password seems premature. That’s not to say for one minute that these sites aren’t secure – I’m sure they are – but be careful. The more places you give out an e-mail address, the more likely you are to recieve spam.

The biggest gripe…

Despite this, by far my biggest problem with these sites is the effect they have on the presentation you are trying to give. Too often, the train of thought runs, “I need to create a presentation on this year’s sales figures. Prezi’s cool. I’ll use that.” The result: a presentation which takes longer than it should to create, longer than it should to deliver, and doesn’t get across the key information in a logical manner. You should be presenting figures – the information needs to be clear, the presentation style can be entirely forgettable.

Similar things can occur when trying to deliver a lecture, with plenty of information available, using Haiku Deck. Sure, it looks pretty, but one line of text per slide along with a vaguely associated image is not enough to deliver a powerful message. Great for showing off a gallery of photos, rubbish for teaching anything meaningful.

These are just examples, and it is of course very easy to create a bad presentation in PowerPoint, but be aware of the basic point – use the right tool for the job. Decide what you want the presentation to look like, and then use the luxury of Web 2.0 and the huge range of available software to create it, rather than letting the tool dictate the way the presentation flows, and the content you put in it.

What will we use?

During the course of my research, I have come across a range of sites which provide a useful functionality for presenting our information. Once the survey data comes in, it will be analysed and processed to turn it in to information we can present and draw conclusions from.

While the overall review will have to be summarised in a word processed report, for distribution and maintenance of records, we will also be trying to come up with innovative ways to get our conclusions across to the academics and students who will make most use of it. This means we are looking into good ways to present information clearly, and most importantly visually.

An infographic created using Canva – using mock data.

As a result, infographics have come up a lot as a good way of getting the information across. There are a number of online sites which can be used for creating the graphic. The first result that appears in a search is VisMe, but a limited number of stock templates, limited adjustability, and fairly poor user interface, mean it is difficult to get the desired result.

Instead, Canva has far more customisation tools and adjustability, and the ability to download the resulting graphic as an image file to be imported elsewhere. All of the functionality is free and only premium templates and graphics are charged for – it is easy enough to generate a well designed graphic without spending a penny.

When it comes to a tool to present the infographic, this is where Prezi comes in to its own. Being able to zoom in on a particular statistic, talk about it, then zoom out again and show the whole poster to see how things all fit together, is a fantastic way to show off a large amount of connected research and statistics. I would look no further.

Example of an infographic used in Prezi

We can also distribute the information and conclusions through the World Wide Web. This was initially designed specifically for the purpose of linking together a collection of multimedia files – exactly what we intend to do. Building a website through HTML is therefore a very feasible way of connecting all of our work together. Best of all, it can be accessed on a range of mobile devices, increasing its catchment.

But hold on – before going diving into a text editor and writing the html from scratch, there are yet more sites which will let you create a presentation-style set of pages, and export into fancy HTML5 format. These range from very professional systems to hobbyists writing code for Github in their spare time. One of the most talked about is impress.js, which is essentially a javascript library which renders html pages in an infinite canvas. It requires reasonably advanced html knowledge to get the most out of, but more than pays that back.

Graph generated with the Google graphing libraries. Data not representative.

Presentation of information in an attractive format does not come as naturally to html as it does to other tools. However, the beauty of open source is that if something can be made better, somebody will write a library that does it. Step forward Google, who have created a library for creating interactive graphs of every type under the sun. An example of what is possible is shown in image format on the left; it can be found in interactive form at this demo page.

In Conclusion…

There are a lot of ways to present information. Collecting it is only part of the puzzle; there is far more that needs to be done to get the point across in a succinct and memorable manner. Thankfully, there are plenty of tools available to do the job… but they have to be used correctly in order to be effective.

We will be using a combination of tools selected because we have chosen how we want to present things, and what we want to look like, and then chosen the tool which completes that task in the simplest way possible. When the presentation is done – keep an eye out for updates.

Footnote

*Data and information: data is raw, unprocessed numbers. Presenting someone with data, e.g. 26.33, will mean absolutely nothing to them. Data is processed and given context to convert it into information, which is meaningful and understandable. In our case, data would be the numbers of respondents in each category – information is the percentage these numbers represent, linked to the categories, so that a trend in the answer can be established.

The Internet refers to hardware. It is the cables, servers and routers which provide connections between all computers connected to it. The World Wide Web is purely data – it is binary data which is transferred over the internet and rendered on web browsers to produce information which can be viewed. The data is in the form of multimedia files which are linked together using hyperlinks, generating a software network on top of the Internet, which is a hardware network. This distinction means very little in common language but is important when considering the availability of software and data.

If It Ain’t Broke, Don’t Fix It

Over the last couple of days I have been working on a report on the current online learning tools provided at the University of Southampton. They are numerous: we have a ‘Sussed’ portal, which acts as a home page with links out to all of the different services, online timetabling, a Blackboard Virtual Learning Environment, a link to the Office365 logon, and a student records system. We also have two different common mobile apps. The large number of services available could in itself be an inconvenience, but more on that later.

The purpose of the report is to discover what we have available for now, the features it provides, and the quality of the interface to the students. This will clearly show areas which are not being used to their full potential at the moment, as well as areas which are not currently suitable for the task. We can move from here to investigate why services are not being used to their full potential, and also to see what services are required and work out how to provide them.

A view of the Sussed homepage

Largely, the report is based on my experiences using the system, and as such I spent most of the time to produce the report looking through my own portal, testing different links and exploring features I didn’t know existed before. Each service (Blackboard, Timetabling, etc.) is looked at individually and the advantages and shortfalls are listed, described, and explained. There is no time or desire at present to discuss how the system could be changed or improved, as there is far more data gathering to do before this becomes relevant.

Blackboard

Blackboard, the VLE, has a number of features which I had not used previously as there had been no reason to use them. It is a more powerful tool than I thought – it is not just a file storing site, because there are blogs, wikis, forums (discussions) and even a full calendar feature. However, as I also discovered, these are not very well used.

Distribution of content types found in a sample Blackboard course set

To analyse the content types found and used on Blackboard, I counted up all of the files found in my directories and looked at the different file types present. As expected, nearly half were PDF files, and it would have been much more if I had not counted videos of recorded lectures as well. The biggest surprise was to find that only 6% of the file types on Blackboard are web technologies and not files. All of these are discussion boards and quizzes – there are no blogs or wikis used in any of my courses. The distribution is seen in the chart here.

The reason Virtual Learning Environments are used over simple file storage sites is because they provide so many more functions and ways for academic staff to interact with the students. But here we see that for 94% of the content on Blackboard, a file storage site may even be optimal, because it would allow users to keep their libraries in sync with the latest version of a file, access documents in the cloud, and share files with specific groups of students or links rather than the privacy being limited to course-wide folders.

Mobile Apps

There are two mobile apps associated with the university that are widely used amongst students at Southampton. Firstly, the MySouthampton app is used for viewing timetables, finding free computers, and library search functions like DelphiS. It is the method of choice for many students for viewing their timetables because the timetabling service online is much more complex to understand. The official Blackboard app, Blackboard Learn, is also used to access files on mobile devices. This comes in to its own when viewing lecture slides on mobile devices in lectures. Students can make notes directly onto the slides using specialised apps specifically for this purpose, or they can simply follow along if the lecturer moves too quickly or too slowly.

These mobile apps are widely used and have by far the best interface of any of the online tools provided. Both provide cut-down versions of the full online alternative and this allows the design to be simpler and easier to use. App design standards are also more focussed on usability where web technologies are not forced to be easy to use, so there are several reasons that the students prefer to use mobile apps over the online alternatives.

Review Points

The report summarises the above points and more into a list of areas on which more research will be focussed. These are areas where the current system is either ineffective or used inefficiently, or where improvements can be made which would improve the service provided by the tools.

• Student engagement in blogs and wikis – they are useful features that are currently unused.
• Content organisation in Blackboard folders – files are often difficult to find wthin the folder structures on Blacboard.
• Integration of content into one dedicated system – specifically the timetables, which are currently in their own system, and would be more easily viewed if they could be exported in a format that could be read by software such as Outlook and iCal.
• Second sign-ins – half of the links available through the Sussed portal require a second input of user credentials. Bringing these services under the first login would be beneficial.
• Mobile devices – can and should more features be added to mobile apps to improve content uptake among students.

Outcomes

If it ain’t broke, don’t fix it: if we cannot find good reason to do away with the current system, regardless of the alternatives, then it should not be replaced. This report has established that while the current services are very capable of providing what is required from a Virtual Learning Environment, many of them are not provided efficiently or in a way that is user friendly and convenient. This means the focus of the review can be on improving user integration with the systems – which typically provide far more functionality than is actually required – than on improving the range of features that is available.

There will be further research into the VLE user groups to discover where it is possible to improve the user interaction and to identify how best to proceed with the recommendations.

First Day at the Office

VLE Review

Welcome to my blog. In June 2015 I was selected for a position on the Southampton Excel Placement Scheme, as a Virtual Learning Environment Research Assistant.

This blog was created to keep a record of the work I do on a multitude of projects, starting with research to assist the review of the Southampton University Virtual Learning Environment. Over the next few months a record will be kept of interesting pieces of information, discoveries, and pieces of work that are done to record the process of reviewing and improving the VLE.

Existing Systems

Today was the first day at the office and a lot of setting up has been done, including Portfolios, twitter accounts and blogs. Research into existing systems has thrown up interesting pieces of information and these have been saved in the developing portfolio on Pathbrite, VLE Review, along with other useful resources.

Alongside our current VLE Blackboard, three further options are available in Lore, Moodle, and Canvas, as purpose built VLEs. However, there is also an outside option in SharePoint, which is essentially a file storage system for business, but with enough additional features that using it as a VLE is possible.

Canvas prides itself on ease of use, directly competing with Blackboard. It has been implemented by Birmingham University with some success. Moodle is a free and open source tool which is scalable and therefore covers a huge range of businesses. It approaches the task from a much more social point of view, suiting the core culture of equality provided by the web.

There is much to consider and the differences between the systems are significant, so a more thorough review of all available options will be required at a later date.