Implementing a learning analytics system across all higher education institutions in Wales provides a model for the opportunities for such services at a national scale.
In August 2018, the world's first national learning analytics service was launched in the UK. The service is offered by Jisc, provider of networking and IT services for UK education and research, and is available to all universities and colleges across England, Scotland, Wales, and Northern Ireland. By pooling resources for this service, institutions enjoy opportunities for sharing experiences and learning collaboratively in the emerging field of learning analytics.
Jisc's learning analytics service provides a cloud-based "learning data hub" for institutions to integrate data regarding students and their learning activities. This draws from sources such as the institutional learning management system, the library, and the student information system. Jisc's tools enable the provision of more timely and comprehensive information to students and staff. The aim is for learners to understand and take greater ownership of their learning while helping institutions tailor support for students and improve their courses.
But something special is happening in Wales, a nation with eight institutions offering higher education courses, where collaboration is being taken to another level. In most of the world, institutions are left to their own devices to implement learning analytics. In Wales, the funding council for the country's higher education system has given an important spur to this development by funding the rollout of Jisc's learning analytics service across the entire higher education sector.
Wales, with a population of 3.1 million, is a small country, but it provides a good case study for higher education institutions interested in the economies of scale and other benefits of being part of a national or regional learning analytics consortium. The Welsh nation's distinctive culture and the close relations between institutions means they can collaborate on a scale that is unusual elsewhere—this despite significant differences between the universities, such as a focus on research versus teaching and the types of students they recruit. At one end of the scale is Cardiff University, a Russell Group university—the UK's nearest equivalent to the Ivy League. At the other are institutions that are more likely to have students with issues that make academic life more challenging, such as some types of disability, low prior educational achievement, family responsibilities, and the necessity to work while studying.
Motivations
Several issues are driving the development of learning analytics in Welsh institutions. First, most of the institutions are facing a retention issue. While dropout rates in Wales are substantially lower than those in the United States, when students do withdraw from studying, it affects institutional finances and reputation, not to mention the life chances of the individuals concerned. This partially explains the Welsh Government's sponsorship of the program: graduates are likely to have better employment possibilities and associated advantages over nongraduates and are thus more likely to help build the national economy.
Officials at some of the institutions are interested in developing dashboards to highlight issues around student wellbeing and mental health. Although the incidence of mental health issues is reported to be lower among students than the general population, student support services in Wales are seeing increasing numbers of students with anxiety, depression, and other conditions.
There have been various high-profile cases of suicide among students, including that of Ben Murray across the border in England at Bristol University. His father subsequently worked with the institution to identify a range of data and information sources that he believes could have indicated Ben's mental state and potentially prompted an intervention before it was too late.
Interest is also growing in Wales for other potential uses of data on students and their learning, such as enhancing the curriculum. At Bangor University, a research project led by Dave Perkins is examining the student workload and its links to stress and performance. The project is developing a visualization tool to help model the amount of effort required to complete assessments and to aid tutors in identifying students who might be struggling.
A further aim at some institutions is to develop capacities in handling and understanding data and analytics in general. For some, this is seen as part of a longer-term trend to move toward a more evidence-based decision-making culture.
Different Stages of Development
Although most of the Welsh institutions are at an early stage in their implementation of learning analytics, some have already gained considerable practice using data to optimize the student experience. Aberystwyth University, for example, developed its own attendance-monitoring systems with RFID technologies—a system of electromagnetic tags and readers—and already has a good understanding of student attendance at lectures and how this correlates with student success.
The University of South Wales is probably the farthest down the road in its development of learning analytics across the institution. As an early adopter, it has been instrumental in influencing the development of the service and the tools provided.
Discovery Visits
Most of the institutions have taken up the offer of a "discovery visit," in which a team of consultants conducts two days of meetings with stakeholders, including senior management, academic representatives, student services, IT staff, librarians, and students. The aim is to assess institutional preparedness as well as to hear from staff, faculty, and student representatives about their plans and concerns.
Many of the participants are often unclear at this stage about what learning analytics involves, and these visits provide the opportunity to inform them about the benefits, the logistics, and the issues they will need to consider when developing their institutional projects. The consultants then follow up with a comprehensive report for senior management, including recommendations on how to progress the project.
Data Sources
A basic premise of using learning analytics to address retention issues is that student participation in learning activities tends to correlate with subsequent student success. Certain data sources are better proxies for engagement with learning than others; this varies according to subject area and the nature of the learning taking place.
Most learning analytics tools require, at a minimum, data from the learning management system (LMS) and the student information system. These data elements enable the identification of students who may be at academic risk due to low engagement with the LMS.
Where face-to-face teaching is involved—still a primary teaching method in Welsh higher education—attendance monitoring systems can supply important information about student engagement in learning. Jisc provides an app called Study Goal that enables students to check in to lectures and other events, as well as providing them with analytics on their learning and how their engagement compares with that of others in their cohort.
Assessment data are also clearly of interest: studies have shown that one of the best indicators of future student success is, unsurprisingly, assessments that contribute to the final grade for the course. Data other than grades that can be significant include the proximity of assessment submission to the deadline. Jisc is also working with Turnitin, the assessment submission and plagiarism-detection software in use at all Welsh universities, to examine how data from that system can be used.
Other data sources identified by Welsh institutions and currently being considered for inclusion by Jisc in the models include library management systems, equipment loans, and viewing of online lecture-capture videos. Whether these data will significantly enhance the models of student risk—and justify the cost of integration—is still under investigation.
Tools Provided
As well as the Study Goal app, Jisc also provides Data Explorer, which enables staff and faculty at different levels in the institution to perform analyses on student data, ranging from the identification of students at academic risk to the correlation of online activity and grades across a cohort.
These applications draw on data held in Jisc's cloud-based Learning Data Hub, an institutionally owned repository that gathers student and activity data from a range of systems in standard formats. As well as using the tools provided, the universities can extract data from the hub and perform analyses using their own business intelligence software.
Community
According to the institutions using it, one of the main benefits of deploying the Jisc learning analytics service is that it enables them to be part of a broader community and learn from the experiences of others. A learning analytics community is already well established in the UK, facilitated by Jisc, that has had sixteen meetings to date where participants hear from each other and from vendors about their experiences and the latest innovations. There is also an active UK learning analytics research group in which participants share experiences in areas such as curriculum analytics, student wellbeing analytics, and the evaluation of institutional learning analytics projects.
Welsh universities are working closely together on this initiative but also benefit from opportunities to network with institutions elsewhere in the UK that are facing similar issues such as meeting the needs of a highly diverse student body.
Conclusion
Some colleges and universities are farther ahead in their deployment of learning analytics than any of the Welsh institutions. However, this initiative, with its common technical architecture, data structures, and networking and collaboration opportunities provides a blueprint for other regional or national groupings of institutions wishing to work together to progress their use of learning analytics.
The input of experienced consultants and technical experts from Jisc in analyzing institutional capabilities and requirements has been a key first stage in the process. There is also plenty of subsequent handholding available from Jisc's data experts to help extract and clean the data, deal with ethical and legal issues, plan intervention strategies, and roll out the systems to staff and students.
The University of South Wales is currently carrying out a major evaluation exercise of its learning analytics project. This project will feed into the design of an ongoing evaluation of the whole Welsh initiative. Anecdotal evidence already indicates that universities appreciate learning from each other's experiences in this area and not reinventing the wheel at every turn. They are also able to influence product development in ways that are not possible with most commercial learning analytics software.
While each institution is developing its individual capacity for learning analytics, there are other potential benefits of working together, such as providing national-level data. Institutions may for example find benchmarking their student data relating to nursing students with other institutions more helpful than comparing their nurses to their art historians. Meanwhile, there is also potential for the Welsh Government to use anonymized data from the initiative in its education planning and policy [https://www.jisc.ac.uk/news/wales-buys-into-jisc-learning-analytics-11-jul-2018] work.
We look forward to reporting back on the initiative overall following the end of the initial funding period in 2021.
Niall Sclater is a learning analytics consultant for Jisc and author of Learning Analytics Explained.
© 2019 Niall Sclater. The text of this work is licensed under a Creative Commons BY 4.0 International License.