Signals – the affordances for an early warning system in higher education

Purdue University developed an early warning system Signals, to improve student retention. Such a system can be useful, but the efficiency of Signals have been analysed in a series of blog entries and no causal connection could be found between students who have used the system and their tendency to stick with their studies (Strausheim 2013)

Signals combines demographic information with online engagement to produce red, yellow or green light to indicate to students how well they are participating in their courses. The information is also provided to the lecturers so that they can provide help before students drop out or fail the course (Strausheim 2013). The method to structure an early warning system has permeated the industry and a few software packages are available such as Course Signals (Ellucian),  Retention Centre (Blackboard) and Student Success System (Desire2Learn). Although early warning systems have been developed based on Purdue University’s claim that it reduced dropout rates, research findings do not validate this claim.

Pistilli, research scientist at Purdue claimed that two Signals-enabled courses offered at Purdue University increased students’ six year graduation rate by 21.48%. This claim is not supported by two other researchers.

Caulfield, Director of blended and networked learning at Washington State University compared the retention rates of the 2007 and 2009 cohort enrolled for Signals courses and suggested that the data Purdue described as data analysis just measured how many courses the students were enrolled for (Strausheim 2013). According to Caulfield the students took more signals courses because they persist, rather than persisting because they were taking Signals courses.

Essa, the vice president of research and development of McGraw-Hill Education supported this finding. According to Essa, who simulated the data, the retention gain is not a real gain (causal) but an artefact of the fact that students who are staying longer at university are likely to graduate.

Researchers do not question the need to integrate early warning systems such as Signals, they only question the claim that early warning systems can improve student retention (Strausheim 2013). Research findings do not support the claim that these systems improve retention, but these systems do not face and existential crisis according to Essa, who helped to design Student Success System for Desire2Learn. According to Essa the aim of early warning systems is to make sure that students are performing well. Therefore, the focus should be on investigating the impact of the early warning system on learner results rather than retention.

it is not sufficient to investigate the impact of early warning systems on student retention. Lecturers need to be able to use the information to intervene by providing Just-In-Time-Teaching while students are taking the course. One of the reasons why staff and lecturers do not take this route, is that it can take months before ethical clearance and permission to use staff and student data are given. In order to report on the success or failure of an early warning system, the following questions need to be asked:

  • What is the impact of the early warning system on student performance during the course?
  • How did the lecturers react on the information provided by the early warning system? In other words, did they provide remedial activities in order to improve student performance? Did they provide support?
  • How did the students react on the fact that their lecturers were informed about non-completions of learning activities? Did they appreciate it or did they feel that their privacy was invaded?
  • What is the relationship between course participation and student success, in other words, do students, who complete all learning activities performa better than those who don’t? If not, the following question can be asked:
  • How effective is the structure of the course? In other words, are the students guided through small steps from unfamiliar with course content to being competent?
  • How can the course be improved to improve student success?

These questions might ignite more questions that can be asked in order to uderstand how early warning systems such as Signals can improve success rates in higher education. To me, it is more of an ethical issue to have the information available, but not acting on it, than not having permission to collect the data that can improve the success of my students while the lecturer is in the position to intervene in order to increase student success.

(34 visits)