Blackboard Predict to be Sunset

Blackboard Predict (BB Predict) is a subscription-based predictive model designed to identify students at risk of receiving a grade of D or F in a course.  UMBC has a long partnership with Blackboard and had an active advisory role in the development of this tool.  While the product produced was high quality, it has been lightly used (although more recently DoIT has partnered with individual faculty and staff to explore ways to use it as part of our student success efforts and as a complement to our First Year Intervention Alert Program).  

Given current usage and financial constraints, we have made the decision to sunset BB Predict and will not renew our subscription.

One factor that limited the usefulness of BB Predict to improve student success through better early alerts is simple.  The model’s predictions only become precise enough to use with confidence at the semester's midpoint and are not “early enough.”  Recognizing that the earlier at-risk students can be identified with precision, the more time we have to help them and the greater the likelihood of their success, the Data Science Team within DoIT and supported by the Provost’s office constructed our own model, now called UMBC Predict.  The team’s goal was to increase the precision of the model’s estimates earlier in the semester, in this case, four weeks from the first day of classes, to allow for greater opportunities for students to seek assistance or change their approach to the class, and for these changes or assistance to have an impact.

UMBC Predict does not attempt to replicate the model and platform of our vendor partner.  The strategy employed by the Data Science Team was to focus on fewer courses in the initial stages of the work.  In particular, the team focused on courses with elevated DFW rates, large numbers of students, and importantly, courses where instructors make relatively heavy use of our learning management system (Blackboard) to engage students and to record assignments and grades throughout the semester.  It has always been true that instructors’ use of the learning management system provides valuable information to students about their progress in the course, but it is also the case that these interactions provide important information that the predictive model can use to sharpen predictions at an earlier point in the semester.

A second important factor underlying our decision to sunset BB Predict is pragmatic and necessary.  With the current pandemic and its associated negative effects upon the state budget, reducing our expenditure on a subscription-based model in favor of an internal model with equivalent or better performance for certain courses allows us to maintain our support for students while freeing up resources to support people and programs.

UMBC Predict is already being integrated into department-level enhanced early alert programs on a pilot basis.  This term, faculty in the Mathematics and Statistics Department and the Dean’s Office in the College of Mathematical and Natural Sciences plan to partner with DoIT and the Analytics and Business Intelligence Group to use targeted communications to students who may be at risk after four weeks.  The communication, delivered over the course instructor’s signature, encourages students to take advantage of their office hours along with other tips developed by the Academic Success Center to improve students’ chances of success both in the course and at UMBC.