← Back to News List

New Report Can Help Faculty Visualize Student Engagement

"Waterfall" course dashboard leverages student use of Bb LMS

DoIT has a new addition to our suite of data visualizations supporting course engagement/instruction and teaching: The Blackboard course "waterfall" report (UMBC vpn req'd if using off-campus). 

As illustrated below, the waterfall dashboard summarizes student interactions within our Blackboard Learning Management System (LMS) on a week-by-week basis. Each row represents a student and each column represents a week in the semester, with the density of an individual cell color representing the student's time spent in the Blackboard course that week. Each course waterfall is organized by the student's final course grade on the left.

Resembling cascading waterfalls -- you can find the Thanksgiving or Spring Break "rocks" at the top of a course's waterfall, when most students tend spend less time in Bb -- the new report can help faculty discern student engagement patterns in prior terms, pinpointing areas where course design might be reflected in student engagement. DoIT first debuted the waterfall report in a 2021 campus presentation and related publication. However the originator of the course "waterfall" idea is David Wiley, formerly a professor at BYU, and now Chief Learning Officer at Lumen Learning, who showed it as part of a keynote presentation at the 2011 Educause Learning Initiative (ELI) annual meeting. It's only taken us 12 years to scale Wiley's idea to all UMBC faculty. 😉

By using the "Archetype" and "Scan All Courses" tabs, the waterfall dashboard also incorporates 2016 research by Blackboard (replicated by UMBC in 2018) showing the LMS course design "archetype" based on the following framework:

Supplemental: high in content but with very little student interaction.

Complementary: used primarily for one-way teacher-to-student communication.

Social: high peer-to-peer interaction through discussion boards.

Evaluative: heavy use of assessments to facilitate content mastery.

Holistic: high LMS activity with balanced use of assessments, content, and discussion.

As one might expect, the strength of relationship between course activity and grades tends to be higher in Blackboard courses with an evaluative or holistic design. This appeared to be the case in 2019 research by DoIT showing student activity was highest in CNMS Blackboard courses, largely "due to relatively more CNMS courses using assessments" like quizzes and exams. 

Finally, the waterfall set of student engagement visualizations is one of the first campus-wide reports to use Tableau, the visualization tool that is being used as part of the Report Exchange (REX) data warehouse migration to HelioCampus. UMBC's waterfall dashboard was developed by Mike Sharkey and is maintained by Tom Penniston, Coordinator of Learning Analytics, in collaboration with the Analytics and Business Intelligence group.

"The faculty course waterfall report is part of UMBC's long-standing commitment to promote academic excellence through democratization of data," says Penniston, who recently gave a UMBC presentation based on a paper on the ethical use of data-informed behavioral nudging at the Human-Computer Interaction International (HCII) conference in Copenhagen, Denmark. "As such, we believe it also complements the Check My Activity (CMA) student feedback dashboard to inform and enhance their own self-efficacy."

If you are unfamiliar with any of the learning analytics tools and data available to faculty, a good starting point is to review the REX guided reports (requires UMBC VPN if accessing from off-campus). Also, stay tuned  for a  needs assessment survey DoIT will be distributing later this term to gauge how best we can support our community's analytics training needs.

Tags:

Posted: October 23, 2023, 2:37 PM

Read Original Post in myUMBC