LMS Evaluation

Overview

The Center for Teaching and Learning embarked on a formal evaluation of the Learning Management System (LMS) as part of our continued commitment to excellence in teaching and learning.

The LMS is an essential tool that supports our mission of providing high-quality education to our students, and we are committed to ensuring that it meets the needs of our community. Anticipating a near-future move from our current Blackboard Learn to Blackboard Ultra, we deemed it necessary to consider other LMS platforms to determine if Blackboard Ultra was the best fit for our university needs.

The LMS evaluation involved a comprehensive review of Blackboard Ultra and an assessment of alternative LMS options. We evaluated various aspects of the LMS, including its functionality, ease of use, reliability, and features. We sought feedback from students, faculty, and staff on their experiences with the current system and their needs and preferences for a new system.

This evaluation was a critical step in ensuring that our LMS continues to meet the evolving needs of our community and enables us to deliver the best possible learning experience. We encouraged all students, faculty, and staff to participate in this evaluation by sharing feedback and suggestions.

LMS Evaluation

Timeline

Evaluation Results

Involving the campus community took several months. We provided the following opportunities for participation in the evaluation process.

  • Faculty Committees: assessment and grading, communication tools, content delivery, course management, student tracking (data analytics and attendance), user experience
    • Committees created feature priorities, utilized the LMS-provided Sandboxes, and completed specific sections of the LMS Evaluation Rubric.
  • Staff Committees: IT/SIS Integration, Organizational Courses (Human Resources), Registrar and Student Tracking
    • Committees created feature priorities, utilized the LMS-provided Sandboxes, and completed specific sections of the LMS Evaluation Rubric.
  • Student Committee
    • Provided feature priorities and provided commentary on the student experience.
  • Virtual Demos with Recordings Available
  • LMS Survey

 

The committees identified the following LMS features as a priority:

  • Highest Priority
    • Attendance tracking through integration with Jenzabar One
    • Customizable rubrics for assignments.
    • Robust exam question pools and easy import from textbooks or documents: one-stop question editing, categorization, and faculty options.
    • A large variety of exam features – question types, multiple-selection, adaptive release, partial credit, etc.
    • Assessment question tagging to learning outcomes, meta-competencies, and cross-course tracking.
    • Hotspot assessment questions, where learning clicks on a specific location on an image to answer a question.
    • Learner progress and data – goal alignment, rubric data, and accessibility based on user level (e.g., student, course director, dean)
    • Video lecture ability – host video content on LMS or integrate with Echo360, whiteboard feature, deep integration with Grade Center for attendance.
    • Grade Center computes grades easily, group assessments populate correctly, calculations can be locked at user-permission level, and grade schemas are easy to update.
    • Content is viewed within the LMS window. Intuitive learning modules and fewer clicks to content, with nesting ability.
  • Priority
    • Mobile-device friendly, with students able to complete coursework using a mobile device.
    • Final course grade integration with Jenzabar One.
    • Easy-to-use calendar with automatic population of due dates.
    • Template and course-copy management that is user-friendly and easy to support.
    • Testing accommodations that can be set at the student-level throughout their entire experience instead of repeated each term by-course.
    • Adaptive release for assignments with customizable “triggers.”
    • Ability to use high-resolution imagines in test questions.
    • Notifications that are customizable by user level.
    • Due date flexibility – manual grade overrides, late work submission, exceptions, etc.
    • Rapid response to higher education integrity concerns, such as creating/supporting tools that detect plagiarism, cheating, and generative AI.

 

Students felt the following LMS features as a priority:

  • An intuitive interface where content is easy to access.
  • Grade book access where grades can be viewed fast and clearly.
  • A robust calendar that accurately populates assignments and activities across courses.

 

After compiling feedback and sentiments across the university landscape, a Request for Proposals (RFP) was submitted to the following LMS vendors:

  • Blackboard Ultra (Anthology)
  • Brightspace (Desire 2 Learn)
  • Canvas (Instructure)

 

Proposals from the following LMS vendors were received by the deadline of 7/28/2023. As indicated in the RFP, failure to meet this deadline resulted in removal of consideration for final LMS selection.

  • Blackboard Ultra (Anthology)
  • Brightspace (Desire 2 Learn)
  • Canvas (Instructure)

 

The three LMS vendors were invited to provide a live virtual demo via Microsoft Teams. They were provided a course file from our current course offerings and instructed to demonstrate that course in their LMS. The recordings of the presentations were viewable until 8/2/2023. Those viewing the demos - either live or recordings - completed a Demo Feedback Form.

  • Blackboard Ultra (Anthology)
  • Brightspace (Desire 2 Learn)
  • Canvas (Instructure)

 

At each evaluation stage, participants provided feedback qualitatively through discussion and quantitatively through rubrics. At the conclusion, committees provided input to their facilitator (CTL department team member). The CTL met on 8/2/2023 to make recommendations based on compiled data.

Between 8/2/2023 and 9/28/2023, the university considered recommendations from this comprehensive review. Key instructional technology administrators met to consider CTL's recommendations and the submitted proposals. Based on the proposals from the three vendors and feedback, a sole finalist for the LMS was recommended to the Provost. Subsequently, the President's Cabinet approved embarking on contract considerations with the sole finalist.

Parker University announced the sole finalist on October 2, 2023 - Desire2Learn Brightspace.

Handouts

Evaluation Resources