Serving Education and Training Markets Since 2006

LMS Evaluations (Learning Management Systems)

Every year the Gilfus Education Group assists dozens of academic institutions in evaluating their existing LMS. In addition, many academic institutions embark on their own LMS Evaluation projects to understand which LMS is the best for them. For various reasons each academic institution selects a specific application, be it open source or proprietary or because of ease of use, costs or, functional and pedagogical reasons. The following are a few examples of independent evaluations done by academic institutions and gathered from their appropriate sites in order to make it easier to compare and contrast each process and the identified results.

LMS Evaluations Gilfus Education Group Process

the Gilfus Education Group will be releasing a white paper shortly providing various insights for the LMS Evaluation and Selection Process. For a look at an in depth LMS Evaluation process take a look at the materials the Gilfus Education Group uses for LMS evaluation’s.

A Gilfus Education Group Consultant can be reached for discussion at 888-861-3375 or by requesting a call through the contact form.

As more resources and examples become available we will add them to this posting as a reference.


Selection Criteria  should be:

1) longevity,
2) customer base,
3) parent-companies,
4) partners,
5) technological sophistication,
6) financial credibility, customer
7) engagement protocol,
8) deployment options and protocols,
9) deployment record,
10) industry experience,
11) conformance with industry standards.


University of North Carolina – Charlotte

Executive Summary: This report describes the results of an evaluative comparison of the Moodle Learning Management System with the University’s current Blackboard system (Vista); Recommends, on the basis of that evaluation, that the University adopt Moodle as its sole Learning Management System; and Outlines the consequences of doing so in conjunction with a plan for making the transition from Blackboard to Moodle.

LMS Evaluation Resources:

  • Student Moodle Survey – Spring 2009
  • Faculty Moodle Evaluation – Spring 2009
  • Final 2009 Report of the LMS Evaluation Committee
  • PowerPoint presentation given by Lorraine Stanton at the March 28th LMS Evaluation Committee meeting

Duration: November of 2008 – June 2009

Conclusion: The conclusion of the Committee is that Moodle offers a number of potential benefits,
particularly with respect to adaptability, ease of learning and use, and relevance to UNC Charlotte’s current and projected instructional mission. Consequently, a trial of Moodle is desirable and should be initiated in a timely manner during the 2008-2009 academic year.

Open Source Collaborative – North Carolina Community College System (NCCS)

Executive Summary: In May 2008, the State Board of Community Colleges approved a contract with the University of North Carolina General Administration (UNC-GA) to establish the joint Systems Open Source Collaborative Moodle Assessment. Open source collaboration was attractive to both the NCCCS and the UNC-GA. Moodle was selected as the first CMS software for pilot use by institutions in both Systems. Emphasis was placed on assessment of Moodle to discover if “Moodle is a viable open source alternative to Blackboard,” the proprietary online CMS used by the vast majority of all North Carolina public institutions of higher education. Open source describes software distributed under licenses guaranteeing anyone the rights to freely use, modify and redistribute the source code.

LMS Evaluation Resources:

Duration: May 2008 – October 2009

Conclusion: The end-of-term student and instructor surveys showed that Blackboard and Moodle are not that different. The real difference is found in student perception of their teachers’ comfort level with the application. There exists a significant correlation between student survey scores of North Carolina Community College System both Blackboard and Moodle with the perceived comfort level of instructors using either application. Thus, student perceptions of both CMSs were influenced by instructor experience, training, and skills. The Assessment Team concludes that this study has only addressed the academic considerations of Moodle as a viable alternative to Blackboard. This study did not fully address the issues of technology and funding as framed in compatibility and interoperability of all learning technology applications.

University of Notre Dame

Executive Summary: The University Council for Academic Technologies (UCAT) requested the Course Management System Subcommittee initiate a thorough review of CMS alternatives for the University of Notre Dame and provide a recommended strategy and course of action for UCAT by the end of the 2007-8 academic year. With the assistance of faculty from a variety of academic disciplines, students, staff from the Office of Information Technologies, and Kaneb Center for Teaching and Learning, the study was conducted from May 2007 – February 2008.

LMS Evaluation Resources:

  • University of Notre Dame Course Management System Replacement Project Findings and Recommendations

Duration: May 2007 – February 2008

Conclusion: The University of Notre Dame continue to use Blackboard Vista as the institutionally supported Course Management System at least through Spring Semester 2011. The University plan to review CMS options again in three years (beginning in 2010). The review process illustrated that each CMS evaluated had strengths and weaknesses and that the course management system “space” is rapidly changing along with the overall online learning system landscape. The committee felt that staying with the current system another three years offered a commitment to faculty that their investment in learning the CMS will not be short-changed by switching in the near future to a new system. The three year period was also influenced by information presented by Blackboard of its intent to develop a “next generation” learning management system which will replace its existing CMS offerings, with a projected rollout of 2010. The Blackboard Vista product line will cease to be upgraded and enhanced once the next generation product is released, thus requiring Notre Dame to switch from Vista to a new product as early as 2011.

University of Northern Texas

Executive Summary: Between March 13, 2007 and April 30, 2008 the University of North Texas conducted a systematic evaluation of learning management systems in planning for its future. The Learning Enhancement Planning Group (LEPG) was charged with carrying out the evaluation project and making a final recommendation to the Provost by April 30, 2008.

Duration:March 13, 2007 – April 30th, 2008

Conclusion:The recommendation followed the conclusion of an extensive learning management system evaluation process conducted by a committee made up of stakeholders representing all of the colleges and schools as well as campus administrative groups that provide technical and end-user support. During the research and testing phases of the evaluation project, the LEPG found that Blackboard Vista1:

  • has all of the features to perform functions desired by stakeholder groups (faculty, students, and support personnel) and ranks highest in the number of features available
  • is the easiest system to use for UNT course designers, faculty, and students
  • integrates with current third-party tools and can be integrated with the SIS (PeopleSoft)
  • along with Angel, worked best with screen readers during accessibility testing
  • is the least expensive in terms of annual license and support costs
  • offers 24/7 technical support options with an acceptable service level agreement for response times, number of contacts, and number of incidents per month
  • rates highly among other clients in terms of satisfaction with stability, scalability, and quality of product support

McMaster University

Executive Summary:The Learning Technologies Resource Centre has embarked on a multi-stage evaluation of next-generation Learning Management Systems to find a replacement for WebCT Campus Edition 4.1 as it reaches end-of-life status when the current licence agreement expires in 2008.. Given the fact that it will take approximately 1 year to migrate to a new system it is crucial that we start the process now. The various and diverse prespectives at this institution (academic, administrative, technical) necessitate that this process be as inclusive as possible. This website will be used to update the community on our activities, events and progress

LMS Evaluation Resources:

  • Currently Unavailable

Duration: November 2006 – April 2008

Conclusion: The Learning Technologies Resource Centre (LTRC) has completed an extensive multi-year project to find the best possible replacement for WebCT Campus Edition 4.1 (currently in use). The LTRC along with the LMS Selection Committee and Senior Administration chose the Blackboard Learning System after extensive analysis of twelve potential products and the teaching and learning needs of the community.

University of Canterbury, New Zealand

Executive Summary: Due to the merger with the Christchurch College of Education in January 2007 the University now has two supported Learning Management Systems (LMSs), Blackboard and Interact (known in the College as StudentNet). At the time of the merger it was agreed to support both systems until a review could be undertaken in 2008, with a view to moving to a single platform for 2009/2010.

It was agreed that the review should consist of:
1. an analysis of the University’s online learning needs, followed by
2. evaluation of the best LMS to meet these needs.

LMS Evaluation Resources:

  • Current LMS Review and Moodle Trial Items

Duration: October 2007 – October 2008

Conclusion: An outcome of this process was a decision to trial Moodle during Semester 2. This took place with approximately 30 courses using the system. The Final Report of the Steering Group gives detail on the process followed, the evaluation of the trial, further information collected, and the reasons for the decision to adopt Moodle.

Australian National University

Executive Summary:

LMS Evaluation Resources:

Duration: October 2007 – October 2008


Scroll to Top