This article describes a participative, action evaluation process that led to a decision in 2003 to change the online learning management system (LMS) software at Central Queensland University in Australia. The evaluation process was initiated, planned and conducted collaboratively by four distinct groups of staff who had been involved in different ways in operating the previous online LMS software. Although the findings of this evaluation are institutionally specific, the process of evaluation provides a useful model for use in other institutions.