Portfolio > 4b > 3. ITS Evaluation
Search this site

4. A collaborative consultant with skills in:
(b) Evaluation and assessment

4b3 Process for evaluating ITS services

Development Plan Portfolio Documentation
Develop and implement a process for evaluating ITS services at Andrews University.  Invite comment from ITS customers. Describe the ITS Assessment tool and results from using it.

Various informal assessments were conducted from time to time, but the results were never tabulated or analyzed.  Nevertheless, they did provide valuable feedback.  One example was during the Y2K Project.  Our Project Coordinator, Bill Wolfer, did a great job as an ambassador for ITS by personally visiting each and every department on campus, on multiple occasions.  We learned a lot about how we were perceived by others from his astute observations around campus.  Another way we gathered feedback on our services took place in the ITS Client Services department.  Every semester or so, for a period of several weeks, the Director of Client Services would give every customer a follow-up phone call after their service request had been closed, asking questions specifically about the quality of the service, etc.  The department had plans to make this kind of feedback standard practice using an automated web response system.  It will be much easier to analyze the data that is collected this way.

The IT management team recognized the need to have a more formal assessment tool to measure customer satisfaction with ITS and to identify areas for improvement and areas of strength.  We saw this as important for ITS as a department, and as an independent way of assessing our individual performance.  We started with the latter idea first, taking the University's Administrator Performance Survey as a starting point.  Our first draft was prepared in September 2000.  We were not happy with this instrument, and together made a lot of suggestions for improvement.  One of the Directors, Gary Dickerson, took the form and our suggestions and presented a significant improvement at a subsequent meeting (Gary also did a lot of work on a survey for evaluating ITS services, but I do not have a copy of that form).

After further discussion and comments from the ITS management, team, I took the form to Dr. Jerome Thayer, the University's resident expert on survey instruments, and asked him to review it for us.  He gave us a number of valuable suggestions from his wealth of experience, and we tried to incorporate those.  The September, 2002 version of the form was judged to be ready for distribution, collection, and analysis, but unfortunately, by this time I was only weeks away from leaving my position at Andrews to return to Australia, so the survey instrument was never put to use during my time at Andrews.

However, we did learn some useful lessons about creating assessment tools.  Our earlier draft omitted the collection of any demographic information.  The "Scoring Key" for the three Service Questions were placed too far away from the boxes where respondents would mark the form.  We were able to redesign the form for ScanTron automated data entry.  In adding two questions about the directors and their teamwork, we placed the director's names under their titles (this was absent in the intermediate draft where introduced these questions).

Another survey that I prepared and this time did get to complete before leaving Andrews was the Word Processing Needs Survey, which was conducted over the web.  My purpose in conducting this survey was to collect specific information about interest in attending Microsoft Word training, as well as trying to get a feel for the readiness of the campus to adopt Microsoft as the standard office suite.  Two hundred and five Andrews faculty and staff responded to the survey, which was a very good response rate.  I prepared a report for the university Computing Committees summarizing the responses to this survey and comparing it with word processor usage in general, and analyzed the whole experience for Competency 6(b) Theories of learning and human development.


Return to 4(b) Evaluation and assessment

Created: Sunday, February 20, 2000 05:41 PM
Last Modified: Monday, May 12, 2003 9:23 AM