Improvement Reviews
Based on the feedback on test-based, course-embedded, and survey-based assessment tools, the School has undertaken several steps to improve the assurance of learning program. Below we outline a number of noteworthy actions we took to “close the loop”:
1. Developed a Core-Concept Test to complement the ETS Major Field Test in the Undegraduate and Graduate Programs.
The Assessment Committee found that the ETS test did not adequately measure student functional knowledge, which is addressed in Student Learning Outcome #5 in Functional Knowledge.
- The ETS Major Field test does not assess all the important learning outcomes identified by the School. For example, it does not cover ethics, and operations management.
- In addition, the ETS-MBA does not include questions on international business and information systems.
In order to measure student functional learning more effectively, the School decided to develop its own Core-Concept test, which covers all learning objectives critical to its program. The Core-Concept test measures all the functional areas of knowledge. Further, the Core-Concept test provides individual level meaures of student functional learning. On the Core-Concept test, the average student functional learning scores are higher than the comparable ETS scores.
- The two tests are administered alternatively to ensure that (1) all students must take one of the tests, and (2) no student is required to take both tests.
- This effort is recognized as a significant assessment effort by major assessment and accreditation bodies. This is exemplified by the fact that the Dean and The Chair of the Assessment Committee have been asked to share their experience in implementing this system at the Assessment Institute in Indianapolis organized by IUPUI and the Assessment Seminar in Dallas, Texas, organized by the AACSB, the premier accrediting agency for business schools worldwide.
2. Incorporate a Business Simulation Exercise (Comp-XM) to Measure MBA Learning Outcome 3b: Critical, Analytical and Integrative Thinking
Because of the compressed curriculum of the MBA program, the School found it difficult to implement an elaborate course-embedded individual-level measure of critical, analytical, and integrative thinking, which is MBA Learning Outcome # 3b. The School decided to use a business simulation exercise to measure this important learning objective.
- Several commercially available business simulation packages were evaluated.
- Finally, the school decided on the Comp-XM simulation package because it is designed to address the question, “do my students understand the business”?
- Each student is assessed in two ways: (1) how well s/he is able to run the simulated company (using a balanced scorecard approach), and (2) how well s/he does on a series of quizzes related to the simulation.
- The Comp-XM package is also cheaper than other products because it comes as an add-on feature to the main simulation package CAPSTONE, currently used in the Business Capstone course.
3. Provide better undegraduate course content, more closely related to the Undergraduate Communications Learning Outcome, to improve student learning on Learning Outcome #4: Communication.
S440: Organizational Communication was a required course for business students mapped to Learning Outcome #4 on Communication. However, student data indicated that student learning in this area was not progressing at expected levels.
- The Committee reviewed the data and concluded that a change was needed to improve student learning in this area.
- The Committee reviewed the syllabus for S122: Interpersonal Communication and decided to replace S440 with S122 in the curriculum.
- The Committee will continue to monitor student data to determine the effectiveness of this change.
- Since January 2007, student feedback on their level of preparedness in the area of communication are improving.
