Tài liệu Reaffirmation of accreditation and quality improvement as a journey a case study

REAFFIRMATION OF ACCREDITATION AND QUALITY IMPROVEMENT AS A JOURNEY: A CASE STUDY by PHUONG THI THANH NGUYEN, B.A., B.S., M.A. A DISSERTATION IN HIGHER EDUCATION ADMINISTRATION Submitted to the Graduate Faculty of Texas Tech University in Partial Fulfillment of the Requirements for the Degree of DOCTOR OF PHILOSOPHY Approved Brent D. Cejda Co-Chairperson of the Committee James E. Brink Co-Chairperson of the Committee Bonita K. Butner Lee S. Duemer Accepted John Borrelli Dean of the Graduate School December, 2005 Copyright © 2005, Phuong Thi Thanh Nguyen ACKNOWLEDGEMENTS I could not have completed this dissertation without the enormous help and influence of many others. I wish to express my gratitude to Dr. Cejda for the outstanding guidance and support that he provided to me throughout my doctoral program as the advisor and chair of my dissertation committee. I am also very appreciative of the analytical comments and excellent assistance received from the other committee members, Dr. Brink, who served as co-chair, Dr. Butner, and Dr. Duemer. This research study would not have been possible without the participants and I sincerely thank each one of them. I am particularly appreciative of the dedication, guidance, and assistance of the Accreditation Liaison at the case study institution. I would like to convey my heartfelt gratitude to Diane for her unwaivering mentoring and support. My most sincere appreciation also goes to Lee for her wonderful help and encouragement. It was Diane and Lee who read numerous drafts, provided suggestions, and reminded me to take a break. I would like to send my very special thanks to Dr. Reckner, Mr. Le Cong Khanh, Steve, and all my other friends at the Texas Tech University Vietnam Center and Vietnam Archive for sponsoring and facilitating my job as a graduate assistant throughout the years of my studies. I greatly appreciate the leadership, Office of Administration and Personnel, Office of International Relations and Research Affairs, Office of Graduate Studies, Department of Education, professors, and colleagues at University of Social Sciences and Humanities ii - Ho Chi Minh City, Vietnam for their support, encouragement, and trust. Especially noteworthy are the late Vice-Rector Prof. Dr. Nguyen Van Tai and his family. I would like to convey my sincere appreciation to the United Board for Christian Higher Education in Asia that generously provided me with a Faculty Development Grant for three academic years. I am also deeply grateful to the scholarship donors from the Texas Tech University College of Education, including the Rushing Endowment Scholarship, the Gordon C. Lee Memorial Scholarship, and the Berlie J. and Laine Fallon Memorial Scholarship. Many others at Texas Tech University were instrumental in my progress, including Dr. Reeve, Dr. Elbow, and Dr. Marshall. I am thankful for all of the administrators, professors, and staff of the Texas Tech University College of Education for their instructions, dedication, encouragement, and help. I am deeply grateful for Alice who generously read and edited this dissertation, as well as raised insightful comments and questions. My sincere thanks also go to Jim at the International Cultural Center and Becky at the Graduate School for their professionalism and helpfulness. Librarians at Texas Tech University were extremely helpful for which I am very grateful. I am thankful to many friends from far and near who have supported me by way of expressions of confidence, concern, hope, and encouragement along the way. Finally and most importantly, I am grateful to my parents, siblings, nieces, and nephews, who have supported me from the beginning. iii TABLE OF CONTENTS ACKNOWLEDGEMENTS ii ABSTRACT xv LIST OF TABLES xviii LIST OF FIGURES xix LIST OF ABBREVIATIONS xx CHAPTER I. INTRODUCTION 1 Background of the Study 1 What Is Accreditation? 1 How Is U.S. Accreditation Structured? 2 SACS-COC and the New Principles of Accreditation 6 Summary 9 Purpose of the Study 10 Statement of the Problem 11 Significance of the Study 12 The Research Design 13 Research Questions 13 Conceptual Framework 14 Theoretical Framework 14 How Open Systems Theory Was Started 15 How Open Systems Theory Was Developed and Used 16 Organizational Open Systems Theory 16 iv Summary II. 21 Delimitations 22 Limitations 22 Assumptions 23 Definition of Terms 24 Chapter Conclusions 27 LITERATURE REVIEW 29 Summary of Research Studies on Institutional Accreditation in Higher Education 29 Historical Overview of Accreditation 37 The Beginning of Accreditation 38 The Evolution of Accreditation 38 Development of the Accreditation Process 38 Five Problems Affecting the Development of Accreditation 41 Effect of the Assessment Movement on Accreditation 48 Public Concern About Student Learning Outcomes 48 Challenges Facing Assessment 48 The Relationship Between Assessment and Accreditation 51 Effect of Institutional Effectiveness on the Self-Study Process 52 Summary 55 Institutional Self-Study Process 56 Views of Accreditation and the Self-Study Process Summary 57 60 Institutional Self-Study Process v 60 Phase One: Prepare and Design the Process 62 Preparing the Process 62 Designing the Process 62 Factors Contributing to a Successful Self-Study Process 66 Adequate Level of Technical Expertise 67 Understanding of Externally Mandated Accreditation Criteria 68 Summary 71 Phase Two: Organize the Study Process 71 Tasks Following the Design and Plan 71 Self-Study Director/Coordinator 72 Steering Group 73 Workload and Teamwork 74 Selecting Team Members 75 Importance of Faculty Involvement 77 Training the Team Members 78 Summary 79 Phase Three: Conduct the Self-Study Process 79 Tasks and Types of Work 80 Importance of Documentation 80 Tools for Data Collection 83 Challenges During the Process 83 Summary 83 Phase Four: Discuss Results and Prepare Reports vi 84 Discussing and Preparing Reports 84 Special Forms of Self-Study Report 87 Summary 88 Phase Five: Host External Peer Visitors 88 Peer Review 88 External Visitors 91 Summary 92 Phase Six: Make Decisions and Evaluate the Self-Study Process 92 Making Decisions 92 Evaluating the Self-Study Process 96 Summary 96 Conclusion 97 Institutional Effectiveness in Relation to Self-Study 97 What Is Institutional Effectiveness? 98 Major Components of Institutional Effectiveness 101 Planning and Evaluation 101 Institutional Research 104 Institutional Effectiveness Paradigm 105 Summary 108 How Is Institutional Effectiveness Related to Self-Study 109 Pre-Self-Study 109 Post-Self-Study 114 Summary 119 vii Conclusion 119 Open Systems Theory 120 Conclusion 123 Chapter Conclusions 123 III. METHODOLOGY 127 Research Design 127 Qualitative Research 127 Strengths 128 Weaknesses 129 Case Study Method 129 Strengths 134 Weaknesses 135 Restatement of the Problem 136 Delimitation and Selection of the Case 136 Instrumentation and Materials 137 Instrumentation 137 Materials 139 Answering the Research Questions Procedures for Data Collection 140 142 Data Sources and Collection Techniques 142 Documents 143 Interviews 144 Observations 146 Audiovisual Materials 147 viii Case Study Data Collection 147 Differences Between the SACS-COC Former and New Reaffirmation Approaches 148 Development and Implementation of Internal Review Processes at USSU 149 Fieldwork Trip to USSU 151 Interviews 153 Observation and Collection of Documents and Audiovisual Materials 156 Data Management 158 Ethical Considerations 159 Bias 161 Procedure for Data Analysis 162 Data Analysis 163 Constant Comparative Method 163 Analytical Procedure 165 Validity, Transferability, and Reliability 168 Internal Validity 168 External Validity, Transferability, or Fittingness 170 Reliability 171 Chapter Conclusions 172 IV. DATA ANALYSIS AND FINDINGS 174 Introduction to the Chapter Structure 174 Response to Research Question 1 176 Overview of the Principles of Accreditation ix 176 Purpose and Philosophy 177 Structure and Process 178 Components of the Principles 179 Importance of Asking this Question 184 Similarities and Differences Between the Criteria for Accreditation and the Principles of Accreditation 185 The Principles 2001 Version Compared to the Current Version 194 Rationale for Changing From the Criteria to the Principles 196 SACS-COC Recommendations for Effective Internal Review Processes 198 Milestones for Communication Between Institutions and SACS-COC 200 Conclusions for Research Question 1 The Case Study Institution 201 203 General Information About USSU 203 Outcomes and Impacts of USSU’s 1995 Reaffirmation 205 Unique Elements Within USSU’s Context 207 Conclusions 211 Response to Research Question 2 213 Rationale for Using the Term “Extended Pilot” 214 Development of Internal Review Processes 215 Forming the Leadership Team 215 Selection of the Accreditation Liaison 216 Selection of the Leadership Team Members 218 Involvement of the Ex-Officio Members and Others 223 x Summary 224 Involving the Technical Support Team 225 Preparing for the Reaffirmation 228 Preparing the Accreditation Liaison 228 Preparing the Leadership Team 230 Preparing the Institution 231 Summary 235 Conclusions for Development of Internal Review Processes Implementation of the Internal Review Processes The Compliance Certification Work Process 235 237 238 Compliance Certification Structure 238 Data Collection 239 Writing the Compliance Certification 250 Presentation of the Compliance Certification 254 Off-Site Review 257 Conference Call With the SACS-COC Staff Liaison 258 Focused Report 259 Conclusions for the Compliance Certification Work Process 260 The QEP Work Process 262 The QEP Structure 263 Defining the QEP Topic 264 Collecting Data for the QEP 272 Writing the QEP 279 xi Hosting the On-Site Review Committee 286 Following Up 297 Conclusions for the QEP Work Process 303 Conclusions for Implementation of the Internal Review Processes 304 Conclusions for Research Question 2 Response to Research Question 3 308 317 What Institutional Effectiveness and Internal Review Processes Are 317 How Institutional Effectiveness Facilitated the USSU Internal Review Processes 319 Addressing Three Specific Accreditation Requirements in the Principles 319 Providing Data and Evidence for Addressing the Requirements and Standards 320 Providing the Foundation for Developing the QEP 330 Summary 332 How the Internal Review Processes Enhanced the Institutional Effectiveness Program 333 Impact of the 1995 Reaffirmation 333 Impact of the Internal Review Processes 335 Making Improvements During the Internal Review Processes 335 Using the Results From the Internal Review Processes 337 Focusing the QEP 339 Summary 340 xii Conclusions for Research Question 3 Response to Research Question 4 340 343 Leadership 345 The Leadership Team 345 The Accreditation Liaison 346 University Leadership 347 Communication 349 The Leadership Team and Others Involved in the Reaffirmation 350 The University Community 352 SACS-COC 355 Other Higher Education Institutions 357 The Public 358 Summary 359 Technical Support 360 Resources 365 The Compliance Certification and the QEP Work Processes 368 Conclusions for Research Question 4 380 Chapter Conclusions 384 V. CONCLUSIONS, IMPLICATIONS, AND RECOMMENDATIONS 392 Overview of the Research Study 392 Conclusions 394 The Principles Versus the Criteria 395 Critical Elements of the USSU Internal Review Processes 396 xiii Preparing Well for the On-Site Review 399 Using Technology Effectively 401 Implications 416 Recommendations 419 Recommendations for SACS-COC 419 Recommendations for the Case Study Institution 421 Recommendations for Higher Education Institutions 423 Recommendations for Future Research 427 Concluding Remarks About the Journey of Reaffirmation 428 REFERENCES 429 APPENDICES 453 A. Interview Participants 454 B. Letters of Introduction and Support From the Case Study Institution 457 C. Invitation Letter, Interview Questions, and Protocol for Participants at the Case Study Institution 459 Letter of Invitation 459 Research Summary 460 Semi-Standardized Interview Questions 462 Interview Protocol 470 Participant Consent Form 471 D. Invitation Letter, Interview Questions, and Protocol for SACS-COC 472 Letter of Invitation and Interview Questions 472 Interview Protocol 474 Participant Consent Form 475 xiv ABSTRACT Institutional accreditation is an excellent vehicle for facilitating change toward the improvement of quality in higher education. It is fulfilled by three phases: (a) a selfstudy, (b) a peer review, and (c) a decision by the accrediting agency. Self-study is at the heart of institutional accreditation; however, it is not always favored. Some participants consider it to be a waste of time and resources. Yet, self-study, if properly approached and packaged, can help ensure a high-quality future for the institution. This qualitative case study research explored the internal review (self-study) processes used by a Level VI public university, United States Sigma University (USSU) (a pseudonym), that successfully went through reaffirmation in 2004 under the Southern Association of Colleges and Schools - Commission on Colleges (SACS-COC) new Principles of Accreditation: Foundations for Quality Enhancement, which was approved in December 2001. Data were collected from reviewing online and other documents; a fieldwork trip to USSU where interviews, observations, documents, and archival materials were gathered; and follow-up correspondence that increased the validity, reliability, and utility of the research study. The data analysis strategy used in this research study was the constant comparative method and the theoretical framework was open systems theory. Because the Principles of Accreditation were new, this research study addressed two fundamental problems: (a) how to deal with this new reaffirmation approach and the associated need for change and (b) how to integrate other quality improvement xv mechanisms with the internal review processes. Results of the study indicate that sixteen critical elements contributed to the success of the internal review processes at USSU: 1. Adapting to the new Principles; 2. Selecting the right Accreditation Liaison; 3. Selecting the right Leadership Team; 4. Starting early and using timelines; 5. Training and preparing the participants and the university community; 6. Using unique strategies for developing the Compliance Certification and the Quality Enhancement Plan (QEP); 7. Having a well-written Compliance Certification and QEP; 8. Preparing well for the on-site review; 9. Having university leadership commitment, support, and belief in accreditation; 10. Communicating effectively; 11. Using technology effectively; 12. Leveraging available resources and matching the reaffirmation with the institutional circumstances; 13. Establishing an effective relationship and regular communication with the SACS-COC staff liaison; 14. Having competent reviewers; 15. Using internal review findings for improvement; and 16. Having a strong institutional effectiveness program. xvi This study also indicates that the more reaffirmation requirements are integrated into institutional processes (e.g., strategic planning and budgeting), the better the institution will do in carrying out its internal review. Sixteen critical elements of the USSU internal review processes and insights gained from this case study are potentially useful to other SACS-COC universities. This research contributes to knowledge for improvement of educational practices. It also meets the SACS-COC universities’ needs for understanding effective ways of conducting internal reviews for quality improvement while simultaneously satisfying the SACS-COC new requirements for reaffirmation. xvii LIST OF TABLES 1. Summary of Research Studies on Institutional Accreditation 2. Abbreviations Used for Interviews From Other Related Research 176 3. Overview of the Principles of Accreditation 180 4. Similarities and Differences Between the Criteria for Accreditation and the Principles of Accreditation 187 5. USSU’s External Timeline for 2004 Reaffirmation 233 6. Grouping and Assigning of Core Requirements and Comprehensive Standards 241 Strategies Used in Development of the Compliance Certification Versus the QEP 306 8. Summary of Milestones for Development of the Internal Review 308 9. Summary of Milestones for Implementation of the Internal Review 309 7. 10. Critical Elements, Correlative Literature, and Contributions to the Literature xviii 31 404 LIST OF FIGURES 1. SACS-COC Organizational Chart 8 2. Conceptual Framework: An Open Systems Perspective xix 133