EIRP Proceedings, Vol 10 (2015)

Comparative Analysis of Ranking and Accreditation: Exploring a Set of Universal Principles for Higher Education Quality Assurance

Steve O. Michael¹

Abstract: All universities are not equal. Universities are not equal in size, scope, curricular offerings, and resources. More importantly, they are not equal in mission, scale of operation, productivity, and quality. Even two universities located within the same geographical locations may differ considerably in productivity and quality let alone those that are located a world apart. Given the wide range of differences in the environments of these institutions, in the political systems within which they reside, in the economic contexts within which they operate, and in their historical origins, the variations among higher education institutions are understandable and frankly speaking should be anticipated. Given the differences among institutions, how should we approach the issue of their quality? In response to this question, the benefits and process of rankings are compared to that of accreditation. The implications of rankings and accreditation for two “randomly” selected institutions in the US are discussed. By reviewing the standards used by two accrediting commissions, a set of principles that is applicable universally is recommended.

Keywords: curricular offerings; quality; higher education institution

Introduction

The truism that all fingers are not equal is applicable to higher education institutions. Human needs for higher education are enormous, complex, and varied; hence, higher education’s responses to these needs must be comprehensive, complex, and varied. If this is the case, why do we sometimes address the issue of quality among higher education institutions as if they were a monolithic entity? The reluctance to embrace a universal scheme of institutional ranking by some is based primarily on the understanding that the differences among institutions are so vast that any attempt to rank them would be futile - a case of comparing apples with oranges. However, as we all know, this concern has not deterred the ranking industry from cranking out their rankings every year. Institutional ranking is not only here to stay, it is gaining grounds across the globe and doing so rapidly.

But if rankings, fraught with myriads of problems, are gaining grounds, why is the movement toward internationalized accreditation stalling? Rankings are a simplistic solution to a complex problem. Parents of prospective students want to select the best institutions for their children and need whatever information there is to help navigate through hundreds of institutions out there. By reducing institutional characteristics and activities to a single number that is ranked, the ranking industry is seen as providing valuable benefit for parents and prospective students. Most parents do not go beyond the ranking number to question the methodology used and the criteria employed. In many people’s minds, rankings describe the quality of institutions. After all, whatever is ranked number one should be better than whatever is ranked number two.

Why does the ranking industry flourish? The simple answer is money. To the extent that rankings enable the producers to sell magazines and to the extent that they can make money from the exercise, the desire to rank will continue to rise. Accreditation on the other hand, provides little information to parents of prospective students insofar as all it offers is a list of institutions that are accredited and nothing more.

Yet, many, if not most, higher education leaders know that accreditation speaks more to the issue of institutional quality than rankings. Accreditation is a painstaking process of evidence-based peer review of internal operations and systems of an institution for the sole purpose of providing further improvement. This definition is a departure from the one that describes accreditation as merely ascertaining the minimum acceptable compliance. The strength of accreditation as embraced by the Association for the Global Advancement of Universities and Colleges (AGAUC) lies in the provision of agenda for further improvement. After all, the pursuit of excellence is a relentless critiquing of the status quo for the sole purpose of transformation.

Purpose of the Article

The purpose of this paper is to compare ranking criteria with the accreditation standards, in this case, the Higher Learning Commission’s and the Western Association of Schools and Colleges’ (WASC) standards, describe the results of ranking and accreditation for two randomly selected institutions, and examine a set of criteria that can be meaningful and useful for international quality assurance in higher education. The Higher Learning Commission (HLC) is the accrediting organ of the North Central Association of Colleges and Schools (NCA) responsible for accrediting over 10,000 institutions in the mid-western U.S. The Western Association of Schools and Colleges is one of the six regional accreditation associations in the US. WASC provides accreditation services to over 4,000 institutions and organizations in the western region of the United States.

Differences among Higher Education Institutions in the US

In the US, there is a wide range of higher education institutions. Currently, there are over 4,000 higher education institutions representing different sectors: private, public, small, medium, large, rural, urban, specialized, comprehensive, teaching, research and so on. As shown on Table 1, in 2009, there were over 2,770 4-year and over 1,720 2-year universities and colleges in the US. Of the 4,495 institutions, over 62% were private institutions, while about 37% were public institutions in 2009. Of the 2,823 private institutions, about 76% of them are 4-year degree granting institutions. The majority of the public institutions are 2-year (58%) associate degree granting institutions. The differences among these institutions are so huge that without a robust scheme to categorize and classify them, it would be difficult to compare their unique challenges and contributions.

Table 1. US Higher Education Institutions By Sectors (2009)

Sectors	4-Year	%	2-Year	%	Total	%
Private	2,102	75.8	721	41.9	2,823	62.8
Public	672	24.2	1000	58.1	1,672	37.2
Total	2,774	100	1,721	100	4,495	100

Source: http://www.census.gov/compendia/statab/2012/tables/12s0278.pdf

A major difference among institutions is institutional size as defined by enrollment. Table 2 shows student enrollment by institutional sectors. Although, over 62% of higher education institutions are private, the private sector enrolled only about 28% of students in 2009, while the public sector enrolled over 72% in spite of the fact that only 37% of institutions were public.

While the number of students educated in the public sector is roughly split between the 4-year degree granting and the 2-year associate degree granting institutions, the public sector educates almost all (94%) the 2-year enrolled students.

Table 2. US Higher Education Institutions By Sectors and Enrollment (2009)

Sectors	4-Year	%	2-Year	%	Total	%
Private	5,197,000	40.3	420,000	5.6	5,617,000	27.5
Public	7,709,000	59.7	7,101,000	94.4	14,810,000	72.5
Total	12,906,000	100	7,521,000	100	20,427,000	100

Source: http://www.census.gov/compendia/statab/2012/tables/12s0278.pdf

Carnegie Classifications of Higher Education Institutions

Given the huge number of higher education institutions and the complexity of their operations, it is almost guaranteed that no one classification scheme will suffice. Therefore, there are several agencies and organizations with different classification schemes that serve different purposes. Although, there are many agencies that use different methods to classify institutions in the US, the most popular and comprehensive classification scheme is provided by the Carnegie Foundation for the Advancement of Teaching² as shown on Table 3 below.

Table 3. Carnegie Classifications Of Higher Education Institutions

Basic Classifications	Descriptors
Doctorate-Granting Universities	Institutions were included in these categories if they awarded at least 20 research doctorates in 2008-09. First professional and Professional doctoral degrees (J.D., M.D., Pharm.D., Aud.D., DNP, etc.) were not counted for the purpose of this criterion.
Master’s Colleges and Universities	Institutions were included in these categories if they awarded at least 50 master's degrees in 2008-09, but fewer than 20 research doctorates
Baccalaureate Colleges	Institutions were included in these categories if bachelor's degrees accounted for at least 10 percent of all undergraduate degrees and they awarded fewer than 50 master's degrees (2008-09 degree conferrals).
Associate’s Colleges	Institutions were included if their highest degree conferred was the associate's degree or if bachelor's degrees accounted for less than 10 percent of all undergraduate degrees
Special Focus Institutions	The special-focus designation was based on the concentration of degrees in a single field or set of related fields, at both the undergraduate and graduate levels.
Tribal Colleges	Tribal colleges are defined as members of the American Indian Higher Education Consortium

Source: http://carnegieclassifications.iu.edu/methodology/basic.php

Note that the number of degrees represents the number of degrees conferred in 2008-2009 as opposed to the number of students enrolled. This classification scheme captures all the higher education institutions. However, it does not convey all the attributes of institutions. For example, while tribal colleges are included, historically black institutions are not and the emerging minority-serving institutions are not reflected by the scheme. The number of these institutions would have been reported by their degree classifications, e.g. Associate, Baccalaureate, Master’s, and Doctorate.

Comparing Apples and Oranges

To illustrate the vast differences among institutions in the United States, Table 4 presents two institutions that are diametrically different from each other. The goal is to show how rankings actually do a disservice to society by attempting to rank or un-rank these institutions.

Looking at both Ohio State University’s and Charles Drew University’s profiles, one would wonder what the two institutions could possibly have in common other than the fact that they are both located in the United States, they both produce medical doctors (among others), they both carry out some research and some teaching activities, and they are both located in big cities. Their differences, however, are staggering. How does one compare an institution with 600 students to the one with 60,000? For every one student enrolled at Charles Drew University, Ohio State enrolls 100. Ohio State is almost 100 years older than Charles Drew University. Ohio State academic program offerings are comprehensive, while Charles Drew University’s academic offerings are narrowed and specialized. Charles Drew University is heavily focused on research with limited teaching; hence, the number of students - a situation that will likely change in the coming years as the school embarks on academic program expansion and increase in student enrollment. Ohio State’s mission describes traditional focus on knowledge discovery and dissemination, while that of Charles Drew University includes specific social agenda.

Table 4. Ohio State and Charles Drew University Comparison

Characteristics	Ohio State University, Ohio	Charles Drew University, California
Sector	Public	Private
Founded	1870	1966
Mission	We exist to advance the well-being of the people of Ohio and the global community through the creation and dissemination of knowledge.	The University develops a diverse group of health professional leaders who seek social justice, promote wellness, provide care with excellence and compassion, and are uniquely qualified to transform the health of underserved populations through outstanding education, research, and clinical services in the context of community engagement.
Academic Programs	Comprehensive	Specialized, Medical
Focus	Teaching and Research	Mostly Research
Teaching Focus	Large Undergraduate and Large Graduate	Predominantly Graduate
Enrollment	60,000 approx.	600 approx.
Location	Urban	Urban
Total Assets	$4,720,629,000*	$144,940,900**

* http://controller.osu.edu/acc/2011_fin_rpt.pdf (Year 2010)

**http://990finder.foundationcenter.org (Year 2013)

Even more pronounced is the amount of resources at the disposal of these institutions. While Ohio State net asset in 2010 was almost $5billion, the total assets of Charles Drew University in 2013 totaled only about $145million. If resources are proxy for quality, then Ohio State’s quality would be astronomically higher than that of Charles Drew University. In this case, rankings would be accurate in their results.

Table 5. Core Expenses Per FTE Enrollment By Function (2013)

Expenses	*Ohio State	**Charles Drew University
Instruction	$16,161	$13,828
Research	$8,037	$26,200
Public Service	$1,938	$4,591
Academic Support	$2,993	$6,201
Institutional Support	$4,952	$20,982
Student Services	$1,624	$2,354
Other core expenses	$1,928	$1,490

*http://nces.ed.gov/ipeds/datacenter/InstitutionProfile.aspx?unitId=adabafb2b4b1

**http://nces.ed.gov/ipeds/datacenter/InstitutionProfile.aspx?unitId=acacacb4b1b1

However, aggregate assets of an institution provide us with limited information. Institutional expenditures per full time equivalent (FTE) enrollment provide a different dimension to our understanding of how two or more institutions spend their resources. Table 5 shows 2013 core expenditures per FTE enrollment by function for Ohio State and Charles Drew University. With the exception of the instructional expense, Charles Drew University spent more on its students than Ohio State, a revelation that was obviously not captured by the ranking exercise. With respect to research, Charles Drew University spent more than three times of what Ohio State spent on research per student FTE. Equally revealing is the amount spent on public service, which is more pronounced in the Charles Drew University mission than that of Ohio State. With respect to institutional support, Charles Drew University spent more than five times Ohio State’s expenses per student FTE.

The differences in spending should, however, be understood in the context of the differences in costs of living. Ohio State is located in Columbus, Ohio, while Charles Drew University is located in Los Angeles, California and for true comparison, a cost of living adjustment should be carried out. Even with that done, nevertheless, Charles Drew University spending per student FTE is laudable.

The Ranking Results of Ohio State and Charles Drew University

Given the differences between Ohio State University and Charles Drew University, how does one of the leading ranking agencies in the US rank them? And more importantly, is the ranking useful in determining institutional quality? What is the take away for parents or prospective students or even the general public from the results of the ranking exercise of the US News and World Report?

In response to these questions, Table 6 provides summary US News and World Report’s rankings for Ohio State and Charles Drew University.

Table 6. The Ranking Results of Ohio State and Charles Drew University

	*Ohio State University, Ohio	**Charles Drew University, California
Ranking Category	National Universities	Unranked
National Ranking	54	Unranked
Global Ranking	34	Unranked

*http://colleges.usnews.rankingsandreviews.com/best-colleges/ohio-state-6883

**http://colleges.usnews.rankingsandreviews.com/best-colleges/drew-university-of-medicine-10365

So what can we learn from this publication other than the fact that Ohio State was ranked, but Charles Drew University was not? How useful is the information to those interested in advancing quality understanding in higher education? To answer the question we would need to first know the reasons why some institutions are unranked by the US News and World Report. According to the US News and World Report, institutions are unranked if one or more of the following applies:

• If a school does not use SAT/ACT score for undergraduate admission;

• If too few respondents rated the school;

• If a school has less than 200 students;

• If there is a large proportion of nontraditional students, and if there are no first year students.

The US News and World Report stated that:

As a result of these eligibility standards, many of the for-profit institutions have been grouped with the Unranked schools; their bachelor's degree candidates are largely nontraditional students in degree completion programs, for example, or they don't use SAT or ACT test scores in admissions decisions.

In total, 148 colleges in the National Universities, National Liberal Arts Colleges, Regional Universities and Regional Colleges categories are listed as Unranked.

We also did not rank 83 highly specialized schools in arts, business and engineering.³

Therefore, the reason for Charles Drew University unranked status includes the fact that it is a specialized institution, it is primarily a graduate institution with perhaps less than 200 undergraduate students, and perhaps relatively unknown beyond California state or the western region of the US. Even if Charles Drew University was ranked, on what criteria would it have been ranked as compared to Ohio State and would the information still be useful for quality decision making?

In response to the question, the criteria and the weights used by the US News and World Report are presented on Table 7.

Table 7. The US & World Report College Ranking Methodology⁴

No	US & World Report Ranking Criteria	Weights	*Degree of Quality Indicator	*Comments
1.	Undergraduate academic reputation	22.5%	Low	This criterion is based on the popularity of institutions among college leaders. Popularity is influenced by non-academic factors. The longevity of an institution, the amount of money spent on advertisement, and success with competitive sports have impact on the popularity of an institution.
2.	Retention	22.5%	Moderate	Retention rate is the proportion of first year students who enrolled fall to fall. Without knowing the GPAs of those who transfer, it is difficult to use this criterion as quality indicator.
3.	Faculty Resources Class Size (classes fewer than 20 students (30%) Proportion of classes with 50 or more students (10%) Faculty salary (35%) Professors with the highest degree (15%) Student-faculty ratio (5%) Proportion of full-time faculty (5%)	20.0%	Moderate	This criterion is a composite one with 6 sub-factors included. Class size suggests the degree of interactions between professors and students. This does not capture the quality of interaction. Faculty salary is important to the extent that institutions can pay higher salary to hire highly talented professors. Proportion of full time faculty suggests that an institution uses professors who can focus on research and students fully instead of working two jobs.
4.	Student selectivity SAT/ACT scores (65%) Graduation in the top 10% (25%) Ratio of admitted to applications (10%)	12.5%	Low	To the extent that high SAT/ACT scores are indicative of highly talented students who actively apply their talents in school, to that extent higher scores suggest higher quality. The same argument is true for graduation in the top 10%. But the ratio of admitted to applications only suggest institutional popularity.
5.	Financial resources (Average spending per student on instruction, research, and student services)	10.0%	Moderate	If money can buy quality, then the richer an institution is, the higher its quality. But to the extent that resources in input, the use of resources per se represents low quality indicator.
6.	Graduation rate performance	7.5%	High	Higher graduation rates suggest a more productive educational environment.
7.	Alumni giving rate	5.0%	Low	Alumni giving rate is used to suggest student acknowledgement of the impact of the institution on their lives. However, experience shows that the giving rate depends on a) the number of alumni produced, b) how effective the institution is in cultivating donors, and c) the culture of giving that exists.

* Author’s assessment and comments

As shown on Table 7, the US and World Report uses only 7 factors (undergraduate academic reputation, retention, faculty resources, student selectivity, financial resources, graduation rate performance, and alumni giving rate) to determine an institution’s rank. Some of these factors are composite. Each of the factor carries a weight determined by the US and World Report. Changes in the weights assigned to these factors produce different ranking results. More troubling is the fact that ranking exercise depends on respondents that may have little or no knowledge of some of the institutions they have been asked to rank.

It is the opinion of this author that the ranking results would be remarkably different and perhaps more meaningful should the participating institutions be given the opportunity to assign weights to the ranking factors based on the relatively importance of these factors to the mission of the institutions. By so doing, institutions would have the privilege of differentiating emphasis on ranking criteria based on the differences in their mission.

The Benefits of Accreditation

Voluntary accreditation of educational institutions is a uniquely American invention. The U.S. Network for Education Information defines accreditation as follows:

Accreditation is the process used in U.S. education to ensure that schools, postsecondary institutions, and other education providers meet, and maintain, minimum standards of quality and integrity regarding academics, administration, and related services. It is a voluntary process based on the principle of academic self-governance. In international terms, accreditation by a recognized accrediting authority is accepted as the U.S. equivalent of other countries' ministerial recognition of institutions belonging to the national education system.⁵

The United Kingdom Accreditation Service (UKAS) defines accreditation as

…. a formal, third party recognition of competence to perform specific tasks. It provides a means to identify a proven, competent evaluator so that the selection of a laboratory, inspection or certification body is an informed choice. UKAS accreditation means the evaluator can demonstrate to its customer that it has been successful at meeting the requirements of international accreditation standards.⁶

Accreditation is generally criticized for being “a collegial pat on the back” exercise, a scheme that focuses mainly on compliance with the minimum expectations, and for all the paperwork involved, it is much ado about nothing. These criticisms may be justified in some places, especially when they are government sanctioned and government managed exercises. However, with the progressive refinement and improvement of accreditation processes in the US, it is increasingly difficult to characterize the work involved as a trivial pursuit.

While accreditation exercise provides the public some level of assurance in the quality of the service and/or product offered by an accredited organization, the new emphasis on continuous improvement is the most promising aspect of the process.

Comparative Analysis of Ranking and Accreditation Processes

Determine Criteria

Assign Weights

Develop Questionnaire

Collect Data

Analyze Data

Figure 1. Typical Ranking Process

The process of ranking starts with the determination of relevant criteria. As mentioned earlier, some of the criteria are composite criteria with other sub-criteria imbedded. Beyond the initial step of determining criteria, assigning weights to each criterion is second most important step. The results of ranking exercise can change significantly as these weights are manipulated. Once the first two steps have been completed, the next step is to develop the questionnaire or instrument for data collection. The data collection phase involves determining who is competent to serve as respondents (in the case of the US News, they are Presidents, Provosts, and Vice President for Enrollment), determining the population, the sample size, the method of selecting the sample, and how respondents will be accessed. It is likely that the US News and World Report sends its questionnaire to all within the population. Once completed questionnaires have been collected, the data is analyzed and the ranking results generated. Although, ranking agencies produce their results annually, the process of completing the questionnaire by respondents takes less than 40 minutes.

Figure 2. A Typical Accreditation Progress

On the other hand, the process of completing an accreditation exercise usually takes a full year. The process starts with the Accrediting Agency usually called the Commission developing a set of standards that the higher education community believes are relevant to determining educational quality. A critical component of accreditation in the US is a formal training of those who serve as external evaluators. Training is completely absent in ranking. It must not be assumed that respondents would intuitively understand and agree on the many complex institutional descriptors and criteria.

Institutions or programs interested in accreditation first apply for eligibility, which requires meeting some initial criteria. If the Commission approves the application, the institution is provided with guidelines and in some places a mentor is also provided to guide the institution through the process. Institutions then begin the process of self-study, which entails a complete audit of all aspects of the institution (governance, staffing, curriculum, students, facilities, finance, reputation, etc.). The process of conducting and writing the self-study takes a full academic year in most places. Usually, an institution will set up a committee or task force charged with the responsibility of producing the report.

The self-study report is submitted to the Commission. Usually, the Commission would have determined a team of external evaluators, making sure there is no conflict of interest with any of the evaluators. The evaluators study the report and schedule a visit. The purpose of the visit is to verify the evidence that supports the institution’s claim. The external evaluator team submits its report to the Commission and the institution is presented with the opportunity to review the report and write its response to every comment and suggestions. The Commission after deliberating all the reports; renders its verdict. Institutions are generally accredited for 5 to 10 years. However, there is increasing emphasis on annual updates.

The two processes are incomparable in terms of complexity and intensity. Perhaps this is the reason why accreditation does not appeal to commercial organizations. There is too much work to do and the end result is not sensationalized.

Accreditation Standards of HLC and WASC

Table 8 presents standards by which two commissions, North Central Association of Schools and Colleges (NCA) and Western Association of Schools and Colleges (WASC), accredit their thousands of schools and higher education institutions. Ohio State is accredited by Higher Learning Commission (HLC) of the NCA, while Charles Drew University is accredited by WASC.

Table 8. NCA and WASC’s Accreditation Standards

*Higher Learning Commission	**Western Association of Schools and Colleges
Ohio State University Accreditation Decision: Accredited	Charles Drew University Accreditation Decision: Accredited
Criterion One: Mission The institution’s mission is clear and articulated publicly; it guides the institution’s operations. Core Components: 1.A. The institution’s mission is broadly understood within the institution and guides its operations. 1. The mission statement is developed through a process suited to the nature and culture of the institution and is adopted by the governing board. 2. The institution’s academic programs, student support services, and enrollment profile are consistent with its stated mission. 3. The institution’s planning and budgeting priorities align with and support the mission. (This sub-component may be addressed by reference to the response to Criterion 5.C.1.) 1.B. The mission is articulated publicly. 1. The institution clearly articulates its mission through one or more public documents, such as statements of purpose, vision, values, goals, plans, or institutional priorities. 2. The mission document or documents are current and explain the extent of the institution’s emphasis on the various aspects of its mission, such as instruction, scholarship, research, application of research, creative works, clinical service, public service, economic development, and religious or cultural purpose. 3. The mission document or documents identify the nature, scope, and intended constituents of the higher education programs and services the institution provides. 1.C. The institution understands the relationship between its mission and the diversity of society. 1. The institution addresses its role in a multicultural society. 2. The institution’s processes and activities reflect attention to human diversity as appropriate within its mission and for the constituencies it serves. 1.D. The institution’s mission demonstrates commitment to the public good. 1. Actions and decisions reflect an understanding that in its educational role the institution serves the public, not solely the institution, and thus entails a public obligation. 2. The institution’s educational responsibilities take primacy over other purposes, such as generating financial returns for investors, contributing to a related or parent organization, or supporting external interests. 3. The institution engages with its identified external constituencies and communities of interest and responds to their needs as its mission and capacity allow.	Criteria A: Organization Core Components: A.1. Vision and Purpose The school has a clearly stated vision or purpose based on its student needs, current educational research and the belief that all students can achieve at high academic levels. Supported by the governing board and the central administration, the school's purpose is defined further by expected schoolwide learning results and the academic standards. A.2. Governance The governing board (a) has policies and bylaws that are aligned with the school's purpose and support the achievement of the expected schoolwide learning results and academic standards based on data-driven instructional decisions for the school; (b) delegates implementation of these policies to the professional staff; and (c) monitors results regularly and approves the single schoolwide action plan and its relationship to the Local Educational Agency (LEA) plan. A.3-5. Leadership Based on student achievement data, the school leadership and staff make decisions and initiate activities that focus on all students achieving the expected schoolwide learning results and academic standards. The school leadership and staff annually monitor and refine the single schoolwide action plan based on analysis of data to ensure alignment with student needs. 4. A qualified staff facilitates achievement of the academic standards and the expected schoolwide learning results through a system of preparation, induction, and ongoing professional development. 5. Leadership and staff are involved in ongoing research or data-based correlated professional development that focuses on identified student learning needs. A.6-8. Resources 6. The human, material, physical, and financial resources are sufficient and utilized effectively and appropriately in accordance with the legal intent of the program(s) to support students in accomplishing the academic standards and the expected schoolwide learning results. 7. The governing authority and the school leadership execute responsible resource planning for the future. The school is fiscally solvent and uses sound and ethical accounting practices (budgeting/monitoring, internal controls, audits, fiscal health and reporting). [FOR CHARTER SCHOOLS ONLY] 8. The school has developed policies, procedures, and internal controls for managing the financial operations that meet state laws, generally accepted practices, and ethical standards. [FOR CHARTER SCHOOLS ONLY]
Criterion Two: Integrity: Ethical and Responsible conduct The institution acts with integrity; its conduct is ethical and responsible. Core Components: 2.A. The institution operates with integrity in its financial, academic, personnel, and auxiliary functions; it establishes and follows policies and processes for fair and ethical behavior on the part of its governing board, administration, faculty, and staff. 2.B. The institution presents itself clearly and completely to its students and to the public with regard to its programs, requirements, faculty and staff, costs to students, control, and accreditation relationships. 2.C. The governing board of the institution is sufficiently autonomous to make decisions in the best interest of the institution and to assure its integrity. 2.D. The institution is committed to freedom of expression and the pursuit of truth in teaching and learning. 2.E. The institution’s policies and procedures call for responsible acquisition, discovery and application of knowledge by its faculty, students, and staff.	Criterion B: Standards-Based Student Learning: Curriculum B.1. All students participate in a rigorous, relevant, and coherent standards-based curriculum that supports the achievement of the academic standards and the expected schoolwide learning results. Through standards-based learning (i.e., what is taught and how it is taught), the expected schoolwide learning results are accomplished. B.2. All students have access to the school’s entire program and assistance with a personal learning plan to prepare them for the pursuit of their academic, personal and school-to-career goals. B.3. Upon completion of the high school program, students will be able to meet all the requirements of graduation.
Criterion Three: Teaching and Learning: Quality, Resources, and Support The institution provides high quality education, wherever and however its offerings are delivered. Core Components 3.A. The institution’s degree programs are appropriate to higher education. 3.B. The institution demonstrates that the exercise of intellectual inquiry and the acquisition, application, and integration of broad learning and skills are integral to its educational programs. 3.C. The institution has the faculty and staff needed for effective, high-quality programs and student services. 3.D. The institution provides support for student learning and effective teaching. 3.E. The institution fulfills the claims it makes for an enriched educational environment.	Criterion C: Standards-Based Student Learning: Instruction C.1. To achieve the academic standards and the expected schoolwide learning results, all students are involved in challenging learning experiences. C.2. All teachers use a variety of strategies and resources, including technology and experiences beyond the textbook and the classroom, that actively engage students, emphasize higher order thinking skills, and help them succeed at high levels.
Criterion Four: Teaching and Learning: Evaluation and Improvement The institution demonstrates responsibility for the quality of its educational programs, learning environments, and support services, and it evaluates their effectiveness for student learning through processes designed to promote continuous improvement. Core Components 4.A. The institution demonstrates responsibility for the quality of its educational progra4.B. The institution demonstrates a commitment to educational achievement and improvement through ongoing assessment of student learning. 4.C. The institution demonstrates a commitment to educational improvement through ongoing attention to retention, persistence, and completion rates in its degree and certificate programs.	Criterion D. Standards-Based Student Learning: Assessment and Accountability D.1. The school uses a professionally acceptable assessment process to collect, disaggregate, analyze and report student performance data to the parents and other shareholders of the community. D.2. Teachers employ a variety of assessment strategies to evaluate student learning. Students and teachers use these findings to modify the teaching/learning process for the enhancement of the educational progress of every student. D.3. The school with the support of the district and community has an assessment and monitoring system to determine student progress toward achievement of the academic standards and the expected schoolwide learning results. D.4. The assessment of student achievement in relation to the academic standards and the expected schoolwide learning results drives the school's program, its regular evaluation and improvement, and the allocation and usage of resources.
Criterion Five: Resources, Planning, and Institutional Effectiveness The institution’s resources, structures, and processes are sufficient to fulfill its mission, improve the quality of its educational offerings, and respond to future challenges and opportunities. The institution plans for the future. Core Components 5.A. The institution’s resource base supports its current educational programs and its plans for maintaining and strengthening their quality in the future. 5.B. The institution’s governance and administrative structures promote effective leadership and support collaborative processes that enable the institution to fulfill its mission. 5.C. The institution engages in systematic and integrated planning. 5.D. The institution works systematically to improve its performance.	Criterion E: School Culture and Support for Student Personal and Academic Growth E.1. The school leadership employs a wide range of strategies to encourage parental and community involvement, especially with the teaching/learning process. E.2. The school is a) a safe, clean, and orderly place that nurtures learning and b) has a culture that is characterized by trust, professionalism, high expectations for all students, and a focus on continuous school improvement. E.3. All students receive appropriate support along with an individualized learning plan to help ensure academic success. E.4. Students have access to a system of personal support services, activities and opportunities at the school and within the community.

*http://ncahlc.org/Criteria-Eligibility-and-Candidacy/criteria-and-core-components.html

** http://www.acswasc.org/about_criteria.htm#cdecriteria

As indicated on Table 8, both Ohio State and Charles Drew University are accredited by their respective accrediting authorities. A careful review of the standards used for accreditation by the two commissions shows a remarkable similarity. However, one can see differences in emphasis. For example, WASC focuses more on student learning as the object of its evaluation.

Principles Derivable from Accreditation Standards

Accreditation standards are designed to encourage institutions to focus on quality, guide institutions in responding to quality matters, and ensure that attention to quality is comprehensive and strategic. Where the accreditation process and procedures are well embraced and reflected in institution’s daily activities, there is greater confidence in leadership, staff, and students. By reviewing the criteria of the two Commissions, it is possible to generate certain principles for quality assurance that should be applicable to institutions irrespective of location.

Principle 1: A quality-oriented institution is guided by a mission statement.

An organization’s mission provides the reason for existence. It makes sense, therefore, for quality assurance to start with ensuring that a mission statement exists, that it was deliberately, inclusively, and strategically developed. Above all, it makes sense that evaluators would be interested in the extent to which the mission guides other aspects of the university life and operations, and that the evidence is palpable and discernible to evaluators who visit the campus.

Principle 2: A quality-oriented institution engages in planning that reflects its mission.

Beyond the mission statement, a quality-minded institution would have a culture of planning, starting with a comprehensive institutional wide mission. A planning-oriented institution is an institution that introspects, examines its challenges and opportunities, audits its resources, forecast the future, sets goals, develops and implements strategies for results.

Principle 3: A quality-oriented institution links its budgets to its plans and its spending to its goals.

Institution’s budget is the financial interpretation of institution’s plan. What is planned but unfunded is an institution’s wishful list, but an organization’s budget reveals the institution’s commitment. A quality-oriented institution, therefore, is one that its budget is dictated by its plan.

Principle 4: A quality-oriented institution is governed by an effective Board that ensures institutional stewardship.

The highest governing authority of an institution in the US is the Board of Trustees. A quality-oriented institution would have an effective board, a board whose members are carefully selected and provided with the orientation and training to competently discharge their duties. An effective board provides the necessary stewardship and holds the institution in trust for the public. The board ensures that the institution fulfills its mission and without being overly intrusive, stays informed about critical aspects of the institution.

Principle 5: A quality-oriented institution demonstrates integrity and an ethical and responsible culture.

As a non-profit, service organization, an institution’s operations and activities are based on public trust. Therefore, a quality-oriented institution would demonstrate integrity in its internal and external interactions, and ensure ethical and responsible culture. The public must trust that the grades given are the grades earned, that the diploma issued carries the weight associated with it, and that their graduates had received the education promised by the institution.

Principle 6: A quality-oriented institution is led by professionals who are responsive and who hold the institution accountable to a publicly declared set of institutional indicators.

A quality-oriented institution is accountable to internal and external constituents. To be accountable, the institution identifies critical institutional indicators and reports its progress on these indicators annually. This implies that a quality-oriented institution is a data-oriented and data-driven institution. The president ensures that there is an effective executive team at the helm of the institution working with him or her; and together, they make sure that the campus culture is inclusive, dynamic, value-oriented, positive and conducive for academic pursuit.

Principle 7: A quality-oriented institution has academic affairs (teaching and research) as central to its operations.

A higher education institution is established for the sole purpose of knowledge discovery and transmission, any other consideration is peripheral and subordinate. Therefore, the central focus of a quality-oriented institution is the students, the faculty, and the interactions between these two. A quality-oriented institution focuses on the welfare and growth of the students and on the work and welfare of the faculty.

Principle 8: A quality-oriented institution demonstrates that it has adequate resources to accomplish its mission and sufficient for its scale of operation,

Excellence is not cheap. It costs money and resources. A quality-oriented institution ensures adequate resources to support its mission. An institution with a lofty goal but pathetic supporting resources cannot be trusted to deliver quality institutional performance. Quality-oriented institutions ensure facilities large enough to accommodate students and staff comfortably, and that are maintained to reveal attention to what is important. Dilapidated buildings and shabby physical appearance hurt the image of the institution and devalue whatever quality education being offered. Also quality-oriented institutions pay their faculty and staff adequately and ensure that allocation to instruction and research receives the priority it deserves.

Principle 9: A quality-oriented institution demonstrates continuous organizational learning and improvement culture.

A quality-oriented institution is a learning organization. Institutional improvement and effectiveness characterize the culture of a quality-oriented institution. The institution is supported by an active institutional research unit and operates a comprehensive assessment system. Quality-oriented institutions show growth over time and are able to provide explanation for lack of growth at any time.

Principle 10: A quality-oriented institution is engaged locally, nationally, and globally.

Principle 10 is not currently emphasized by accrediting agencies; at least not to the level that it should be in a globalized higher education environment. There is a growing demand for higher education to be relevant to society. While theoretical pursuits without regard to constraints are crucial, application of knowledge for the purpose of transforming society is also critical. Institutions that are engaged bring their curricula alive, infuse passion and purpose into their educational experience, and inspire a sense of relevance in their students and faculty. Global engagement has become a quintessential element of a quality-oriented institution in an increasingly global society. The idea that all politics are local is becoming a myth, economies are more integrated than ever, and global market is increasingly becoming our reality. Therefore, preparing graduates for global leadership is now a high priority for higher education institutions.

Principle 11: A quality-oriented institution enjoys academic freedom and voluntary accountability.

Where governments or governmental parastatals accredit institutions, the tendency toward bureaucratization and stifling academic freedom is high. The benefits of accreditation are enhanced when institutions participate voluntarily and where they enjoy a great deal of academic freedom. In the United States, while participation is voluntary, unaccredited institutions are denied some benefits, which include lack of access to federal government funded programs and initiatives. Consequently, obtaining accreditation is a high premium achievement for higher education institutions in the US.

Principle 12: A quality-oriented institution embraces a campus-wide culture of excellence.

These institutions do not wait for the cycle of reaccreditation before gathering data and preparing a self-study report. Rather, all campus systems are structured with the goal of documenting evidence that demonstrates fulfillment of the principles listed above. Consequently, external evaluators’ visit becomes an opportunity to confirm and perhaps congratulate the institution for a well established habit of continuous improvement.

hen Ranking Makes Sense

The US and World Report has another dimension (institutional efficiency ranking) to their best colleges ranking. This ranking provides a quantitative comparison of how much institutions spend to obtain their ranking points in the US and World Report Best Colleges. The efficiency ranking has an opposite interpretation of expenditure per student FTE done under best colleges rankings. Under the best colleges rankings, the higher an institution spends per student FTE, the higher the rank (all things being equal). However, under the efficiency ranking, the lower an institution spends, the higher the institution’s efficiency.

The US and World Report describes financial resources component and expenditures per student as follows:

Expenditures per student: Financial resources are measured by the average spending per full-time-equivalent student on instruction, research, public service, academic support, student services and institutional support during the 2012 and 2013 fiscal years.

The number of full-time-equivalent students is equal to the number of full-time students plus one-third of the number of part-time students. (Note: This includes both undergraduate and graduate students.)

We first scaled the public service and research values by the percentage of full-time-equivalent undergraduate students attending the school. Next, we added in total instruction, academic support, student services, institutional support and operations and maintenance (for public institutions only) and then divided by the number of full-time-equivalent students.

After calculating this value, we applied a logarithmic transformation to the spending per full-time-equivalent student, prior to standardizing the value. This calculation process was done for all schools. 

If a school submits fewer than two years of expenditures per student, then the average is based on the one year that is submitted.

Higher average expenditures per student score better in the ranking model than lower average expenditures per student. In other words, financial resources do matter in terms of being able to provide students with a high-quality college experience.⁷

Explaining the importance of the Efficiency ranking, the US and World Report stated that:

…For this analysis, U.S. News looked at the public and private colleges that scored the highest on overall undergraduate academic educational quality, as measured by their position in the 2015 Best Colleges rankings, but that spent relatively less on their educational programs to achieve that quality.

Amid restricted growth in many state budgets to fund higher education and increased public scrutiny about the rising cost of going to college, it's vitally important for many colleges to efficiently spend their limited resources to produce the highest possible educational quality.

U.S. News measures financial resources by taking into account how much a school spends per student on instruction, research, student services and related educational expenditures. The financial resources indicator has a 10 percent weight in the Best Colleges ranking methodology.

The lists [above] are based on operating efficiency, which U.S. News has defined as a school's 2013 fiscal year financial resources per student divided by its overall score – the basis U.S. News uses to determine its overall numerical rank – in the 2015 Best Colleges rankings.

This calculation reveals how much each school is spending to achieve one point in its overall score and thus its position in the rankings. The premise of the analysis is that the less a school spent relative to its position in the overall rankings, the more efficient it was in its ability to produce a top-quality education.

Schools that are featured on these lists are doing a good job in managing their financial resources relative to other schools that may have far greater financial resources because of more state funding, higher tuition or larger endowments. In the National Universities category, many of the schools listed are likely to be more affordable in terms of tuition than others in the same category, since most of them are public universities.⁸

Table 9. The Ten Top Ranked Most Efficient National Universities⁹

Institution	U.S. News National Universities Rank	Financial Resources Rank	Spending per student for each point in the U.S. News overall score
Miami University--Oxford	76	205	$383.66
Florida State University	95	214	$392.77
University of Alabama	88	198	$423.02
Binghamton University--SUNY	88	185	$437.23
College of William and Mary	33	110	$441.82
Brigham Young University	62	156	$457.29
Indiana University--Bloomington	76	156	$469
Clemson University	62	138	$486.02
University of Missouri	99	171	$499.61

It is interesting that none of the institutions on Table 9 is in the top 30 on the US News and the World Report ranking. The closest to the top is the College of William and Mary, ranked 33. The University of Missouri spends over $100 more on each student than Miami University-Oxford to earn their spots on the US News ranking. The efficiency ranking offers a better promise than the Best Colleges rankings in focusing institutions’ attention to the rising cost of higher education and to debunk the myth that more money produces higher quality. However, the final results of efficiency ranking are based on the overall score of the best colleges rankings, which is fraught with subjective weights and questionable factors.

Conclusion

Attention to quality assurance in higher education will continue to increase in an increasingly global society where borders are becoming porous, technology is integrating systems at a global scale, and institutions are harmonizing academic programs through international joint-degree collaborations. Realizing the growing general public demand for schemes to differentiate quality institutions from struggling institutions, the ranking industry has risen to the challenge. However, what the ranking industry offers is at best a poor surrogate for quality. Higher education is a complex enterprise and any attempt to rank institutions without taking into consideration their complexity, should be viewed suspiciously.

The consequences of ranking for two institutions, Ohio State and Charles Drew University, reveal the challenges of a simplistic approach to institutional differences. In fact, the ranking industry does greater damage to some low-ranked and unranked institutions without truly helping even the top ranked institutions.

The danger of institutional comparison lies in the fact that some unique advantages of an institution may be neglected, while irrelevant factors are amplified. Is it conceivable that an institution such as Charles Drew University would have benefits above an institution such as Ohio State? The answer is yes! Charles Drew University is a specific service-mission driven institution. This offers several advantages, which include focused educational experience, cultivation of altruistic culture, channeling student and faculty energy toward societal need, and reducing the temptation of an institution to be all things to all people. The small school environment makes it hard for students to fall through the cracks without someone noticing. Such an environment provides the best context for disadvantaged students who require a more intimate pedagogical approach to succeed. What weights can rankings possibly place on an institution’s effort to serve and meet the needs of the underserved populations?

Accreditation, on the other hand, offers a better promise in addressing institutional complexity and in focusing attention to quality matters. However, accreditation must continue to evolve from prescribing the minimum standards for compliance to putting greater emphasis on institutional adopted agenda for improvement. By focusing on an agenda for improvement, it is possible to identify a set of universal principles that can be adopted by all institutions, irrespective of mission, size, location, and wealth. The ten principles identified in this paper can serve as the foundation for structuring the work of the Association for the Global Advancement of Universities and Colleges in its quest for a global accreditation system.

Online Sources

http://carnegieclassifications.iu.edu/.

http://www2.ed.gov/about/offices/list/ous/international/usnei/us/edlite-accreditation.html

http://www.ukas.com/aboutaccreditation/What_is_Accreditation/What_is_Accreditation.asp

http://www.usnews.com/education/blogs/college-rankings-blog/2015/01/15/data-show-which-top-ranked-colleges-operate-most-efficiently

http://www.usnews.com/education/best-colleges/articles/2014/09/08/how-us-news-calculated-the-2015-best-colleges-rankings?page=2.

http://www.usnews.com/education/best-colleges/articles/2014/09/08/best-colleges-ranking-criteria-and-weights

Refbacks

• There are currently no refbacks.

This work is licensed under a Creative Commons Attribution 4.0 International License.