Another sad story about outcomes assessment

Kevin Carey has written a number of very compelling articles relating to the use of outcomes measures in higher education(see Educational value added, Sept.1, 2006) .  One of his more recent was an op-ed, Blocking Public Comparisons Obstructs Knowledge, Too, in the Chronicle of Higher Education.   His “case study” for this article is the State University of New York at Buffalo, which apparently is suffering mightily from the economic downturn.  The President, John B. Simpson is turning to the legislature for both increased aid, and the ability to raise tuition.  However, as Carey points out:

Yet even as the SUNY system lurches toward financial crisis, it has squandered a golden opportunity to make its case to the taxpayers and their elected representatives, to demonstrate success in doing what the majority of those people care most about: helping students learn. By refusing to provide public, comparable measures of student-learning results, New York's great public-university system has sown the seeds of long-term marginalization. In that, too, it has plenty of company nationwide.

The situation facing President Simpson is particularly difficult because in 2003 the SUNY Trustees passed a resolution calling on the SUNY system to come up with uniform “before-after”measures of student attainment of general education goals. In other words, value added measures in the core area of general education.

What Carey describes as happening after that is not surprising -nor is it the type of response that is limited to the SUNY system:     

But in the year following the resolution, SUNY leadership was subjected to intense pressure from New York's Faculty Senate and other interest groups opposed to the plan. As a result, SUNY put in place a watered-down scheme in 2004. Value-added estimates of learning growth were no longer required. Instead of common systemwide measures, every university would choose its own standards, tests, and sampling procedures, making institutional comparisons difficult. They would also be made impossible because campus-specific data would be "used for confidential in-house discussions." The results, it was stressed, should never be used to "punish, publicly compare or embarrass faculty, courses, programs, departments, or institutions either individually or collectively.

Unfortunately, as Carey points out, even the resulting hodgepodge of measures contained some embarrassing results for specific programs, so additional efforts were required to water down the program.  He reports that a system task force studying implementation of the assessment program stated:

Institutions responded very strongly against the requirement that institutions must report to System Administration the percentage of students who 'exceed, meet, approach, or fail to meet standards

Apparently the solution is, if there is a problem, it will go away if we don’t tell anyone.  This solution then formed the System approach to the legislature:

Having insisted that institution-level results be incomparable, and thus of little value, the SUNY institutions proceeded to argue that the results shouldn't be reported to state officials, on the grounds that they were incomparable, and thus of little value.

The moral of this sad tale, according to Carey, is that President Simpson is now in a very weak position as he goes hat in hand to the legislature.  He can’t show data that will demonstrate that his institution is doing a good job in educating students.  In fact, legislators probably know that his institution has worked to assure that data on learning is not available.  Unfortunately, this conclusion would hold for most of the institutions of higher education in the country. 

At the same time that traditional higher education is strongly resisting outcome measures, many of the for-profit institutions of higher learning are putting considerable effort and resources into developing sophisticated learning outcome measures for their institutions.  For them, success on these measures will be the opening to wider societal acceptance and thus greater competitive position. There is a very high probability that in a few years we will find ourselves in a world in which the accepted learning outcome measures will have been created by the for-profit institutions - because the traditional institutions refused to get into the game seriously until it was too late.   The only way to assure that the future accepted measures focus on aspects of higher education that the traditional institutions hold most dear is for them to begin to develop those measures - not to stubbornly deny that such measures can be developed.