Kevin Carey has written a very provocative piece for the Washington Monthly called Is our students learning?: The measurements elite colleges don’t want you to see. In it, he addresses the issue of measuring the educational value added by an institution. Although the article is addressed to the “elite” institutions, it is clear that his points apply to all of higher education.
Carey asks us to imagine reading a “best mutual fund” guide that does not include the bottom line of rate of return. Most of us would find that an unacceptable guide for investing our retirement funds, but, Carey argues, that is pretty much the kind of guide we use when choosing a college. While acknowledging that finding the bottom line for education is more complex and difficult than finding the bottom line for a mutual fund, Carey makes a number of interesting proposals.
First, he proposes using standardized testing. Subject matter testing is not particularly useful, he suggests, because of the many different pathways students take through their education. However, he does propose testing “ the higher-order critical thinking, analysis, and communication skills that all college students should learn (and which employers value most).” For this, he proposes using the Collegiate Learning Assessment, a test developed by the Council for Aid to Education and RAND that is currently used by a few hundred institutions. Second, he points out that educational research has indicated the teaching characteristics and environments that are most likely to lead to effective learning. A student survey has been developed to measure these indicators, the National Survey of Student Engagement, and Carey proposes this as his second component of the bottom line. Finally, he proposes looking at what happens to graduates of particular institutions. He proposes linking educational records held by higher education institutions with governmental records on income and field of employment. These data - with all identifiers removed - could give prospective students a better idea if a particular institution is a good match for their aspirational goals - and worth the price.
The last of these proposals seems to me to be the most problematic on several grounds. First, having watched the government introduce several rather complex higher-ed related computer programs, I am not sure that it is capable of making this happen without enormous pain and cost to everyone. Second, so few students actually grow up to do what they think they wanted to do when they were 18, that it may well not be beneficial to know enough about institutions to be able to pick the one that most closely matches that 18 year-old vision. However, there is no arguing that having data that backs up (or refutes) recruitment claims ("we are very successful in producing graduates who work in an NGO,.. on Wall Street,.. in Hollywood,.. as neuroscientists, etc.") would be very valuable.
As for the first two proposals, I am quite supportive, with the caveat that I am not an expert on either of the instruments. Certainly, neither instrument is perfect, but what I read about them is very positive, and it looks like both provide useful information for both prospective students and the institutions themselves. I have previously commented on higher education’s shortcomings in some of the areas tested by these two instruments in How are we teaching cognitive skills?, July 4, 2006, and How people learn, May 1, 2006. I think it ultimately it would be very good for higher education if the focus of the consumer were turned towards these more fundamental aspects of education.
Carey also calls attention to the work of the National Center for Academic Transformation. Over recent years, this Center has supported a number of course redesigns based on use of technology. The challenge posed by the Center was that the new courses should be both more effective from the learning perspective, and less costly then were the predecessor course. The success record has been quite high thus far.
Carey closes with a warning that the lack of output information on higher education is beginning to cause a serious backlash in Washington, and in the public. His words are well worth considering.
Interesting perspectives on Carey’s article can be found in blogs by Dr. T and Matt Johnston. It deserves even more comment.
Comments