Learning outcomes information and the quality of higher education

But the biggest problem with American higher education isn=t that too many students can=t afford to enroll. It=s that too many of the students who do enroll aren=t learning very much and aren=t earning degrees. For the average student, college isn=t nearly as good a deal as colleges would have us believe.  Kevin Carey

 

 Kevin Carey, one of the most articulate critics of American higher education, has a powerful new article entitled A That Old College Lie@  in the Winter 2010 issue of Democracy.  (Some of my previous posts about Carey=s work are: Sept.1, 2006; Oct.11, 2006; April 13, 2009).  Richard Vedderer also has an excellent recent post talking about Carey=s article.

Carey begins by writing about Pell grants, and the plans of the Obama administration to increase them. He points out that Pell grants have become less important over time as college costs have increased much more rapidly than CPI.  As a consequence, student borrowing is increasing, as are loan default rates.  With this, he segues into his main theme, which is given by the quote that begins this post.  He argues that students at all levels of colleges and universities are not learning all that they could be, and that Aquality@ of American higher education is not what we imagine or claim it to be.  He rightly points out that what we (and much of the world)  think of as the Ahigh quality@ of the American system is Adriven by the top 10 percent of institutions and the students who attend themBHarvard, Stanford, MIT, and the like.@ 

Carey argues, however, that we actually know very little about how good our institutions are at helping their students to learn.  Publicly available data on that subject is almost nonexistent.  As a consequence, Aquality@ is actually primarily Aprestige@, and prestige is defined through combinations of Awealth, admissions selectivity, price, and a generalized sense of fame that is highly influenced by who=s been around the longest and who produces the most research.@  Learning is just simply left out of the mix that defines quality in higher education.   However, if comparable information on teaching and learning were readily available, a number of problems would be addressed.  For example, this information would enable newer institutions to compete more effectively with the old guard, thus increasing competition and leading to greater cost controls. A resulting customer focus on teaching and learning would lead colleges to allocate their more of their resources to that end, rather than, e.g. high priced professors who don=t teach.   

Carey point out that some groups have recently begun to collect information on learning outcomes that would greatly aid parents and students in making informed choices when picking a college - and would help us to give a new and important redefinition of quality.   However, he argues that this information in being held captive by the colleges themselves, and by educational lobbies that represent the colleges.  One has only to review some of the submissions made by such organizations as ACE and AAU against the outcomes focus of the Spellings report to see the merit in Carey=s argument. 

Finally, Carey tells us what needs to be done:

The Obama Administration has proposed huge new increases in Pell Grants and other higher education programs, amounting to more than $70 billion over the next decade. It should require institutions receiving these funds to provide more information to the public in exchange. It should invest in R&D to develop new methods of gauging student success. And it should be prepared to fight a scorched‑earth political battle against the entrenched special interests that will, if history is any guide, surely rise in opposition.

None of these points will surprise readers of this blog.  However, Carey says it all very well, and covers a lot of important points I have not mentioned in this post.  The one issue he did not cover that I think is missing is the Bologna process, which focuses on learning outcomes in higher education (see The Bologna Process- a significant step in the modularization of higher education Sept 12, 2008).  The Bologna process now involves 46 European nations, and dozens of institutions in South America.  Our refusal to seriously consider learning outcomes risks leaving us outside the next big global movement in higher education. This could result in the US losing its leadership role higher education, and becoming a follower with lots of catch-up to do.  

Carey=s solution is one that the Obama administration really should consider- this is an unusual opportunity to move American higher education to a more effective plane, increasing our ability to cope with the changes going on in the wider world. 

Comments

[Lloyd Armstrong]

Thanks Chris for a great comment. Unfortunately, it has so many great threads it is hard to respond to succinctly. A great challenge of outcome measures is, indeed figuring out what to measure. The measures most used now look at critical thinking and communication skills, with an eye to seeing how these improve over 4 years. None are looking at learned information per se, although that will be a secondary part of the Bologna approach. Those measures, and NESSE, generally do not support your assumption regarding the differences between "the highest institutions and the lowest". In fact, some of the "lowest" show the greatest improvement in critical thinking, and highest involvement. Finally, I analyzed the same data as the Washington Post in http://www.changinghighereducation.com/2009/11/the-elephant-of-college-pricing.html and came to a very different conclusion. Much of the aid is in the form of loans. The statement of dropping cost of tuition is true only if one views the loans as free. If one includes the loans in the cost, it has gone up 32% in real $'s over the 15 years of the report quoted.

[Chris Aldrich]

I agree with Arnaldo that Quality Control might be an effective means of increasing the overall quality of the system, however one will need to be careful not to place too much emphasis on just the factual content in defining a Learning Object – arguably the biggest part of an education isn't just learning the facts, yet being able to approach things from a larger holistic whole. The sum of the facts learned are not necessarily the whole of the education - in fact, it's learning how to learn, cope, adapt, and extend in the future that is one of the greatest factors in the educational process. Utilizing the simplest concept of LOs, may pander to the lowest common denominator, and on this basis the local community college would be on par with the highest research institutions. One would have to take into account the learning atmosphere, the surrounding conversations and interactions, the level of the competition, and the learner's peers as additional benchmarking in such a quality control process.

One of the largest disparities between the highest institutes and the lowest is, in part, the desire to learn and take the most advantage of the opportunities granted at an institution. Towards this end, I'm curious to hear everyone's thoughts on the concept of what the National Survey of Student Engagement (NSSE)(http://nsse.iub.edu/) is doing. Today's article in the Washington Post on its rankings seems like a very apropos one given the lack of real benchmarks in many of the other rankings and surveys. http://www.washingtonpost.com/wp-dyn/content/article/2010/02/14/AR2010021402968.html

The other major factor to take into consideration is the cost of the process and potentially a measure of cost per unit quantity. The problem here is the "per unit quantity" portion of the equation which needs some significant work to accurately define. Event the general economics of the cost part of the structure are in wide question as last week’s Washington Post points out at: http://www.washingtonpost.com/wp-dyn/content/article/2010/02/09/AR2010020903832_pf.html. This article essentially reports that though the price tag of an education is rising, the actual paid cost by individual students has decreased over the past several years. Does this portend the coming of a Wal-mart (or possibly even a Costco) Educational system?

Is anyone aware of educators taking the principles of JM Juran, W.E. Deming, et al. and applying them to education instead of manufacturing? Given that, since the start of the industrial revolution, the delivery of an education is one of the few processes that hasn't improved exponentially as other areas of "manufacture" have, will it really be improving the means of measuring delivery and imposing more quality control that will make it easier and less expensive to deliver an education?

[Arnaldo Ghersi]

Dear Friends:
I believe the disruptive innovation in the Educational industry will occur when we start implementing a Quality Assurance System.
Education, as any other continuous process industry, needs to implement a quality control system; a Total Quality Assurance. There is a big difference between having a QC system, and measuring the quality of a given education. While the second choice provides metrics after the process has been completed, the first choice offers the managers (the teachers) the possibility to act according to the responses and make the necessary changes to achieve the desired quality. A QC system requires metrics in real time; a continuous evaluation.
The other thing that the Education industry needs is a “measurement” system. Not a set of standardized tests. A QC system uses standard units to measure the different steps of a process. The concept of Learning Objects, used in the e-learning industry as standard of content, can easily be adopted by the Educational industry to measure the courses. Instead of using “credits” or “units”, LOs could be used to quantitatively and qualitatively measure the syllabus contained in a curriculum. Correlating content between institutions would be just a matter of matching the LOs of the desired curriculum. LOs also could be used for internationalization of career titles or certificates.
This Virtual platform will potentiate teachers’ capacities to become “teaching managers”. I would like to paraphrase Deming: Teachers who will work ON the system, monitoring study performance data of individuals, and correcting their weaknesses on-time, will achieve the desired knowledge (quality).

Lloyd responds: great comment. I agree that disruptive innovation will hit its "tipping point" when we get some valid outcomes measures (or your LO's) that enable comparison between approaches and institutions.