Fitting a square disruptive peg into a round sustaining hole

“The difficulty lies, not in the new ideas, but in escaping from the old ones, which ramify, for those brought up as most of us have been, into every corner of our minds

John Maynard Keynes

 

The Higher Education Reconciliation Act (HERA) of 2005 opened the door to competency based degrees by stating that 

instructional programs that use direct assessment instead of credit or clock hours to measure student learning may qualify as a Title IV-eligible program if the assessment is consistent with the school’s or program’s accreditation.

 Since that time, the Department of Education (DOE) has moved extremely tentatively towards approval of competency based degrees, approving only three thus far.  Recently we learned that the Inspector General (IG) for the Department of Education  has found that the Department's extremely tentative moves towards accepting competency based degrees were not tentative enough: 

We found that the Department did not adequately address the risks that schools offering direct assessment programs pose to the Title IV programs and did not establish sufficient processes to ensure that only programs meeting Federal regulatory requirements are approved asTitle IV eligible.

Although numerous issues are raised in the report, the key problem seems to be how to convincingly demonstrate the number of credit hours to which the competency based program is equivalent - although it is not obvious to me that congress suggested that demonstration was necessary.

The IG's report emphasizes for me some of the otherworld Alice-In-Wonderland aspects of the official discussions about competency based degrees. In this post, I lay out some of these aspects, and then consider why they occur and what they imply.

******  

The credit hour is, of course, a well established concept in higher education - a complicated consequence of Andrew Carnegie's requirements for a college to participate in TIAA. This 100 year old concept is firmly embedded in the processes of the higher education business model, and even more important, is the key metric used in determining eligibility for Federal financial aid.  Just to be sure we know how the credit hour is defined, the DOE provided a definition in its 2010 Program Integrity Rules:

one hour of classroom or direct faculty instruction and a minimum of two hours out of class student work each week for approximately 15 weeks for one semester or trimester hour of credit, or ten to twelve weeks for one quarter hour of credit, or the equivalent amount of work over a different amount of time

Thus, since the "average" student takes about 15 credit hours per semester, this definition states that that student should be in class or studying for at least 45 hours each week. This understanding underlies the awarding of Federal student aid, and that presumably is what the government thinks it is paying for.

The concern about competency based work is that something will slip through the approval process that lets an institution (call it a Correspondence Course, as does the IG) and its students cheat the Federal government by awarding credit for courses that aren't rigorous enough to demand at least three hours per credit of immersion in study and learning each week.

There is a problem with this picture, of course. The National Survey of Student Engagement (NSSE) does an annual survey of college students. One of the results that gets enormous coverage in the popular press (but apparently not anything the people in DOE read) describes how many hours per week the average student is involved in academic work. That number has continued to drop slowly over the years, and now is around 30 hours a week. Faculty also get surveyed, and it turns out that is about what they expect. So the actual "credit hour" definition from the academic side is really 1 hour in class, 1 hour of study per week - 1/3 less than the official definition.

So my first Alice in Wonderland point is: 

the credit hour, whose definition was recently re-sanctified by the DOE in order to give it the appearance of  precision, is in fact a fictional quantity that has no existence in the real world, and the difference between reality and fiction increases every year. 

Nevertheless, one of the IG's specific concerns is that :

A school might develop credit- or clock-hour equivalencies for the program that are not based on the regulatory definition of a credit or clock hour.

My second Alice in Wonderland point follows directly from this and the data above:

The IG is demanding that the direct assessment course be equal in quality to a clock-hour course that doesn't generally exist in higher education any more, i.e. one that follows the official definition of clock hour. Thus, the direct assessment course should be 1/3 better than the average traditional course in order to be approved. Possibly a good step to raise quality, but hardly a fair standard.

Finally, since we are all undoubtedly pleased that the IG wants to stamp out fraud against the US Government, I am led to a third Alice in Wonderland point:

while the IG is focusing on the possibility that fraud might be introduced by the move to competency-based learning, he or she is ignoring a massive misconception in the current system: the government currently thinks it is supporting students who work full time at their studies, and it gets students for whom study is a part-time job! Should we investigate the status quo for fraud on a massive scale?

Moving to a more fundamental point, consider the issue of finding the credit hour equivalent of a competency based course. These two approaches are 180 degrees out of sync with each other: the one counts seat time, the other head success. Credit hours describe the hours of academic immersion , but contain no mention of course effectiveness (faculty competence, course rigor, etc.), student attention or attendance at the classroom hour, or learning outcomes. Competency based courses describe a set of learning outcomes that students are expected to achieve, but do not constrain how students will achieve those competencies or how long it will take. To say that one approach is apples and the other is oranges is to underestimate the conceptual and practical differences between these approaches.   

Thus, my fourth Alice in Wonderland point is:

can the DOE create a logical system that will tell us with some accuracy how many apples equal an orange, thus satisfying the IG?

There are a number of other off-the-track issues in the IG's report, such as requirements of student faculty interactions that more closely parallel the interactions that occur in the current credit-hour approach. The comments generally do not reflect the actual current situation in higher education, or an understanding of how direct assessment programs would support student success. 

The IG is not the villain of this piece, of course. The IG is generally just judging whether the DOE has faithfully followed its own rules. Thus, the underlying problem is that the DOE has promulgated rules to implement the HERA that lead us down the rabbit hole into an alternative world.

******

The real question is why seemingly intelligent people set up situations that demand precise answers to nonsensical questions. The answer can be found in some of Clayton Christensen's work on disruption. In Christensen's terminology, a sustaining innovation is one that can be used to improve a product without changing the game, that is, the underlying business model of the enterprise.  However, a disruptive innovation is one that can change the game, that is, finds its power in a very different business model that is optimized around the innovation.  I discussed innovation and the  higher education business model in How can we think about the wave of new innovations in higher education?

Christensen points out that in many cases, an innovation can be used in both sustaining and disruptive ways, with very different outcomes. For an innovation to be used disruptively, it must be incorporated into a new business model that is designed to optimize the power and effectiveness of the innovation. On the other hand,  the innovation is unlikely to be taken into an existing business model unless it can be used in a sustaining way. That is, it must be used in a way that produces some added value but does not disturb significantly the existing business model. In this case the full potential of the innovation can't be realized because the existing business model was created to optimize a different set of resources and procedures.  

Direct assessment programs present an enormously powerful new Process for higher education. This approach could lead to changes in faculty duties, workload, and training. It could obviate the need for semesters, and remove the (also Andrew Carnegie "mandated") four-year time to a degree. Implications for existing expensive physical plant could be enormous.  In sum, an educational program optimized around direct assessment could look very, very different from the traditional model - it would reflect an entirely different business model. 

As Christensen would tell us, bringing such a powerful innovation into our existing business model for higher education requires that direct assessment be tamed - molded and confined such that it fits into our present structures without destroying them.  The explains the awkward requirement that direct assessment be demonstrably equivalent to existing credit hours, even though they are enormously different concepts, and the misplaced worries that traditional student-faculty interactions be maintained. Both of these are requirement that help to "shoehorn" direct assessment into higher education in a sustaining way. 

I wrote about accreditation in a recent post, Accreditation:ally or obstacle as education wrestles with change?. In that article, I concluded that accreditation as it exists today is intrinsically structured in such a way as to encourage sustaining innovation, while suppressing disruptive innovation. The interesting aspect of this DOE action is that it reveals how fundamentally the DOE is immersed in a deeply traditional mindset that leads to the same behavior - support of sustaining innovation, suppression of disruption.

My earlier post concluded that the only way we would see disruptive change in higher education was if we created an alternative system of accreditation. I now understand that we also need to create an official US Department of Disruptive Education in order to see the creation of alternative approaches to education that are less expensive, more effective, and more attractive to students who feel their needs are underserved by the limitations of the current approach. 

Comments

[jackd. rollins]

My reaction to this issue is easily put: (forget about credit hours--they are meaningless as any form of measurement, other than clock time) a grade is, and only should be, a reflection of the extent to which a student has mastered the material in the syllabus. You cannot grade a student on something that you have not taught him. It is his/her competency that should be the sole criterion for evaluation.

Lloyd comments: I agree with one caveat. The typical college syllabus gives very little information regarding the actual learning outcomes that are expected. Direct assessment clearly defines the learning outcomes expected for a course, and focuses on whether the student has mastered the material, rather than on how the student learned the material.