Changing Higher Education

Is online learning ready to become a disruptive technology?

In 2000, I wrote an article in Change discussing major disruptive impacts on higher education that distance learning might produce (see, How about distance learning, March 3, 2006).  In it, I brashly postulated that:

The experience will certainly be different from that found in the classroom of a great teacher, but in the end DL may well provide a competitive or even superior way to learn.

A recent analysis by the Department of Education of learning outcomes achieved by various on-line learning courses compared to those of traditional courses finds that my “in the end” may actually be now!

The DoE identified over a thousand empirical studies of online learning between 1996 and 2007. From these studies, they chose a set to subject to meta- analysis with the goal of answering 4 research questions:

1. How does the effectiveness of online learning compare with that of face-to-face instruction?
2. Does supplementing face-to-face instruction with online instruction enhance learning?
3.. What practices are associated with more effective online learning?
4. What conditions influence the effectiveness of online learning?

The analysis selected from all of the empirical studies using three stringent conditions:

Limit the search to studies of Web-based instruction (i.e., eliminating studies of video- and audio-based telecourses or stand-alone, computer-based instruction);
Include only studies with random-assignment or controlled quasi-experimental designs; and Examine effects only for objective measures of student learning (e.g., discarding effects for student or teacher perceptions of learning or course quality, student affect, etc.).

The results of the meta analysis are both impressive and thought provoking :

•    Students who took all or part of their class online performed better, on average, than those taking the same course through traditional face-to-face instruction.
•    Instruction combining online and face-to-face elements had a larger advantage relative to purely face-to-face instruction than did purely online instruction.
•    Studies in which learners in the online condition spent more time on task than students in the face-to-face condition found a greater benefit for online learning
•    Most of the variations in the way in which different studies implemented online learning did not affect student learning outcomes significantly
•    The effectiveness of online learning approaches appears quite broad across different content and learner types
•    Effect sizes were larger for studies in which the online and face-to-face conditions varied in terms of curriculum materials and aspects of instructional approach in addition to the medium of instruction

There are many data, and numerous caveats, behind each of these conclusions.  I strongly recommend a reading of the report to better understand what these conclusions mean.  However, a couple of these results deserve some comment here.  First, the result that online students who spent more time on task than students in face to face conditions learned more is at some level somewhat of an obvious outcome. More time spent, more material learned.  However, this response involves an acknowledgment of relatively equal time- learning efficiency coefficients for both approaches, which would be a step forward for some.  It also begs the very interesting question of why the online students were motivated to spend more time on task than their classroom peers.

The last of the results above regarding effect size emphasized that online and face to face courses had the more nearly equal learning outcomes when the courses were most similar.  Thus, since online courses seem to be more effective in general, most online courses are differentiating themselves from the face to face courses in ways that lead to increased learning outcomes.  Unfortunately, the data are not robust enough to produce many clear differentiating factors. There is too much “context”hidden in the data  - undefined differences in instructor behavior and content.  Time on task does emerge as important, as do methods that encourage student reflection.

All in all, this report provides both a powerful signal (warning?) that online learning is rapidly growing up, and a call for additional well controlled studies that will show how to further increase the effectiveness of the online experience.  It also raises some interesting policy questions regarding various regulations that view online courses as necessarily being of lower quality than face to face courses.

Bologna finally comes to the US

Thanks to the Lumina Foundation, an exciting educational experiment is underway. InsideHigherEd reports that Lumina is leading a US project that applies the  “Tuning” approach of the Bologna process to  several different undergraduate majors.   Numerous higher education institutions of differing size and mission in Utah, Indiana, and Minnesota are participating. 

The Bologna process, overall, tries to bring some consistency to the meaning of degrees around the Bologna region in order to facilitate movement of students around the region, and acceptance of degrees by employers (see The Bologna process - a significant step in the modularization of higher education, Sept 12, 2008). At the same time, the process does not seek to challenge the differences in approach and viewpoint that characterize the various member states. Thus, there is agreement on the intellectual capacities that should describe someone who has attained a degree of a certain level, but no limitations on the approach that got the student to that point.   Clifford Adelman has just published another detailed and very insightful report on the process,  The Bologna Process for U.S. Eyes: Re-learning Higher Education in the Age of Convergence. Highly recommended.

The Tuning approach is the discipline specific part of this process. The approach seeks to create guidelines that faculty can use as they develop statements of expectations for such things as learning outcomes and  levels of learning for individual disciplinary degrees.  The process involves surveys of graduates, employers, and academics to get a clear picture of the learning outcomes that should be expected of specific disciplinary degree programs in the 21st century.  The process has moved along well in Europe, and has led to the formation of a Latin American Tuning Process that now involves institutions in about 18 Latin American countries.  High time the US joined the world and experimented with the approach! 

It is disappointing, but no surprise, that InsiderHigherEd reports that Cary Nelson, president of the American Association of University Professors, is concerned that all of this could provide a threat “academic freedom”.  This is the usual argument advanced to derail attempts to define desired learning outcomes or to measure them. However, at one level, Nelson is certainly correct.  If faculty do not participate fully and creatively in helping to develop appropriate and meaningful statements of desired learning outcomes, someone else will eventually impose standards - and those externally  imposed standards quite possibly will provide a threat to academic freedom. 

Another sad story about outcomes assessment

Kevin Carey has written a number of very compelling articles relating to the use of outcomes measures in higher education(see Educational value added, Sept.1, 2006) .  One of his more recent was an op-ed, Blocking Public Comparisons Obstructs Knowledge, Too, in the Chronicle of Higher Education.   His “case study” for this article is the State University of New York at Buffalo, which apparently is suffering mightily from the economic downturn.  The President, John B. Simpson is turning to the legislature for both increased aid, and the ability to raise tuition.  However, as Carey points out:

Yet even as the SUNY system lurches toward financial crisis, it has squandered a golden opportunity to make its case to the taxpayers and their elected representatives, to demonstrate success in doing what the majority of those people care most about: helping students learn. By refusing to provide public, comparable measures of student-learning results, New York's great public-university system has sown the seeds of long-term marginalization. In that, too, it has plenty of company nationwide.

The situation facing President Simpson is particularly difficult because in 2003 the SUNY Trustees passed a resolution calling on the SUNY system to come up with uniform “before-after”measures of student attainment of general education goals. In other words, value added measures in the core area of general education.

What Carey describes as happening after that is not surprising -nor is it the type of response that is limited to the SUNY system:     

But in the year following the resolution, SUNY leadership was subjected to intense pressure from New York's Faculty Senate and other interest groups opposed to the plan. As a result, SUNY put in place a watered-down scheme in 2004. Value-added estimates of learning growth were no longer required. Instead of common systemwide measures, every university would choose its own standards, tests, and sampling procedures, making institutional comparisons difficult. They would also be made impossible because campus-specific data would be "used for confidential in-house discussions." The results, it was stressed, should never be used to "punish, publicly compare or embarrass faculty, courses, programs, departments, or institutions either individually or collectively.

Unfortunately, as Carey points out, even the resulting hodgepodge of measures contained some embarrassing results for specific programs, so additional efforts were required to water down the program.  He reports that a system task force studying implementation of the assessment program stated:

Institutions responded very strongly against the requirement that institutions must report to System Administration the percentage of students who 'exceed, meet, approach, or fail to meet standards

Creativity and the Research University:II

This is a continuation of Creativity and the Research University
II. The creative faculty

The issues are different here from those encountered in looking at student creativity, because we are dealing with a class of accomplished scholars who have already shown capacity for creativity.  Thus the environment in which the faculty work becomes a critical determinant of whether or not they can reach their creative potential.

Amabile describes some of the environmental conditions that help to promote creativity.  Among them are:

• stability of employment - this lowers attention to problems not related to the main tasks of research and teaching
• low bureaucracy - similarly, this enables faculty to keep their attention on the important things
• encourage rational intellectual risk taking, accept failure
• encourage interdisciplinary conversations - this help connections into different networks of knowledge
• expectations high but reasonable
• rewards not controlling - faculty choice in tasks, methods

In many ways, we see that the university is set up relatively well to meet most of these conditions. Tenure, for example, provides the desired stability of employment - for those who have it. Faculty are given broad choice by the university in how they will carry out their tasks.  However, even here there are reality constraints that can lessen creativity. For example, funding for the research component of the employment is increasingly difficult to obtain, and faculty generally find themselves devoting larger fractions of their effort to finding needed research funds. The funding, when found, often is narrowly defined and can squeeze out any significant creative flights of fancy. Worst of all, many universities push their faculty to have the largest possible grants at all times - it helps the rankings. Such controlling pressure probably does not lead to the highest creativity.  As Amabile has pointed out, the external funding can be most useful if it comes after the “aha” moment of creativity, since at that point it is not controlling, but facilitating.  Thus some relatively small internal funds to support research through the necessary first idea steps ultimately could lead to increased creativity of proposals.

In addition, other improvements need to be made in the existing university structure.  Bureaucracy, unfortunately, has grown significantly in universities over the years due to both internal and external pressures. Universities tend to have very intrenched administrative systems and groups, and in many institutions strong leadership will be required to create the kinds of administrative restructuring needed to create organizations that more closely meet tomorrow’s needs. For example, on many campuses I hear very vocal complaints about inefficiencies in the research offices, which seem not well organized to meet new requirements of the Federal Government, alert faculty to new research opportunities, provide timely grant financial data, etc.  Internally, the important principle of shared governance has often led to a profusion of sometimes overlapping faculty committees that both demand considerable time on the part of the participants, and lead to frustration on the part of other faculty who feel that their ideas and innovations are being subjected to unfair or confused scrutiny. 

Finally, many of our leaders do not do as good a job as they should of creating a supportive atmosphere of high expectations in their institutions.  Distinguished senior faculty are the most important players in creating this atmosphere of supportive expectations.  For example, I.I Rabi was a Nobel-prizewinning physicist at Columbia, one of the founders of modern quantum mechanics.  He was legendary in his ability to stimulate his colleagues to “try harder”. One of his colleagues was quoted in Rabi’s obituary as saying “The most spectacular thing about Rabi was that during a 15 year period there were four Nobel Prizes all in different fields of physics at Columbia.  Although Rabi wasn’t involved in the specific work, he was the key motivating person.” According to a widely quoted (and perhaps apocryphal) story, Rabi would roam the halls of the department, dropping into offices to ask about the latest research of individual faculty. When it had been explained to him, he would ask, “ Is this the most important problem in your field? If not, why aren’t you working on the most important problem?” With more senior faculty helping to define expectations like that, we would have a much more creative atmosphere!

One of the areas where universities generally are not really good is in encouraging interdisciplinary conversations.  Many claim to do it well, but when you look at it closely, few are actually good at it.  It is interesting to look at the brand new King Abdulla University of Science and Technology (see my comments here and here), which is organized with the goal of actually being at the forefront of  interdisciplinary knowledge creation: 

KAUST is a modern research university unlike any other.  The University is designed to allow physical and human networks – operating largely without regard for organizational or national boundaries – to flourish, thereby creating a critical vehicle for the exchange of ideas and the development of new knowledge.

The KAUST approach is obviously not the only one for increasing interdisciplinary activities and conversations.  However, it does give some feeling for the elements to be considered, and the significant organizational changes that might be required to maximize interdisciplinary exchange of ideas.

Transatlantic joint and double degree programs

A fascinating report Joint and Double Degree Programs in the Transatlantic Context has just been published by the Institute of International Education and the Freie Universitat Berlin .  It describes the results of a survey of American and European universities looking at the rapidly increasing phenomenon of transatlantic joint and double degree programs. The survey was not meant to be representative of activity in an absolute sense (only 180 institutions participated), but rather to show trends and developments.
The report notes:

One of the more prominent recent developments involves the emergence of transatlantic degree programs, such as dual diplomas, joint degrees, consortia and other forms of curriculum cooperation arrangements. Among European countries the introduction of joint and double degree programs has long been a vital part of internationalization strategies in higher education, helping to create stronger links and flourishing institutional partnerships,as well as preparing students for a global workplace. In the North American context, such programs have been until recently a less common feature of internationalization strategies for higher education institutions. However, the interest in curriculum cooperation is gaining momentum not only in the U.S. but in most countries around the world. In an increasingly global and competitive higher education market, collaborative programs, or “codesharing” as airlines would call it, can offer a set of advantages and are an important asset in the struggle
for attracting the best and the brightest.

The report also describes another troublesome aspect of American higher education that such programs may help correct:

Another challenge to transatlantic academic exchange is the increasing predominance of short-term study abroad programs in the U.S. While Europe still remains the leading destination for U.S. students who study abroad, the length of study abroad sojourns has declined dramatically in the past decade. Currently, only 6% of all U.S. students who study abroad, spend a full academic year in the host country, according to IIE’s Open Doors Report. The majority of study abroad programs are short-term programs of eight weeks or less, which may have only limited impact on the development of intercultural skills and foreign language immersion. Medium-term and long-term study abroad sojourns, especially if conducted in a structured way in cooperation with local partner institutions and including exposure to local student body and faculty, hold far greater opportunities in this respect.

Among the many  findings are that double degrees are much more common than joint degrees, but that European institutions offer about twice as many joint degrees as do American institutions.  The US institutions focus on undergraduate joint and double degrees, the European institutions on graduate programs.  The most common programs are in business and engineering.   The most common partners for European institutions are US institutions and other European Universities, but there are a significant number of partnerships with Latin American and Asian institutions as well. For the US institutions, European partnerships are the most common, and Asian partnerships play a somewhat more important role than is the case with European universities. 

At several points, the report indicates the importance of the Bologna process (see The Bologna process- a significant step in the modularization of higher education , Sept 12, 2008) in stimulating such programs in Europe. One very interesting sentence caught my eye:

Built on consecutive, intertwined modules, European BA programs are beginning to find new ways of integrating study abroad components into the curriculum.

Another competitive aspect of Bologna that we in the US should carefully note!

Creativity and the Research University

The world is obviously in considerable turmoil at the moment, and predictability is in shorter supply than usual.  The role of the US in the world is changing rapidly, and we must struggle to maintain our competitiveness in many dimensions.  Under these trying circumstances, I am tempted to reflect once again on what the role of the research university might be in the next decades. One of the keys to being successful in the competition of  a global knowledge economy is to be more creative than the opposition.  Thus it would seem to be a worthwhile time to reflect on what higher education can do to help creativity.  This is an open ended question, and thus the following should considered to be a beginning,  an imagination of some of the changes that might lead to a more Creative University.  I invite my readers to jump in with their take on this question.

I see that universities can contribute to the creativity of three constituencies:

• students - how do we educate students so that we unleash their creative capacity?
• faculty - how do we organize our institution and reward faculty so that they can reach their creative potential?
• regional - what role can a university play in helping a region to become a center of economic growth in creative technologies?

Before talking about these three roles, let me review some general characteristics of creativity that I find useful in thinking about this issue.  I have spoken before about Csikszentmihalyi’s systems approach .  He emphasizes that a creative idea comes out of a three-part system composed of: a domain consisting of symbolic rules and procedures; a field composed of the judges or gatekeepers to changes in domain; and the person with the idea. Csikszentmihalyi emphasizes that all must work together for an idea to be judged as creative, i.e. enter into the domain. As a consequence,  there is no such thing as a creative idea in the abstract. Rather the idea must be judged as creative by some set of humans - and the judgements may change over time as the people in the field change.

Teresa Amabile has looked in detail at the characteristics of the creative person. Clearly, there are “talents” and “personality characteristics”correlated with creativity that are to large degree innate, especially for those of great creativity.  However, there are also a number of aspects of creativity that can be very important for almost everyone. Amabile defines these as: teachable domain relevant skills; teachable creativity relevant skills; and motivation.

The domain relevant skills are really the set of facts, approaches, and connections out of which one can construct a creative solution to some problem. These  include a broad knowledge of the basic factual knowledge of the domain, the principles that pull together these facts, the techniques or scripts that have been developed to manipulate information in the domain, and an understanding of what is considered to be “new” in the domain.  A critical component of domain skill is that the facts be organized and stored in the individual’s mind according to general principles, not specific contextual tidbits.  This is often referred to as expert knowledge, and it leads to flexibility of thinking within the domain, rapid recall of facts, etc.

There are many creativity relevant skills described by Amabile and others. Among the most important are: breaking perceptual set, that is being able to see objects, ideas in new ways ; understanding of and comfort with complexity; keeping responses open as long as possible; suspending judgement; using wide categories to categorize information in order to better see relationships; having the ability to break out of well used algorithms, scripts; and knowing  creativity aids (heuristics)  for generating new ideas.

Amabile has shown that creativity most often correlated with intrinsic motivation - self determination.  And correspondingly, extrinsic motivation most often negative in stimulating creativity - it is the opposite of self determination. Extrinsic motivation can be positive, however, if it seems to confirm competence without connoting control.  It can also be beneficial if it enables the individual to do exciting work, again, without seeming to connote control. For example, after the famous “aha” phase of creativity, there is a period in which considerable work is needed to test the viability of the idea, and move it along to a point that the field can begin to evaluate it. Extrinsic motivation (resources) can be invaluable and positive at this point.  In my view, two of the most important results concerning the motivational aspect of creativity are that failure is a necessary part of the creative process that can provide valuable information, and that institutional encouragement of creativity is very important.

Yet a third aspect of creativity that can be useful in imagining what a university can do has been described by Richard Ogle, who views creativity from a network perspective. He also emphasizes that the creative person is not alone. She is hooked into a network of ideas created by others - the domain, and into a network of the field that will judge her ideas.  Learning how to get into these networks is critical, and much of creativity can be viewed as being able to see how to hook into a different network of ideas and bring some of these ideas back to your network.  My favorite demonstration of this aspect is described in my discussion of InnoCentive, Inc.

I will consider the first of these three areas of the creative university, the creative student, in this post, and the other two in a subsequent post.

Changes in science teaching - and learning

The New York Times had a very nice article yesterday (1/12/2009) on new ways of teaching freshman physics. I have written previously about some of the drivers behind these new approaches (A D- in science education) . A key driver was the demonstration that students generally were learning only a small percentage (about 30%) of the concepts  that were being taught in the courses, and that this result was independent of lecture quality, class size, or institution.

The article focuses on MIT’s new method of teaching freshman physics courses, Technology Enhanced Active Learning, or TEAL .  I visited MIT last year to look at the changes, and found them to be very impressive.  The old lecture and lecture hall are gone, replaced by an interactive space where students working as groups learn from each other and from other groups, with a faculty member providing guidance and brief presentations of principles.  Data I was shown on learning were very positive, and indicated that the new methods were working very much better than the old.  Still, as I talked to some of the senior faculty in the department (i.e. my age peers), I found significant support for the old methods of teaching and learning - “they worked for you and me”.  The article notes this ongoing discussion in the department. It also notes that no other science department at MIT has changed its teaching approach.  That raises the question - have the other fields (e.g. math, chemistry, biology) looked at whether their students are really learning the concepts being taught in their courses?

Up the street from MIT at Harvard, Eric Mazur was one of the very early proponents of these new methods of teaching.  He has an article in the January 2, 2009 issue of Science (p.50, subscription required) describing his own transformation from lauded lecturer to leader of the movement away from lectures.  He also tells us what he thinks has been accomplished:

Data obtained in my class and in classes of colleagues worldwide, in a wide range of academic settings and a wide range of disciplines, show that learning gains nearly triple with an approach that focuses on the student and on interactive learning

That is a pretty good argument for interactive learning.  However, Mazur also describes the personal cost to him of the changes

So, evidence is mounting that readjusting the focus of education from information transfer to helping students assimilate materials is paying off. My only regret is that I love to lecture.

Change is hard, even when it is desired.

One wonders why they protest so much??

In Another ranking of higher education - with a radical twist, I reported that OECD was thinking of doing international comparisons of national outcome measures for higher education.  I suggested that this focus on outcomes rather than inputs might lead to an interesting and enlightening set of information. 

It is not surprising that the American Council on Education is working to derail this effort since that organization has been very active in efforts to stop outcomes testing in the US.  Once again, David Ward (Pres. of ACE) argues that there are too many variables - funding methods, mission, etc. Further, he raises the possibility that the national rankings will be used to rank individual institutions, and that funding agencies might misuse this data.   Clearly, ACE is simply continuing to trot out its usual “we are too complicated to be held accountable” arguments, in this case bolstered by a threat of yet another ranking system for institutions of higher education. 

It is not obvious that the OECD effort would produce data that could be used to create another ranking system, given the way that previous OECD data has been reported.  However, since there are a  number of such ranking systems already in existence that focus on input measures, the fact that Ward focuses his displeasure on this particular possibility would seem to indicate that he thinks rankings based on outputs are even more misleading than rankings based on inputs. I would have argued the opposite. 

There are some obvious weaknesses in the system that OECD has outlined.  However, I think American higher education would be greatly strengthened if its leaders stopped focusing on preventing accountability measures, and worked instead to make sure the right measures exist and are adopted. 

Another Global Ranking of Higher Education - with a radical twist

A recent article in the Economist informed me that the Organization for Economic Co-operation and Development (OECD) is planning a new global comparison of universities.   Readers of this blog know my suspicions of national rankings in higher education (see, e.g.Better Rankings-but do we need them? Oct.11, 2006)), and my concerns regarding international rankings are even higher.  A recent post on Beerkins’ Blog has an excellent discussion of the problems with both national and international rankings. However, the OECD comparison will look at the issue from an entirely radical perspective - learning outcomes for the different national systems of higher education!

The Spellings Commission report has engendered considerable discussion in the US regarding the desirability of looking at learning outcomes, and/or whether any useful measures of learning outcomes can be developed considering the multiple goals of higher education (see also Spellings and transparency, Oct. 3, 2006).  While we in US higher education debate this issue at great length, generally making every argument that will forestall it happening, it looks like the OECD is going to move ahead to come up with some global scorecards on effectiveness of national approaches to higher education!

Real income vs educational level- a problem for higher education

An article in Foreign Affairs, and recent reports from the Pew Trust and ETS all have recently made similar, and very important, points about education and the American economy.  The first article talks about falling real wages and the relationship to protectionism; the Pew Trust looks at decreasing economic mobility in the US, and ETS considers the impacts on the US of a “perfect storm” of divergent skill distributions, the changing economy, and demographic trends.  Taken together, these reports raise some important questions for higher education.

Kenneth F. Scheve and Matthew J Slaughter, writing in the July/August 2007 issue of Foreign Affairs, discuss generally falling wages in the US, and their connection with increasing protectionism.  They point out that real income growth recently has skewed significantly in favor of high earners, with a strong correlation to educational level. They report:
Less than four percent of workers were in educational groups that enjoyed increases in mean real money earnings from 2000 to 2005; mean real money earnings rose for workers with doctorates and professional graduate degrees and fell for all others....Even college graduates and workers with nonprofessional master’s degrees saw their mean real money earnings decline.
In particular, the mean real earnings of college graduates fell by almost 4% between 2000 and 2005, while the mean real earnings of the MBA, JD, MD group rose by about 10%.  Hardest hit, not surprisingly, were high school dropouts, whose mean real earnings dropped by about 5% over that time period.