OECD Feasibility Study for the International Assessment of Higher Education Learning Outcomes (AHELO)

The OECD Assessment of Higher Education Learning Outcomes (AHELO) is a ground-breaking initiative to assess learning outcomes on an international scale by creating measures that would be valid for all cultures and languages. Between ten and thirty-thousand higher education students in over ten different countries will take part in a feasibility study to determine the bounds of this ambitious project, with an eye to the possible creation of a full-scale AHELO upon its completion.

 The OECD has just announced a project with the grand title that I took for this post, that has the potential to change much of the ongoing discussion in the US regarding assessment of learning outcomes (see, e.g. Learning outcomes information and the quality of higher education).  The OECD is initiating a very interesting pilot project to see if it is possible to create measures of learning outcomes valid across countries and cultures.  (A very useful earlier OECD report on learning outcomes can be found here.)

The OECD expresses concerns that the current ranking systems used in higher education almost ignore what should be the key point of such a ranking: the quality of the education that the students receive.

 Rankings tend to give research, award-winning faculty, and older, pre-1920’s universities priority over teaching and learning.

As a consequence:

Current ranking systems threaten the diversity of higher education. Universities are under pressure to cut programmes and redefine missions in a fight for survival that depends on clambering into the top 10, 50 or 100 universities. They may throw their best assets overboard in the rash attempt to keep their university afloat. Such pressure breeds an unhealthy copycat behavior among HEIs, the result of which can only be a bland standardization.

This project, then, has as a major goal the development of instruments that individual institutions can use to measure their own effectiveness in the area of student learning. Change within universities can then be driven by the goal of producing greater student learning, rather than by the goal of making it into the upper echelons of an ultimately meaningless ranking.  

The project actually has 4 strands that will ultimately be used to look at student learning.  They are:

1.    Generic – Analytic skills, critical thinking, written communication, etc

2.    Discipline specific – looking at the “above content” part of learning: the capacity to use what has been learned, often in novel situations

3.    Context – learning takes place in the context of the individual institution, which involves physical and organizational characteristics; education-related behaviors and practices; psyco-social and cultural aspects; and behavioral and attitudinal outcomes.

4.    Value added- given the characteristics of the incoming students at a particular institution, what intellectual advances occur over the course of the degree?

For the first of these, the Collegiate Learning Assessment, appropriately modified for use across cultures, will be used to show proof of concept.  For the second, two fields, economics and engineering, will be used to see if meaningful instruments can be developed. According to Inside Higher Education, the Australian Council for Educational Research will lead a consortium working on this aspect. For the third, existing data, such as persistence and graduation rates, surveys of student expectations, quality of student-faculty interactions, self-reported learning gains, will be gathered and analyzed. Ultimately, surveys of alumni and employers will join this mix. The fourth strand will not actually be included in this pilot project since the time required to create the instruments will exceed the period of the pilot. Instead, this will be a time of reflection on the types of methodologies that might be used in a larger project

Although the OECD document does not make it clear, Inside Higher Education reports that the US is participating in this study in two ways: providing some financial support and some students in Connecticut, Massachusetts, Missouri, and Pennsylvania will be part of the test cohort for the Generic strand.  This is in stark contrast with the situation two years ago, when the Bush administration declined to participate in an earlier stage of this project.  Thus it appears that the interest in outcomes assessment that was so prominent in the Bush administration has not disappeared with the coming of the Obama administration, as many in higher education had hoped.

 

Comments

[UAE universities]

The OECD is doing a great project for learning but the main thing is that what is the output of this project.Anyway keep it up these interesting post i like that and also like to give my comment i hope to read your blog everyone takes the interest this topic in future.