Whither alternative (and improved) credentialing?

I recently served on a panel at a meeting organized by the California Higher Education Innovation Council to look at "Alternative Credentials and Unbundling the Degree: Meeting Employer Needs or Short-Circuiting Proven Approaches?" Our panel was challenged beforehand by its moderator, Ryan Craig, to imagine how conditions had to change over the next decade in order for alternative credentialing ("e.g. nanodegrees and badges" according to the meeting invitation) to become a major force in higher education. I will make no attempt to review the many arguments advanced on this subject at the meeting, but simply describe some of my own thoughts (however tentative) that were stimulated by this challenge.

There are obviously three broad constituencies interested in questions of higher education credentialing: students, government, and employers. My belief is that the most important of these in determining whether alternative credentialing takes hold will be employers: if employers find it truly useful, most students will enthusiastically sign on, and government will see little reason to block something that employers and students find to be of real value. 

Clayton Christensen's "Jobs to be Done" approach emphasizes that to understand an issue such as this, one must understand the problem that the customer (employers in this case) are buying the product (higher education credentialing) to solve. I think the primary answer is rather simple: employers are "buying" higher education credentialing to help maximize the probability that a new hire or a promotion will work out well for their company. There is an interesting extension of this, however, articulated by Frank and and Cook in The-Winner-Take-All-Society  two decades ago: if the hire does not work out, management is significantly shielded from blame if they hired the person with the most prestigious and established credentials.

In order to maximize their probability of success in hiring and promotion, employers need to get as much information as possible about the skills and capabilities of their applicants. Those might be divided roughly into three categories: 1)subject matter scope and mastery (e.g, level in economics); 2)higher order intellectual skills (e.g. critical thinking, postformal reasoning, creativity); 3)personal characteristics (e.g. perseverance, ability to work in groups, honesty, competitiveness). Different jobs obviously would lead to different weightings of the importance and applicability of these categories in the eyes of the employers.

Present credentialing, in the form of a degree and transcripts, is a rather blunt instrument for providing the desired information. For the relatively rather small fraction of the roughly 4,500 accredited institutions that are very selective, the degree signifies that the graduate was among the very best high school students in the nation the year that she graduated. For that subset, the degree also often tells something about the students underlying interests and strengths (a student who chooses Yale vs one who opts for MIT). For the majority of students who come from rather non-differentiated, non-selective institutions, little information of these types is contained in the degree.

For students from all institutions, the degree shows a level of perseverance, and names the major and the level (e.g.Master's in Economics). However, similar designated degrees signify greatly different things from different institutions (and probably even for two students from the same institution), and supporting information in the form of a transcript is pretty useless in teasing out desired information. What does an A or B in intermediate economics mean at this particular institution? What was covered, and what did the professor emphasize in awarding the grade?

Thus, current credentialing provides suggestions of answers to questions that employers have, but not much more. Large, data driven companies such as Google increasingly report that they can find no correlation between information contained in a transcript and the value of a hire to the company. As a consequence, there is considerable potential to make products that better address components of the problem that employers are trying to solve. That is where alternative credentialing comes in. 

******

Alternative credentials typically are designed to certify levels of competency in specific domains. To be more useful than traditional credentialing, these alternatives must provide additional information about competencies not provided by the traditional degree. For example, a "badge" that certifies that I have successfully completed five Coursera courses or a course in some powerful new computer language likely will be of minimal value to employers unless supplemented with real information about competencies demonstrated. Thus the supplemental value of alternative credentials really depends on ability to articulate and measure competencies that are of interest and value to employers.

******

Higher education thus far has resisted rather ferociously defining and measuring competencies that describe higher-educational attainment. The usual position is that the benefits of higher education are too numerous and of such breadth that they cannot be captured by such limiting concepts as competencies. I argued in Reputation and brand in the world of changing higher education that in fact, higher education produces both outcomes that can be measured at graduation and outcomes that can't be measured because they are produced over time by the interaction of the education, the characteristics of the individual and future life experiences. Acknowledging that some important outcomes are indeed measurable, and quantifying those measurable outcomes would enable universities to provide improved credentialing that would be of increased value to employers. One already sees that as competency based programs increase in popularity, they are likely to result in transcripts (e.g.North Arizona University) that are considerably more informative than the current norm. To be seen, however, is whether these competencies are those of interest to employers, or only describe competencies that interest faculty.

******

Thus once again the issue focuses on defining and measuring competencies  - hopefully those that both employers and faculty believe to be of importance. Because of historical academic opposition to the concept of competencies, we know less about what they should be and how to measure them than would be useful at this juncture. Outside of the US, the Bologna Process and its Tuning process have struggled with aspects of the competencies issue for degrees over the past 15 years; the Lumina Foundation brought Tuning to the US in 2009 with its Degree Qualifications Profile. Lumina is now leading another effort to define a similar framework for other types of credentials. This new framework apparently seeks to define areas of skills and levels of achievement in all three of the categories above using a Bloom's Taxonomy - like approach

Competencies in the Tuning Process are defined by consulting graduates, employers, and academics. This is thus a very good step in helping to define the components of both alternative and improved credentialing. However, perfection is not easily attained. It is obvious that many words may mean different things to different groups. For example, faculty rather uniformly insist that students are learning critical thinking; employers rather uniformly say that graduates are not able to think critically. Although this different perspective may be due to differences in definition, it is important to note that neither group is basing their position on actual measurements of level of critical thinking - only impressions.  Thus, not only must the competencies be clearly defined, but methods of measuring levels of those competencies as they are defined must be developed and agreed upon. Only then will we know whether all groups are talking about the same thing, and only then will employers be able to decide if particular competencies are valuable for their employees.

******

All of the above discussion relates to what we might call Instantaneous Credentials (IC) - outcomes of education that can be measured at the end of the educational experience. Another type of credential is beginning to appear, however, that we might call Real Time Credentials (RTC). Real time credentials typically are based on analysis of big data, often obtained from social media,to determine the competencies of a particular person in real time based on actual performance in life situations. That is, RTC seeks to measure some of those competencies that were not measurable at the end of the educational process.

Gild is an employment start up that has attracted considerable attention using this approach. As described by the Undercover Recruiter:

Gild says that it “goes where developers hang out” on the Internet and scores developers on the quality of their public code and professional knowledge. It then scores these developers—and has done so millions of times—to offer recruiters a deeper look into a candidate’s true skills. This eliminates some of the guesswork and blind faith that recruiters who don’t happen to be developers must employ. In addition, Gild also gathers social media activity for each candidate to help you determine culture fit.

Gild's approach does not focus on instantaneous credentials (degrees, badges, etc.) describing learned competencies, but looks at how an individual demonstrates competencies in actual work situations. In doing so, it seeks to provide important information on all three of the categories of capabilities described above. Given the discussion above regarding the limitations of current instantaneous credentials in meeting employer needs, it should be no surprise that much of the development of real time credentials is taking place in companies such as Gild that focus on enabling employers to be more successful in filling positions.

LinkedIn is taking a related but somewhat different approach to the same problem: it wants us to input our own educational and work data - us being an ever-increasing fraction of the workforce of the world. Simultaneously, it has been developing an understanding of the skills required to do the many jobs that appear as we all input our histories. By comparing and analyzing all of this big data,  LinkedIn can understand our individual competencies in real time - and then help companies looking for new talent to find a better match to their needs. Of course, it can also help us to find a new job that really fits our skills so long as we go through LinkedIn to find that job.

 

As suggested by the above, a common attribute of most of the RTC work is that the RTC is the property of the company that created it, while we own our own underlying data. Thus the RTC is not "portable" in the sense of a CISCO credential or a college diploma. As Ryan Craig has pointed out, potential employers are likely to want to look at some of the data underlying the RTC, thus setting up an interesting ownership discussion.

 

One challenge facing the creators of  RTC is very similar to that of the producers of IC - competencies must be defined, the ways in which they are manifested or measured must be determined and the pertinence of those competencies to employers must be demonstrated. Success in a particular job is unlikely to be determined by a unique set of competencies, and there are many hidden variables when comparing one position with another.   

******

But what of Ryan Craig's original question to the panel: what must change for these various alternative credentialing approaches to become a major force in higher education? First, there is a "push" towards alternative credentialing caused by the lack of useful information for employers contained in traditional degrees and transcripts. The obstacle that must be overcome, however, is that we don't know how to define and measure many of the most desirable competencies.  That is the change that must occur to enable a major perturbation to the current near monopoly for credentialing held by traditional higher education.