I’ve recently written several posts related to research and education in China and India, and the comments led me to examine how their best universities are ranked among the universities of the world. The answer is “Low”, but the measuring rod looks crooked.
The “Academic Ranking of World Universities” (ARWU) is widely regarded as the best objective, international measure of university quality, and the ARWU says that excellent universities in Asia are scarce. It ranks only 5 in the top 100: the Tokyo, Kyoto, Osaka, Tohoku, and Nagoya universities, all established long ago in the only major Asian nation that reached a West European level of wealth more than a decade ago — and in fact, reached that level 40 years ago.* (And as we’ll see, age matters.)
The ARWU, compiled by Shanghai Jiao Tong University, ranks China’s leading universities, Tsinghua and Peking, below below 150, and among India’s elite IITs, it ranks only one (the oldest, IIT Kharagpur) above 500.
This seemed to me to be out of line with reality, and on further investigation, I concluded that the ARWU has a strong negative bias as a measure of the current quality of rapidly advancing universities.
In examining a paper on the ARWU methodology [pdf], I found that its scores place great weight on numbers of Nobel Prizes and Fields Medals won, to publications in Science and Nature, and to publications listed in the Science Citation Index. The problem is that, as a consequence, the scores are weighted toward cumulative numbers, which are poor measures of rapidly rising institutions, such as the leading universities in China and India. For example, if an identical twin of Harvard materialized in Somerville or Beijing today, its rank would be abysmal for years to come.
(A presentation I heard at Tsinghua focused on a similar publication-based metric in describing the university’s progress, and the literature on science and technology policy in China details a system in which the pressures to perform well by this metric are pervasive.)
In summary, the much-cited Academic Ranking of World Universities is very much a lagging indicator of quality. I’d like to see a ranking of similar objectivity that provides a more reliable measure of current performance, and a companion ranking that includes momentum in the score, giving some indication of what to expect in the future. Based on various sources, my impression is that the best universities in China are not now in the top ranks worldwide, but are close, and advancing rapidly.
I welcome pointers to new and rising objective metrics.
* This wonderful, dynamic, interactive data graphic shows world per-capita income levels vs. time. Since time is represented as time, the data plot can represent not one, but two additional quantities, each of which can be selected from among dozens of data sets. But there is, in effect, another dimension, encoded in the changing sizes of the moving dots. But their color is informative, too…
(As a horizontal axis, I included infant mortality. Please pay no attention to the position of the U.S.)
See also:
- Indian education, top to bottom
- E-Drexler in India — where do all the visitors come from?
- Nanotechnology research: A better picture of work in China
- Nanotechnology research papers: The world’s most prolific authors
{ 4 comments… read them below or add one }
Chris Phoenix 11.08.09 at 11:31 am UTC
It looks like many of the countries that float above the trend line (with infant mortality that’s high for their income) are those with oil wealth. Perhaps the US float above the line corresponds to a relatively sudden influx of tech wealth in the last half-century?
Eric Drexler 11.08.09 at 8:00 pm UTC
You can test that hypothesis by dragging the slider to 1950, clicking “Play”, and watching the history — you’ll see that the other relevant countries have advanced faster, starting from a lower base.
Andrew W. 11.16.09 at 10:43 pm UTC
For example, if an identical twin of Harvard materialized in Somerville or Beijing today, its rank would be abysmal for years to come.
Eric Drexler 11.17.09 at 12:38 pm UTC
Yes, I agree, but by “materialized” I meant to imply “by magic” — a thought experiment, to illustrate a problem with the scoring algorithm. Equivalently, imagine that Harvard were to be shut down, but with its campus, programs, and people transferred to a (so-called) new university, Harvard-2; then consider the result if Harvard-2 were to be scored as a new university, with only the briefest history, no accomplished alumni, no pre-2009 papers in the ISI, etc.
A more realistic case would be a rapidly rising institution that moves from mediocre to top-level performance in 10 years, but is substantially underestimated for the same reasons that Harvard-2 would be.