New webometrics ranking of world universities released

Of all possible rankings of universities that are available, the Webometrics Ranking of World Universities takes an odd place. It only looks at the website performance of the university. Their rankings have been updated somewhere earlier this month.

I have mixed feelings with their approach, but it is a prelude newer rankings than those solely based on scholarly output and impact. However I think that their approach needs more time and better tools than are available at the moment. The leading  researchers in this field are in the group of Mike Thelwall. Their measurements are based on their own crawlers and tools to explore, measure and investigate the academic Web. They have can understand and interpret their results completely. The Cybermetrics Lab (CINDOC) which produces the Webometrics rankings uses publicly available tools such as Yahoo!, Google and Exalead over which they don’t have control. And more importantly they don’t know whasoever how these results come about. Another problem with e.g. Google is that the number for search results are notoriously unreliable. It depends amongst others on time of day, Web Traffic, Server Load at Google and Data Center dat is being used.

So for the moment we have to take these results with a spoon full of salt rather than a pinch. It is also a question what is being measured. Take for instance the size of university Websites.  In Utrecht all staff and students appear to have personal webpages on the University Website. These are all included in the count, whether they actually contain some usefull information or not. At our University the mainstay of the indexed webpages consist of catalog records from the library. I really wonder if you really want to compare these apples and pears.

As for the measure of rich files I really wonder if they have been able to harvest all the material deposited on our repository. Looking a the statistics such as provided by OAISTER on OA harvestable documents, Wageningen University has one of the larger content rich repositories in the Netherlands. In the Webometric we are the bottom fish for this measure in the Netherlands. That we are making use of proprietary software but still adhering the OAI-PMHH protocol, of that the repository is hosted as a directory http://library.wur.nl/way should not effect the rankings as it does for the moment.

On other measures they are completely vague about the exact measure. Take for instance the Google Scholar measure. They state: “Google Scholar provides the number of papers and citations for each academic domain. These results from the Scholar database represent papers, reports and other academic items.” How do they combine publications and citations in a single measure? It is not explained. Google never gives more than the first 1000 results. How do they arrive at all citations for an institute? How did they search for the name of an institute? Did they include medical training hospitals with the University.

I do use these rankings for one point though. That is to push for the improvement of our University and Library Website wherever possible. In some aspects that is really badly needed. But I really want to take these rankings more seriously. For the moment I can’t. They have been updated again that should be the message of this post, since their blog has been defunct for quite some time already. A pity.

6 thoughts on “New webometrics ranking of world universities released”

  1. Thank you for your critical comments. The keyword here is feasibility. For a complete and precise description of world universities we will need several dozens of variables and a high level of standardization of such variables (what is a professor in different countries?). Moreover for many of these variables there is no reliable source even from the own university. This is the reason other Rankings usually focus on Top 200 or Top 500 institutions.

    When checking ARWU (Shanghai) or THES (Times) some of the variables are not very useful for ranking large sets. Consider there are only a few universities with 2 or more Nobel Prizes, so ranking with low numbers is not fair. This is can explain the lack of reliability of positions below 200 in the cited rankings.

    So, why web ranking? First, the numbers involved are far larger, millions instead of hundreds; Second, it is easier to obtain data for institutions worldwide: Webometrics rank 16000 universities including most of the developing countries not included in other rankings; Third, Web reflects (or at least should in the near future) the full set of activities of an academic institution (teaching, research, transfer, community involvement), not only number of papers published. Probably there is a lot of “noise” but there are certainly very important aspects of this noise for candidate students (sports!) or other citizens (general knowledge, half of the faculty members came from Hunmanities and Social Sciences) not very prone to publication; Fourth, Web is reaching hundreds of millions (paper thousands at the best). Electronic publication should be mandatory, or at least an evaluation system should take it very seriously. Webometrics is paving such road.

    Regarding methodology, it is possible to use crawlers instead of search engines, but this is unfeasible for such a large task. Moreover, most of the users use Google and Co. for recovering information, so if you information are not in that databases it is invisible, it does not exist. For avoiding inconsistencies we choose specific data centers, not the central webservers, collect the figures twice, correct the biased results and select the most relevant formats. Of course there are problems with Scholar, but we have contacted with them and they will provide new tools in the non-beta version.

    Finally not to forget the results. Webometrics ranks highly with THES & ARWU (0.6-0.7), provide useful insights about crazy web policies (Imperial College, Paris & Catalonian Universities, Johns Hopkins, ..) or great repository initiatives (Penn State, Linkoping, Complutense).

    Please, take the Web seriously

  2. @ Isidoro,
    Yes I do take the Web seriously, and the presence of universities at the Web as well. However, I think that the results of the Webometrics ranking of World Universities are a bit coarse to say the least.

  3. Of course we are open to criticism and to suggestions for improvement. As a research piece we are working of fine tuning the Ranking and you can expect a major refinement on the visibility indicator in the next edition. There are currently two sources of misunderstanding in our rankings: Digital Divide, as some institutions are not working as expected on the Web (Caltech, Cambridge, French!), but this is a very interesting result of our effort. And non-university institutions as we intend to include other higher education institutions. If you find secondary education institutions please inform us.

Comments are closed.