When you work on a nearly daily basis with the products of Thomson ISI and have developed a love and hate relationship with the databases you sieve all information on these products you can find. It was therefore a welcome interview with Keith MacGregor, executive VP of Thomson’s academic and government strategic business unit, and James Testa, senior director, editorial development and publisher relations for Thomson that Nancy K. Herther published in the last issue of the Searcher (not free on-line).
The interview itself was rather too nice, the interviewer was perhaps too polite to raise really sensitive subjects. The parting thoughts listed by Herther at the end of the interview were the most interesting points of the whole article. A real pity that the two executives did not have a change to formulate their opinions on those points. In addition to the parting thoughts listed by Herther I would have loved to hear the opinion of these two gentleman on the stubborn ISI/Thomson Scientific policy not to change anything of the data collected in WoS. This results in all kind of inconsistencies in journal and author names when these are subject of study for a longer time period. I have the feeling that they try and correct some of the data in the software environment, but when you have to deal with the output as an analyst or collection development librarian, you end up with a load of data inconsistencies.
Only a few days ago I had to look into the citedness of T.B. van Wimersma Greidanus who published between 1969 and 1996. Impressive publication list, but really difficult to collect all those 300+ references from the cited ref search. For journal titles I have blogged already on this subject before and even before.
According to a few, Thomson is opening up a bit. However Herther wrote “I read a great deal of the published criticisms of citation data used for ranking individuals and institutions. I was therefore surprised at the absence of Thomson Scientific’s voice in many of these debates”. Which confirms my impression. But then again, perhaps times they are a-changin’.
reference
Herther, N. K. (2007). Thomson Scientific and the citation indexes : an interview with Keith MacGregor and James Testa. Searcher 15(10): 8-17.
Did I report yesterday on the first preprint article that compared Impact factors calculated with JCR and Scopus, later that day a second journal was published on e-lis covering the same subject. Gorraiz and Schoegl (2007) took the analysis really a step further than Pislyakov (2007). Not only did they include a larger set of journals in their sample 100 compared to 20, they also looked at the other bibliometric indicator the immediacy index.
Interesting is the determination of the authors to look for journals in the chosen subject area, pharmacology, that were not included in the JCR but should have been there on the basis of their citations. In the journal selection process of Thomson some other factors are taken into account, but in practice we expect all top journals in a certain category to be included in the JCR/WoS database. So it is interesting to learn that there are a number of journals that should have been included on the basis of citation data in the databases of Thomson.
At the beginning of the article the authors state:
Since there are more journals included in Scopus than in WoS, a journal in Scopus has a higher cace to get cited in general. Therefore the the values for the impact factor and the immediacy index should also be higher in Scopus
This might sound plausible, but in actual fact the effect of a larger journal base is much smaller. Because Web of Science already covers virtually all top journals in the subject category they also cover the journals where most citations take place. Outside the top journals relatively little citation traffic takes place. This has been demonstrated by Ioannidis (2006) and is also indicated in journal selection policy of Thomson where they refer to some of their own research:
More recently, an analysis of 7,528 journals covered in the 2005 JCR® revealed that as few as 300 journals account for more than 50% of what is cited and more than 25% of what is published in them. A core of 3,000 of these journals accounts for about 75% of published articles and over 90% of cited articles.
What really is disturbing from both the articles of Gorraiz and Schoegl (2007) and Pislyakov (2007) is that both databases are not one hundred percent reliable when it comes to number of article published in a given year. For Scopus there we can expect some minor discrepancies since we are dealing with a young database that shows still some fluctuations in content. Elsevier still has some work to do. For WoS it is sometimes just sloppiness in indexing and that is unforgivable.
References:
Gorraiz, J. & C. Schloegl (2007). A bibliometric analysis of pharmacology and pharmacy journals: Scopus versus Web of Science. Journal of Information Science 00(00): 00-00. http://eprints.rclis.org/archive/00011966/
Ioannidis, J. P. A. (2006). Concentration of the Most-Cited Papers in the Scientific Literature: Analysis of Journal Ecosystems. PLoS ONE 1(1): e5. http://dx.doi.org/10.1371/journal.pone.0000005
Pislyakov, V. (2007). Comparing two “thermometers”: Impact factors of 20 leading economic journals according to Journal Citation Reports and Scopus. E-Lis. http://eprints.rclis.org/archive/00011865/
You only had to wait for it. With the rich resource of citation data available in Scopus, somebody was going use it and calculate Impact Factors. Quantitative journal evaluations was once the single domain of Thomson Scientific (formerly ISI) but nowadays they face more and more competition. Elsevier, with Scopus, has so far hesitated to step into the arena of journal evaluation, but Vladimir Pislyakov (2007) has made a start for the 20 top journals in economics.
He compared the Impact factor from the JCR with the Impact he construed for the same journals with citation data from Scopus. In his methodology he made small mistake by not excluding the non citable items, which is quite easy to do in Scopus. But this will not invalidate his results. What was to be expected, confirming our experience with higher citations in Scopus compared to Web of Science, is that overall more citations per article were found in Scopus. This resulted in slightly higher IF as calculated by Scopus. What is more interesting is that the rankings of the journals based on Scopus data differed from the ranking based on the JCR impact factors. Overall they correlated well, but looking into detail, there was a journal that dropped from rank 5 to 13, another from 11 to 18. So there is merit to investigate this on a larger scale than those 20 journals in economics.
In the end the author makes a big mistake, he states
“Since impact factor is considered to be one of the crucial citation indicators which is widely used in research assessment and science administration, it is important to examine it critically from various points of view and investigate the environment in which it is calculated.”
Those are practices we should stay away from. The IF as such is only of interest for scientists when they select a journal for publication. IF should not be used for research evaluation of grant applications.
Reference:
Pislyakov, V. (2007). Comparing two “thermometers”: Impact factors of 20 leading economic journals according to Journal Citation Reports and Scopus. E-Lis. http://eprints.rclis.org/archive/00011865/
Latest Comments