Journal Impact Factor Weighted by SJR and 5-Year If indicators of Citing Sources

Aliguliyev and Adigozalova: Journal Impact Factor Weighted by SJR and 5-Year If indicators of Citing Sources

Authors

INTRODUCTION

The publication of numerous periodicals have led to the rapid growth of scientific information in recent periods. There emerged a need to evaluate the quality of those periodicals and to review their impact on other sectors for the purpose of the management of large-volume information. First article devoted to the consideration of citations was written in 1927[1] in order to evaluate the scientific journals. IF as the main indicator of scientific journals measuring the impact of citations on the article has been proposed by Eugene Garfield.[2] IF indicator of a journal is based on Web of Science (WoS) database and presented by Journal Citation Reports annually. Despite the broad use of IF in the assessment of scientific journals in last 60 years, it has been subject to critique. Among its limitations, the disregard of the prestige of a citing source and the consideration of the two-year period only are emphasized. It is because two-year period is considered as insufficient for measuring IF of journals in several fields. Hence, 5-year IF has been developed which considers the longer citation period later.[3]

The consideration of self citation in methodological aspects of index computation, the low comparability among resources and English as the primary language of publications create challenges in assessing the quality of citation. Numerous researchers have proposed various approaches to the evaluation of scientific journals. Their reflections are comprised while considering the prestige of obtained citation. Therefore, Eigenfactor Score indicator has been developed by researchers of the University of Washington in following periods. In 2004, SJR indicator has been developed based on PageRank algorithm on Scopus database by the SCImago Research Laboratory. SJR indicator is a quality indicator of journals included in Scopus database and carries out calculations considering 3 year period of references included in the database. SNIP indicatos was developed by Henk Meod based on Scopus database in 2010.[4-5]

The aforementioned allow to say that, the consideration of the prestige of a citing source in IF calculations is of great importance. A question can emerge in this case on how to determine the importance of a citing source and which indicator must be taken as primary. Considering that 5IF is a stability measure of importance and SJR indicator characterizes the prestige of a citation, this article calculates a weighted IF by using two main prestige indicators of WoS and Scopus databases.

MATERIALS AND METHODS

The evaluation of the quality of research bears importance for institutions and organizations, as well as scholars. Traditionally, journals have been ranked by expert evaluation (for example, Association of Business Schools). Notewithstanding this, new indicators considering various factors have been proposed, for example: IF, 5IF, SJR, Eigenfactor, h-index, etc.[6]

IF is calculated as a ratio of the number of citations to articles published in recent 2 years in current year to the number of articles published in those two years.[2,7-8]

1
IFjt=Σi=1njtcijtajt1+ajt2 https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g000.jpg

Here, IFjthttps://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g001.jpg - denotes the IF of a j journal in t year, njthttps://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g002.jpg - is the number of journals referring to j journal in t year; cijthttps://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g003.jpg - denotes the number of references from i journal to j journal in t year, and ajthttps://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g004.jpg - is the number of articles published in t year.

Huang and Lin reckoned that, 2 year period is not sufficient for the IF calculation of a journal and it is more purposeful to develop indicators covering various citation periods for various fields. Another group of researchers emphasized that, IF covered the narrower time period due to the disconsideration of a particular period of time devoted to collecting citations in particular fields. Considering the critical points published in Thomsom Reuters, Leydersdorff, Zhou and Bornmann has mentioned that, 5IF indicator covering 5 years extended from 2 years is being developed.[3]

The Professor of the University of Washington Karl Bergsterm and his colleagues have developed Eigenfactor Score covering not only the number of citations of scientific journals, but also the prestige of a source and based on PageRank algorithm.[9]

It is also to be mentioned that all journals are not indexed in WoS and hence, researchers require other indicators for the evaluation of their quality. Scopus database covering larger scale journals has been established later and these journals used indicators such as SJR and SNIP. SJR indicator usable in very large networks was developed by SCImago Research Laboratory in 2004. This indicator not only considers the total number of citations, but also the prestige of those. SJR applies PageRank algorithm and is a metric alternative to IF.[10-11]

SNIP indicator compares articles (publications) of various scientific subject fields by considering the intensity of being cited in each scientific field. SNIP indicator is calculated as a ratio of citation corresponding to each paper (raw impact per paper, RIP) to the relative database citation potential (RDCP) indexed by a journal.[12-13]

RIP denotes the ratio of the number of citations to a journal published in the analyzed year to the total number of articles published in last 3 years. For instance, if 100 papers were published in one journal in 2008-2010 years and 200 references were made to these articles in 2011, then journal RIP is =200100=2https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g005.jpg

It is to be noted that, RIP is similar to IF, however, the time period is taken not as 2, but 3 years.[14]

Comparison of WoS and Scopus databases indicators is illustrated in Table 1. As mentioned, citations play an important role for the evaluation of research. 2-year IF is considered as one of the most useful tools demonstrating the scientific prestige of a journal.[15-18] However, the weakness of IF is the equal weight assigned to citations obtained from various prestige journals. It is because the citations obtained from more prestige journal is more important than that obtained from less prestige journal.

Table 1

Comparison of WoS and Scopus databases indicators.

DatabaseWeb of ScienceScopus
IF5IFSJRSNIP
Cıtation period1 year1 year3 years3 years
Citation window2 preceding years5 preceding years3 preceding years3 preceding years
Journals providing citationsOnly cited journalsAll
Weight of citationsEqualEqualDepending on the prestige of the citing journalNot important
Self citationIncludedNot includedIncluded
Cited articlesOnly cited
(article and review)
All

In order extend the evaluation of the quality and importance of a journal, it is more purposeful to assign weights to citations obtained from a more prestige journal unlike citations obtained from a less prestigious journal. Considering the aforementioned, several researchers have proposed a weighted Impact Factor (WIF) covering not only the citation as such, but also the prestige of a citing journal. Despite the existence of accurate calculation tools of citations obtained from prestigious journals, Kochen[19] and Pinski and Narin have proposed another approach.[20] In the approach proposed by Pinski and Narin, weight coefficients for the normalization scheme and the evaluation of a weight of a particular journal are determined with the following formula:

2
W=total number of citations to certain journal from other journalstotal number of references from that journal to other journals https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g006.jpg

At present, PageRank algorithm employed by GoogleTM[21] for web-page ranking is also applied in the assignation of equal weighting to citations during weighted IF calculation. In order to determine PR algorithm of a web-page via iterative process, GoogleTM considers not only the number of citations made to a page from other pages, but also the degree of importance of citations made to that page.

Y-factor index proposed by Bollen, Rodriguez, and Sompel[22] for weight calculation has been developed as a result of a merge of an IF value of a journal and PR algorithm. Y-factor index is determined as a product of IF value of a particular journal and PR value.

Y = F × R    (3)

Later, IF considering the prestige of a citation has been proposed by Buela-Casal,[23] Habibzadeh and Yadollahie,[24] Waltman and Eck,[25] Zitt and Small,[26] Zyczkowski,[27] Zitt.[28] As noted, this approach encountered in scientific literature has been officially proposed in writings by Pinksi and Narin.[20]

Among those, WIF proposed by Habibzadeh and Yadollahie (H and Y)[24] in 2008 can be shown.

4
WIFjt=Σi=1njtWijt×cijtajt1+ajt2 https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g007.jpg

Here, wijthttps://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g008.jpg denotes the weight of the journal i to the relative journal j in year t:

5
wijt=10×10.828×eqijt1+16.183×eqijt https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g009.jpg

Here, qijthttps://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g010.jpg is a ratio of IF of a citing journal to IF value of a cited journal and calculated as following:

6
qijt=IFtt1IFjt1 https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g011.jpg

The weakness of WIF proposed by H and Y is that q coefficient of a citation obtained by a prestigious journal is smaller than the coefficient of a citation obtained from a less prestige journal. If the IF value of a citing journal in WIF is equal to the IF value of a cited journal, then the weight is equal to 1, if the IF of a citing journal is larger than the IF of a cited journal the weight is greater than 1 and vice versa, if the prestige of a citing journal is lower than than of a cited journal, then the weight is denoted as less than 1.

For instance, assume that a journal with Fi = 4 is given and this journal has been cited in two journals with different IF values as Fj1 = 1, Fj2 = 2. In this case, according to results obtained from (5) formula, qijthttps://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g012.jpg coefficient of first journal will be smaller than that of second journal (wij1 > wij2)). It can be concluded that, the weight of a citing journal with smaller IF will be greater than that with larger IF.

Alguliyev, Aliguliyev and Ismayilova[8] proposed the following version of the JCR IF.

7
W5IFjt=Σi=1njt(5IFit1+1)cijtajt1+ajt2 https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g013.jpg

where njthttps://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g014.jpg - denotes the number of journals citing j journal in t year; cijthttps://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g015.jpg is the number of references from i journal to j journal in t year, ajthttps://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g016.jpg - denotes the number of articles published in the year t , and 5IFjthttps://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g017.jpg - denotes 5-year IF of the journal i in the year t.

In,[29] linearly and non-linearly penalized impact factors by self-citations, encouraged impact factor, considering distribution scale of citing sources are proposed.

Impact factor linearly penalized by self-citations is defined as follows:

8
LPIFjt=β1×scjt+β2×(cjtscjt)ajt1+ajt2 https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g018.jpg

where β1 and β1 are the rate coefficients of self-citations and non-self-citations which 0 < β1β2 < 1 and β1 + β2 = 1, where β1=13https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g019.jpg and β2=23https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g020.jpg.

cjthttps://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g021.jpg is the total number of citations received by journal j in the year t, cjthttps://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g022.jpg denotes the number of self-citations of journal j in the year t, ajt1+ajt2https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g023.jpg is the total number of articles published in journal j in the two previous years t – 1 and t – 2.

Impact factor non-linearly penalized by self-citations is defined as follows:

9
nLPIFjt=IFjt×log(cjtscjt) https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g024.jpg

Impact factor encouraged by the number of citing sources takes into consideration an influence sphere of the journal:

10
EIFjt=NjtNt×IFjt https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g025.jpg

where nt is the number of journals registered in JCR in the year t, njthttps://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g026.jpg is the number of journals citing the journal j in the year t.

In[30] a network scientometric approach is proposed for the identification of contextual productivity. In this work, for the assessment of contextual productivity of authors and journals, weighted 2 mode networks indices are analyzed and these indices can be used for gathering insights about most productive authors and journals by online databases and digital libraries.

In[31] research IF uses mathematical and statistical methods to analyze scientific publications and IF is a fundamental and universal measure of the journal’s value. Authors intend to publish their works in prestigious journals but journals’ editors intend to publish contributions that will be cited. In generally, compare with other tools for the evaluations of journals exist, the IF last 50 years has a strong prestige.

In[32] is showed presented that journal IF is able to discriminate between researchers who published their paper not only in the short term, but also in a long term.

In[33] are described general over view and approaches the Highly Cited Researchers by Clarivate Analytics. In paper JIF is proposed for assessment of ‘‘quality’’ of a researcher, their work, or a journal, and contributes to a great extent to driving scientific activities towards a futile endeavor.

Vincent Larivière and Cassidy R. Sugimoto research on a brief history, critique, and discussion of adverse effects of the JIF.[34]

In[35] research are discussed results on the use of the journal impact factor for assessing the research contributions of individual authors. “Minimum performance standards” include “number of authors on a paper”, “difference in citation density in various fields and subfields”, “citations differ in importance”. Imperfections and limitations of citation-based indicators make it difficult to gauge the differences in performance among highly productive authors. In research noted that, using a set of bibliometric indices (total citation number, Hirsh index, JIFs) and peers’ reviews are preferable for analysis of individual performance.

Proposed Version of Impact Factor

This section employs the indicators of two different WoS and Scopus databases in order to review the impact of the prestige of obtained citation on its IF. Hence, 5IF and SJR indicators are taken together as a measure of the prestige of a citation and weighted IF (IFα ) is determined by Eq.(11):

11
IFjα=IFjα(5IF,SJR)=Σi=1njt(1+α×5IFi+(1α)×SJRi)cijajt1+ajt2 https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g027.jpg

Where α (0 ≤ α ≤ 1) is a weight coefficient, SJRi is SJR indicator of a citing journal, respectively.

Unlike other weighted IFs (4, 7), in the proposed version, indicators of two various databases (WoS and Scopus) such as 5IF and SJR are used for the consideration of citing source prestige. For these indicators control their weighted linear combination are taken. Here you can control the effect of SJR and 5IF on the final IFα indicator by changing α[0;1]https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g028.jpg parameter. If α = 0, then the prestige of the citing source will only be determined by the SJR indicator. If α = 1, then the 5IF indicator will be included as the prestige of the citing source. If α = 0.5, then both indicators will be equally attributed, as the prestige of the citing source.

If, 5IF and SJR indicators equal 0 for any source, 5IF=SJR=0. In this case, if formula (11) did not have the first term (i.e. 1) under the sign of sum, then citations from this source(s) would not be taken into account in weighted calculation. Considering this case in formula (11), 1 was added to the expression. Thus, as the value of α increases from 0 to 1, in formula (11) the effect of the 5IF indicator will increase, and the effect of the SJR indicator will be decrease.

Data collection

In order to evaluate the weighted IFα indicator, journals in computer science field indexed in WoS and Scopus databases in 2013 is selected. In Table 2, 5IF and SJR values of citing journals in 2013 are presented in Table 3.

Table 2

Indicators of random selected journals in WoS and Scopus databases.

Title of JournalNumber of articles published in
2011-2012
Number citations to articles published in 2011-2012 in 2013
1Neural Computation226383
2Swarm Intelligence2648
3Neural Processing Letters7694
4Artificial Life4893
5Cognitive Computation8897
6Computer Speech And Language67121
7Fuzzy Optimization and Decision Making4567
8Genetic Programming and Evolvable Machines4245
9International Journal of Applied Mathematics and Computer Science136189
10Journal of Ambient Intelligence and Smart Environments7480
11ACM Transactions on Applied Perception4042
12ACM Transactions on Knowledge Discovery from Data3742
13ACM Transactions on Information Systems4255
14ACM Transactions on the Web3962
15ACM Transactions on Sensor Networks5479
16ACM Transactions on Software Engineering and Methodology3754
17IEEE Transactions on Computational Intelligence and AI in Games5058
18IEEE Transactions on Dependable and Secure Computing143163
19IEEE Transactions on Autonomous Mental Development5473
20World Wide Web: Internet and Web Information Systems5894
Table 3

5-year impact factor and SJR of citing journals and number of citations from them.

Journal 1
Neural Computation
Journal 2
Swarm Intelligence
5IFSJRNum. of cit.5IFSJRNum. of cit.5IFSJRNum. of cit.5IFSJRNum. of cit.5IFSJRNum. of cit.
135.89014.46129.4.0491.508757.2.6101.1242085.1.5720.56316.2263.2421
234.36027.85130.4.0172.128158.2.5671.128186.1.5500.63315.1651.8102
331.02019.58331.3.8792.63159.2.5261.869187.1.5291.03723.4482.2341
423.1700.923132.3.8792.336160.2.5251.178188.1.4020.55713.4051.6851
516.41011.79133.3.8441.473161.2.5010.9771089.1.4022.24013.0971.1782
616.40012.68434.3.7102.597162.2.4961.048190.1.3860.55722.7471.8881
714.5605.119135.3.7071.527163.2.4841.203291.1.3380.47211.9570.8321
814.46010.05136.3.6762.764164.2.3840.883192.1.3361.52811.6151.0191
913.4407.721137.3.6681.257265.2.3391.535193.1.3192.00411.5450.5362
1011.3406.318338.3.6461.21166.2.3071.289394.1.3141.71111.3641.0561
1110.5806.967939.3.6322.853967.2.2870.95395.1.3051.10810.9530.3332
1210.4404.835140.3.6122.598268.2.2700.808196.1.2310.5331
139.9245.586241.3.6072.621869.2.1581.323197.1.2160.2901
147.8692.631042.3.5681.444170.2.1431.632298.1.1921.1361
157.4635.316243.3.2912.341171.2.0000.8199.1.1830.5631
167.0634.374444.3.2191.777172.1.9471.2541100.1.1820.8321
176.8952.141345.3.1461.94173.1.9381.0753101.1.1400.7323
186.1447.212546.3.1082.033174.1.9220.2793102.1.0740.4701
196.0002.998147.3.0693.865175.1.8710.8411103.1.0321.0561
205.9393.2721448.3.0683.297376.1.8111.0078104.0.9700.5941
215.4843.346149.3.0501.355177.1.7670.8221105.0.8400.4051
224.8854.27150.2.9981.283378.1.7451.2051106.0.8160.4781
234.5442.126151.2.9271.829279.1.7320.6321107.0.6970.9491
244.4791.856452.2.8951.706180.1.7240.9691108.0.6720.4222
254.4221.472153.2.8921.554481.1.6431.2931109.0.6000.4361
264.2841.874554.2.7431.525182.1.6000.4831110.0.5480.3941
274.2502.049155.2.7331.196183.1.5960.7161111.0.2970.0001
284.2441.7221156.2.6531.407284.1.5951.1671112.0.1870.2462
Journal 3
Neural Processing Letters
Journal 4
Artificial Life
Journal 5
Cognitive Computation
Journal 6
Computer Speech and Language
Journal 7
Fuzzy Optimization and Decision Making
5IFSJRNum. of cit.5IFSJRNum. of cit.17.71010.4915IFSJRNum. of cit.5IFSJRNum. of cit.5IFSJRNum. of cit.
17.8543.422134.0.6400.355113.5605.73619.9245.58617.6942.09423.6762.7643
24.2682.161135.0.6220.73117.5102.81917.8695.68212.6431.67612.2181.7012
33.6762.764136.0.5610.64817.4355.70317.2983.72712.3950.64922.1671.4971
43.6322.853237.0.4970.24116.6900.98914.4791.85612.3391.53522.1651.5731
53.5131.641138.0.4970.33916.2263.24214.3722.42711.9521.53421.8141.46515
63.2191.77715.1651.81024.2441.72231.9361.07811.7210.7251
72.5010.97724.7281.72714.0172.12811.9150.75821.6741.1602
82.4571.20034.4461.50113.6741.82811.7081.53711.5791.3142
92.3840.88024.4061.89913.5981.37611.5200.84481.3860.5571
102.3391.53514.2441.72273.2622.38111.4231.01451.3641.0564
112.1431.63212.4961.04812.9981.28321.4101.14011.1830.5632
121.8310.66212.3330.70512.8471.04011.1460.20710.8460.3582
131.8111.00782.3071.28912.5380.99811.1370.58740.7460.5571
141.7100.12112.0000.79412.5251.17811.0740.47020.6120.4031
151.5291.03711.9450.66612.5010.97730.9770.29810.2690.00016
161.4541.19521.7771.06612.4451.44910.9590.6141
171.4200.53111.5450.50812.3391.53510.9320.4381
181.3600.99611.4541.19512.1941.40210.7670.2601
191.3290.62511.3640.44112.1581.32310.6640.3031
201.2310.533101.3360.66411.9381.07520.6170.8221
211.2160.29010.9530.33321.9361.07810.5050.4901
221.1830.56320.8160.47811.8111.07720.4660.2151
231.0740.47050.6170.82211.7451.20510.3050.3361
241.0740.68210.4800.35311.5960.6801
251.0400.51530.4170.24711.5291.0372
260.8980.50911.5200.8441
270.8660.39111.4230.5191
280.8400.40521.1570.4602
290.7740.56451.1370.58714
300.7550.32410.8460.2731
310.7530.55910.7350.3461
320.7160.62710.5920.6501
330.6820.25450.3260.2821
Journal 8
Genetic Programming and Evolvable Machines
Journal 9
International Journal of Applied Mathematics and Computer Science
Journal 10
Journal of Ambient Intelligence and Smart Environments
Journal 11
ACM Transactions on Applied Perception
Journal 12
ACM Transactions on Knowledge Discovery from Data
Journal 13
ACM Transactions on Information Systems
5IFSJRNum. of cit.5IFSJRNum. of cit.5IFSJRNum. of cit.5IFSJRNum. of cit.5IFSJRNum. of cit.5IFSJRNum. of cit.5IFSJRNum. of cit.
13.6762.76414.2441.722124.1.2160.29033.3822.03516.1447.21214.3953.70513.6762.7641
23.0275.08213.6762.764125.1.2010.77222.7001.51214.2832.91224.2441.72213.3711.9963
32.5261.86913.6013.715126.1.1830.56322.6321.51614.0172.12814.0172.12813.0371.8442
42.5010.97713.2121.545127.1.1820.83212.5251.17812.6202.09613.9592.58612.5661.9051
51.9381.07512.6401.2872281.1580.52412.3391.53512.6101.12413.6762.76432.4461.6721
61.8111.00712.6202.096129.1.1460.756522.0030.99812.5661.01013.3711.99612.3391.5352
71.7951.74312.4571.200130.1.0240.32111.9471.25412.4451.44923.2631.57111.7451.2051
81.7260.81022.3822.837131.0.9320.43811.8111.00722.3950.64913.0683.29711.7163.6373
91.6250.87512.2550.000132.0.8980.50921.6400.583162.2921.30612.4261.85811.5860.4271
101.3900.88212.1512.286133.0.8290.58711.5291.03712.0070.00012.3391.53531.3182.6371
111.3490.60412.0401.545434.0.8000.31611.1690.95112.0000.00011.8381.83611.1090.7062
121.2821.22181.7580.733135.0.8000.99931.9051.41311.8111.00710.4660.2151
131.2310.53311.6741.160136.0.6710.63711.3600.032121.3590.6222
141.6510.691137.0.6100.00011.2690.66350.7390.7401
151.6250.875138.0.5940.51031.2160.29010.7070.3231
161.5291.037239.0.5480.39411.1120.5591
171.5040.832140.0.4830.42210.6750.3601
181.4540.503141.0.4360.37810.5000.3021
191.3681.340242.0.4100.23610.4530.3011
201.3641.181143.0.3950.2771
211.3641.056144.0.3700.2901
221.3590.622145.0.2690.0001
231.2891.3502
Journal 14
ACM Transactions on the Web
Journal 15
ACM Transactions on Sensor Networks
Journal 16
ACM Transactions on Software Engineering and Methodology
Journal 17
IEEE Transactions on Computational Intelligence and AI in Games
Journal 18
IEEE Transactions on Dependable and Secure Computing
5IFSJRNum. of cit.5IFSJRNum. of cit.5IFSJRNum. of cit.5IFSJRNum. of cit.5IFSJRNum. of cit.5IFSJRNum. of cit.
17.8543.42226.3483.89723.6122.59813.2121.54516.8952.14111.1061.6091
24.3953.70516.1462.71213.3711.813.0712.44913.6762.76441.0930.7613
33.6762.76423.5873.54822.0631.30432.3391.53523.3711.99621.0190.6281
43.3711.99613.3711.99612.0311.34421.9361.07813.1912.84410.9540.5772
53.0371.84422.7471.88811.7561.29811.6250.875113.0712.44930.9450.1631
62.9271.82912.4851.00221.6921.12921.2821.22112.7443.08110.8740.3241
72.4461.67252.3950.64911.3220.86310.9540.57712.4261.85810.8670.3622
82.4240.00022.2031.71411.1671.15440.8320.42412.3391.53520.8190.3481
92.3391.53531.9570.83210.8670.36210.7230.26132.2591.53240.7930.4081
102.1581.32331.8591.44310.8190.34812.0671.44410.7120.5691
112.0331.25511.7580.73320.7850.56412.0211.58510.7010.1331
121.4691.86311.2270.68110.7270.76911.8941.37110.6970.3301
131.4520.84611.1830.56340.7210.68511.7260.810100.6970.9492
141.3880.80621.1690.95110.6820.39621.5760.91820.6250.4542
151.3841.10211.0920.76310.430.24911.4200.79910.6060.7501
161.3220.86321.0020.56910.3360.28821.3900.88210.6050.2311
171.1090.70610.7650.38510.269011.3411.64510.4970.3392
180.9430.42720.6050.23110.2680.21511.3220.86320.4630.2632
190.7850.56410.430.24951.3170.88910.430.2493
200.7650.38511.2910.74910.3950.2772
210.6050.23111.2341.12310.2970.0001
221.2270.68110.1870.2461
Journal 19
IEEE Transactions on Autonomous Mental Development
Journal 20
World Wide Web: Internet and Web Information Systems
5IFSJRNum. of cit.5IFSJRNum. of cit.5IFSJRNum. of cit.5IFSJRNum. of cit.
14.2441.7224111.4230.51913.6762.764111.1.5860.4271
23.5873.5481121.1370.58713.2191.777212.1.4520.84623
32.5010.9774131.0430.08013.1912.844113.1.2510.9381
42.2581.00017140.8980.50912.4461.672114.1.1690.9511
52.2021.5221150.8170.40812.4261.858115.0.9400.3661
61.9471.2541160.6750.36012.4030.000116.0.3320.1941
71.8591.4431170.5670.29112.3391.535217.0.3020.2491
81.6151.01932.0311.344118.0.1740.2761
91.5450.53621.9550.0001
101.5291.03711.8381.5471

In order to evaluate the proposed version IFα we have selected 20 journals in the Computer Science field indexed in JCR 2013. The proposed indicator has been calculated for these journals and compared with their 5-year IF, SJR, SNIP indicators and with the indicator W5IF proposed in.[8] Table 2 gives a list of the random selected journals analyzed in this study and their bibliometric characteristics, i.e. number of articles published in 2011-2012 and number of citations in 2013.

RESULTS

Table 4 presents the results of the proposed weighted by IFα taking 5IF and SJR indicators together as a measure of prestige of a citing source.

Table 4

Results of the proposed weighted IF.

Title of JournalIF
2013
5IF
2013
W5IF
2013
SJR
2013
SNIP
2013
IFα
α = 0α = 0.1α = 0.2α = 0.3α = 0.4α = 0.5α = 0.6α = 0.7α = 0.8α = 0.9α = 1
1Neural Computation1.6942.2217.5440.881.0974.94975.20915.46855.72795.98736.24676.50616.76557.02487.28427.5436
2Swarm Intelligence1.83303.4720.7571.6952.60272.68972.77662.86362.95063.03753.12453.21153.29843.38543.4724
3Neural Processing Letters1.2371.3282.8050.5331.5452.07002.14352.21712.29062.36422.43772.51122.58482.65832.73192.8054
4Artificial Life1.931.84.7620.5081.1743.20253.35843.51433.67023.82613.98204.13794.29384.44974.60564.7615
5Cognitive Computation1.11.3942.5940.5871.1381.87471.94662.01862.09052.16242.23442.30632.37822.45022.52212.5941
6Computer Speech And Language1.8121.7762.9041.0143.2272.36812.42172.47542.52902.58262.63632.68992.74352.79722.85082.9044
7Fuzzy Optimization and Decision Making12.0553.1511.4652.5552.66662.71502.76342.81192.86032.90882.95723.00573.05413.10263.1510
8Genetic Programming and Evolvable Machines1.0651.42.0141.2213.5681.81291.83311.85321.87341.89351.91371.93391.95401.97421.99432.0145
9International Journal of Applied Mathematics and Computer Science1.391.3172.4650.7561.712.09612.13302.17002.20692.24382.28072.31762.35452.39142.42832.4652
10Journal of Ambient Intelligence and Smart Environments1.0821.2521.7580.5831.631.39671.43291.46901.50511.54121.57731.61351.64961.68571.72181.7579
11ACM Transactions on Applied Perception1.0511.5662.2470.6632.0241.61601.67911.74221.80541.86851.93161.99472.05792.12102.18412.2472
12ACM Transactions on Knowledge Discovery from Data1.14702.611.1043.3432.13272.18052.22832.27612.32402.37182.41962.46742.51522.56302.6108
13ACM Transactions on Information Systems1.31.672.310.8363.2612.16412.17882.19342.20802.22262.23722.25192.26652.28112.29572.3103
14ACM Transactions on the Web1.5951.9663.8281.6724.9762.91463.00593.09733.18873.28013.37143.46283.55423.64563.73693.8283
15ACM Transactions on Sensor Networks1.4632.7542.6071.0023.1932.14212.18852.23502.28152.32802.37452.42102.46752.51402.56052.6070
16ACM Transactions on Software Engineering and Methodology1.4721.6942.4131.3043.9712.12412.15302.18192.21082.23972.26862.29752.32642.35532.38422.4131
17IEEE Transactions on Computational Intelligence and AI in Games1.1671.2741.880.8752.7331.57541.60591.63641.66691.69731.72781.75831.78881.81921.84971.8802
18IEEE Transactions on Dependable and Secure Computing1.1371.2761.9850.9183.1741.69621.72511.75391.78281.81161.84051.86931.89821.92701.95591.9847
19IEEE Transactions on Autonomous Mental Development1.3481.8753.03712.5862.15712.24512.33302.42102.50902.59692.68492.77292.86082.94883.0368
20World Wide Web1.6231.362.8320.8462.4272.33642.38592.43542.48502.53452.58402.63352.68302.73262.78212.8316

ANALYSIS

In order to compare the results obtained by IFα indicator with the results of IF, W5IF, SJR and SNIP indicators, we have used Pearson correlation, cosine measure and Euclidean distance.

The cosine dissimilarity measure between the vectors A = (a1, a2, ..., an) and B = (b1, b2,... bn) can be calculated as follows:

1– cos (A, B) = disscos(A, B)    (12)

where cos(A,B) is the cosine similarity measure between the vectors A and B:

13
cos(A,B)=Σi=1naibiΣi=1nai2Σi=1nbi2 https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g029.jpg

The Euclidean distance between A and B vectors can be calculated by the following formula:

14
dist(A,B)=Σi=1n(aibi)2 https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g030.jpg

The results of Pearson correlation between Fα and other indicators are given in Table 5.

Table 5

Pearson correlations between each pair of journal indicators.

5IFW5IFSJRSNIPIFα
α = 0α = 0.1α = 0.2α = 0.3α = 0.4α = 0.5α = 0.6α = 0.7α = 0.8α = 0.9α = 1
IF0.10460.5937-0.0489–0.07040.61390.61230.61040.60830.60620.60400.60180.59970.59760.59560.5937
5IF0.28380.23250.12330.30190.29980.29770.29570.29370.29180.29000.28830.28670.28510.2837
W5IF0.0166–0.32740.98030.98600.99020.99330.99560.99720.99840.99920.99970.99991.0000
SJR0.82090.14510.12520.10750.09180.07760.06490.05340.04290.03340.02460.0166
SNIP–0.2093–0.2284–0.2452–0.2599–0.2729–0.2845–0.2949–0.3043–0.3127–0.3204–0.3273

As seen from Table 5, the correlation between the IFα and IF is not strong and as the value of α incrases in [0;1] interval, in other words, as the weight of SJR in weighed IFα increases, the correlation weakens. The most noteworthy feature is that, IFα is more poorly correlated with 5IF [0.2837; 0.6139] than with IF [0.5937; 0.6139]. That is, as the value of α increases, the correlation becomes weaker due to the fact that, the proposed IFα considers the impact of SJR indicator. Among these indicators, IFα is the most weakly correlated with SNIP indicator. Same case is also observed between IF and SNIP (– 0.0704). However, the correlation between IFα and SNIP [-0.3273; -0.2093] is lower in than the correlation between IF and SNIP (– 0.0704).

The results of cosine dissimilarity between the weighed IFα and other indicators is given in Table 6.

Table 6

Cosine dissimilarity measure between all pair of indicators.

5IFW5IFSJRSNIPIFα
α = 0α = 0.1α = 0.2α = 0.3α = 0.4α = 0.5α = 0.6α = 0.7α = 0.8α = 0.9α = 1
IF0.08760.05170.07240.09650.03060.03260.03470.03690.03900.04120.04330.04550.04760.04970.0517
5IF0.10760.09460.12540.08630.08840.09060.09270.09490.09710.09920.10140.10350.10550.1076
W5IF0.12330.19350.00610.00460.00340.00250.00170.00110.00070.00040.00020.00000.0000
SJR0.02290.08260.08720.09170.09610.10030.10450.10850.11230.11610.11970.1232
SNIP0.14200.14810.15390.15960.16500.17020.17520.18010.18470.18920.1935

As seen from Table 6, the results of the IFα and IF are more similar [0.0306; 0.0517] than the results of 5IF, W5IF, SJR and SNIP. As the value of α increases, in order words, as the weight of SJR in weighted IFα proposed becomes larger, the similarity increases. The results of IFα and 5IF [0.0863; 0.1076] are more similar rather than IFα and IF results [0.0306; 0.0517]. As the value of α of IFα increases, the similarity of W5IF results decline [0; 0.061]. The similarity between the weighted IFα and SJR is stronger [0.0826; 0.1232] than the similarity between IF and SJR (0.0724). The similarity between the results of IFα and SNIP [0.1420; 0.1935] is stronger than the similarity between IF itself and SNIP (0.0965).

The results of Euclidean distance between the weighted IFα proposed and other indicators is given in Table 7.

Table 7

Euclidean distance between all pair of indicators.

5IFW5IFSJRSNIPIFα
α = 0α = 0.1α = 0.2α = 0.3α = 0.4α = 0.5α = 0.6α = 0.7α = 0.8α = 0.9α = 1
IF2.99328.67562.75727.15514.96565.32805.69336.06106.43076.80207.17467.54857.92338.29918.6756
5IF 8.56083.79506.90825.10355.42595.75596.09236.43406.78037.13067.48427.84078.19978.5609
W5IF  10.77308.53543.84473.46033.07582.69132.30681.92241.53791.15340.76900.38460.0015
SJR   8.08897.01707.38857.76138.13528.51008.88569.26199.639010.016410.394510.7730
SNIP    6.32066.47376.64566.83477.03997.25967.49257.73767.99368.25978.5348

As seen from the Table 7, as the value of α increases the Euclidean distance between IFα and IF increase thus their results diffren from each other [4.9656; 8.6759]. The results of the Euclidean distance of IFα and 5IF differ more as the value of α increases. The results of the Euclidean distance varies less as the value of α of IFα and W5IF increases. The results of the proposed weighted IFα differs from SJR more [7.017; 10.773] than the results of IFα (2.7572). As the value of α increases, the results of Euclidean distance of the proposed weighted IFα and SNIP differ more [6.3206; 8.5348] than IF and SNIP results (7.1551).

In conclusion of all results, it is clear that, the proposed weighted IFα has made an impact on W5IF results. Figures have been used in order to visually illustrate the aforementioned in Tables.

As seen from the Figure 1, 2, 3, the indicators IF, 5IF and SNIP have demonstrated a deterioration from the value of α in [0;1] interval, however, W5IF has shown an improvement within [0;1] interval in all graphs.

Figure 1

Impact of the parameter α to the Pearson’s correlation coefficient between IFα and other indicators.

https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g031.jpg
Figure 2

Impact of the parameter α to the cosine dissimilarity measure between IFα and other indicators.

https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g032.jpg
Figure 3

Impact of the parameter α to the Euclidean distance between IFα and other indicators.

https://s3-us-west-2.amazonaws.com/jourdata/Jscires/JScientometRes-7-2-94_g033.jpg

CONCLUSION

Consideration of the reputation of citing source is necessary for the assesment of Journal IF. In this regard, a number of IF modifications are proposed by various researchers. The study showed that using only one indicator as a prestige of citing source is not so good. For this purpose, as prestige of citing source it is advisable to use different indicators. In paper to verify the accuracy of the results, it is inevitable to use various metrics (Pearson correlation coefficient, cosine and Euclidean distances) for comparing value with IF value. Because outcome can differ from one metric to another. Experiments affirmed it once again. In the proposed method as the prestige of citing source using two various indicators at the same time are suggested. Using not only two but also more indicators are the advantages of proposed method. Considering all aforementioned, prospective research works will review new and modified methods for more efficient evaluation of journals.

CONFLICT OF INTEREST

We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing the position presented in, or the review of, the article entitled, “Journal Impact Factor Weighted by SJR and 5-Year IF indicators of Citing Sources”. The study was conducted by the authors, no other people involved in conceiving, performing, and writing this study.

ABBREVIATIONS

IF

Impact Factor

WIF

Weighted IF

5IF

Five Year Impact Factor

W5IF

Weighted Five year Impact Factor

SJR

Scientific Journal Rankings

SNIP

Source Normalized Impact per Paper

WoS

Web of Science

RIP

Raw Impact per Paper

RDCP

Relative Database Citation Potential.

REFERENCES

1. 

Gross PLK , author. College libraries and chemical education. 1927. p. 385–9

2. 

Garfield E , author. Citation indexes for science. A new dimension in documentation through association of ideas Science. 1955;122:108–11

3. 

Liu XL, Gai SS, Zhang SL, Wang P , authors. An Analysis of Peer-Reviewed Scores and Impact Factors with Different Citation Time Windows: A Case Study of 28 Ophthalmologic Journals. PLoS ONE. 2015. p. 10

4. 

Sadeghi R, Sarraf AS , authors. Comparison between Impact factor, SCImago journal rank indicator and Eigenfactor score of nuclear medicine journals. Nuclear Medicine Review. 2012;15(2):132–6

5. 

Moed HF , author. SNIP Journal Impact Indicator Accounts for Differences in Citation Characteristics and Database Coverage between Properly Defined Subject Fields. Against the Grain. 2010;22(4):34–8

6. 

Mingers J, Yang LY , authors. Evaluating journal quality: A review of journal citation indicators, and ranking in business and management. European Journal of Operational Research. 2017;257(1):323–37

7. 

Stefanie H , author. Multidimensional Journal Evaluation (Analyzing Scientific Periodicals beyond the Impact Factor). De Gruyter Saur. 2012. p. 408

8. 

Alguliyev RM, Aliguliyev RM, Ismailova NT , authors. Impact factor weighted by 5-year impact factor. Problems of Information Technology. 2015;2:31–40

9. 

Colledge L, Moya-Anegon F , authors. et al. SJR and SNIP: two new journal metrics in Elsevier’s Scopus. Serials. 2010;23(3):215–21

10. 

Moed HF , author. Measuring contextual citation impact of scientific journal. Journal of Informetrics. 2010;4(3):265–77

11. 

Cantin M, Munoz M, Roa I , authors. Comparison between Impact Factor, Eigenfactor Score and SCImago Journal Rank Indicator in Anatomy and Morphology Journals. International Journal of Morphology. 2015;33(3):1183–8

12. 

Leydesdorff L, Opthof T , authors. Scopus’s Source Normalized Impact per Paper (SNIP) Versus a Journal Impact Factor Based on Fractional Counting of Citations. Journal of the American Society for Information Science and Technology. 2010;61(11):2365–9

13. 

Waltman L, van Eck NJ , authors. Some comments on the journal weighted impact factor proposed by Habibzadeh and Yadollahie. Journal of Informetrics. 2008;2(4):369–72

14. 

Falagas ME, Kouranos VD, Arencibia-Jorge R, Karageorgopoulos DE , authors. Comparison of SCImago journal rank indicator with journal impact factor. Faseb Journal. 2008;22(8):2623–8

15. 

Altmann KG, Gorman GE , authors. The usefulness of impact factor in serial selection: A rank and mean analysis using ecology journals. Library Acquisitions-Practice and Theory. 1998;22(2):147–59

16. 

Garfield E , author. The history and meaning of the journal impact factor. Jama-Journal of the American Medical Association. 2006;295(1):90–3

17. 

Pinto AC, de Andrade JB , authors. Impact factor of scientific journals: What is the meaning of this parameter?. Quimica Nova. 1999;22(3):448–53

18. 

Saha S, Saint S, Christakis DA , authors. Impact factor: a valid measure of journal quality?. Journal of the Medical Library Association. 2003;91(1):42–6

19. 

Kochen M , author. Principles of information-retrieval. Management International Review. 1975;15:134

20. 

Pinski G, Narin F , authors. Citation influence for journal aggregates of scientific publications - theory, with application to literature of physics. Information Processing and Management. 1976;12(5):297–312

21. 

Rogers I , author. The Google pagerank algorithm and how it works. 2006. http://www.iprcom.com/papers/pagerank/

22. 

Bollen J, Rodriguez MA, Van De Sompel H , authors. Journal status. Scientometrics. 2006;69:669–87

23. 

Buela-Casal G , author. Evaluating quality of articles and scientific journals. Proposal of weighted impact factor and a quality index?. Psicothema. 2003;15:23–35

24. 

Habibzadeh F, Yadollahie M , authors. Journal weighted impact factor: A proposal. Journal of Informetrics. 2008;2(2):164–72

25. 

Waltman L, van Eck NJ, van Leeuwen TN, Visser MS , authors. Some modifications to the SNIP journal impact indicator. Journal of Informetrics. 2013;7(2):272–85

26. 

Zitt M, Small H , authors. Modifying the journal impact factor by fractional citation weighting: The audience factor. Journal of the American Society for Information Science and Technology. 2008;59(11):1856–60

27. 

Zyczkowski K , author. Citation graph, weighted impact factors and performance indices. Scientometrics. 2010;85(1):301–15

28. 

Zitt M , author. Behind citing-side normalization of citations: some properties of the journal impact factor. Scientometrics. 2011;89(1):329–44

29. 

Alguliyev RM, Aliguliyev RM , authors. Modified Impact Factors. Journal of Scientometrics Research. 2016;5(3):197–208

30. 

Lathabai HH, Prabhakaran T, Changar M , authors. Contextual productivity assessment of authors and journals: a network scientometric approach. Scientometrics. 2017;110(2):711–37

31. 

Andrzej GMD, Rafał PJD , authors. Impact factor: Universalism and reliability of assessment. Clinics in Dermatology. 2017;35(3):331–4

32. 

Bornmann L, Williams R , authors. Can the journal impact factor be used as a criterion for the selection of junior researchers? A large-scale empirical study based on ResearcherID data. Journal of Informetrics. 2017;11(3):788–99

33. 

Jaime A , author. Teixeira da Silva, Sylvain Bernès. Clarivate Analytics: Continued Omnia vanitas Impact Factor Culture. Science and Engineering Ethics. 2018;24(1):291–7

34. 

Vincent L, Cassidy R , authors. Sugimoto. The Journal Impact Factor: A brief history, critique, and discussion of adverse effects. https://arxiv.org/abs/1801.08992

35. 

Pudovkin AI , author. Comments on the Use of the Journal Impact Factor for Assessing the Research Contributions of Individual Authors. Frontiers in Research Metrics and Analytics. 2018;3(2):1–4. doi.org/10.3389/frma.2018.00002.