Introduction

Several studies in the literature that aim to ascertain the scientific status of a journal have mostly used citation analysis (1, 2, 3, 4, 5). The sequential analysis of a journal’s publications with citations provides standard data regarding the publication trends with citation indicators of the journal, allows to evaluate the improvement of a journal in terms of scientific status and enables the editorial board to consider their publication policies to augment the impact of a journal. It also enables to measure the effect of important experiences of a journal such as the inclusion of scientific indexes or change in publication policies.

The Turkish Archives of Otorhinolaryngology (TAO) is the scientific, peer-reviewed, open-access journal of the Turkish Otorhinolaryngology Head and Neck Surgery Society. TAO is the first Turkish otorhinolaryngology (ORL) journal that is in publication since November 1962 (6). The publication of the TAO represents a milestone in the Turkish ORL history and is regarded as a necessary step on the developmental path of the Turkish ORL literature. Professor Mehmet Hikmet Altuğ, the journal’s first editor, expressed his visionary stance pertaining to scientific publishing in the first issue as: “Our archive, we believe, fills a gap in a world getting smaller with radio, television, and everyday changing medicine,’’ (7).

With the intense efforts of previous Editorial Boards along with the current successor, TAO was indexed by the TÜBİTAK ULAKBIM TR database, as well as two top international indexes—the Emerging Sources Citation Index (ESCI) and the PubMed Central in 2015. In the same period, also the journal’s language of publication was changed to English. While such episodes that involve major changes can significantly affect the scientific status of a journal, the exact impact of these alterations should nevertheless be verified with a comprehensive analysis. Studies on this subject matter are rare in the Turkish otorhinolaryngological discipline (8, 9, 10, 11). The primary aim of the presented study was to perform sequential analyses on the characteristics of the studies published in the TAO during the periods from 2010 to 2014 and 2015 to 2019. A comparison in terms of citation performance and characteristics was also performed for these periods.

Methods

The articles published in the TAO were initially reviewed through the archives of the journal for the periods from 2010 to 2014 and 2015 to 2019 in terms of study type (experimental study, clinical study, case report, review, letter to editor, clinical image, editorial article, historical article), study topic (otology, rhinology, head and neck, pediatric ORL and general ORL), language (Turkish/English) and country of origin. Subsequently, the citation analysis of the articles was made using the Google Scholar and Web of Science (WoS) databases based on the total number of citations, the mean number of citations per study, the mean number of citations per study type, the country of origin of the citations, the branch of the journals (ORL or other) in which they were cited, number of citations in TAO from the previous issues, WoS index types of citations, and the total number of citations with the mean per article for the studies published from Turkey and foreign countries. Google Scholar can count a paper multiple times, therefore duplications were reviewed and removed to obtain the accurate number of citations in the Google Scholar database analysis. WoS index types include Science Citation Index Expanded (SCI-E), ESCI and other WoS indexes (Conference Proceedings Citation Index-Science, Book Citation Index-Science). Our study was done in January 2021, hence, the analysis included citations made until this date and only articles written in English or Turkish were accepted as citation. However, due to the absence of citation reports of the articles published from 2010 to 2014 in the WoS database, only Google Scholar was used for the analysis of these articles. Then, the journals where the citations were published were analyzed manually to ensure the involvement of WoS indexes for this period.

Currently, the TAO is not indexed by Clarivate Analytics (previously the Thomson Institute for Scientific Information), therefore, no official impact factor was calculated for the journal. Nevertheless, the estimated impact factor (EIF) of a journal can be calculated by dividing the total number of citations made in the projected year to the total number of citable articles (original investigation, case report and review) published in the past two years. Because WoS databases started to include the articles published in the TAO after 2015, it was possible to calculate the EIFs only for the years 2017-2020. EIF was calculated manually according to the citations obtained from WoS databases.

Data analyses, including mean values and proportions, were calculated using Microsoft Excel 2010.

Results

The total number of articles published from 2010 to 2014 and from 2015 to 2019 was 144 and 214, respectively. Characteristics of articles published in the TAO from 2010 to 2014 and from 2015 to 2019 are shown in Table 1. From 2010 to 2014, the most frequent study topic was head and neck with the case reports having the highest rate among the study types. In 2015–2019, the most frequent study type had changed to original investigation and topic to general ORL. Of the original investigations 58 (98.3%) were clinical and one (1.7%) was experimental study in 2010–2014, whereas there were 116 (94.3%) clinical and seven (5.7%) experimental studies in 2015–2019.

From 2010 to 2014, the total number of citations was 75 for 122 articles according to the Google Scholar database with a mean of 0.61 (Figure 1). Twenty-two articles could not be retrieved from the database. Five of all citations (6.7%) were performed in other articles published in the TAO. While 20 out of 75 citations (26.7%) were of articles written in Turkish, the remaining 55 were (73.3%) of articles written in English. The total number of citations in the journals listed in WoS indexes was 31 (0.21 per study) (Table 2), of which 18 citations (58.1%) were in journals indexed in SCI-E and 13 citations (41.9%) in journal indexed in ESCI (Figure 2). Of the 31 citations, 13 (41.9%) were published in ORL journals, and 18 (58.1%) in other journals. While 16 citations (51.6%) were made by a group with a Turkish first author, 15 (48.4%) were cited by authors from other countries (Table 3). The numbers of citations made by Turkish authors indexed in ESCI and SCI were nine and seven, respectively.

According to the WoS and Google Scholar databases, the total numbers of citations for the 214 published articles in the period from 2015 to 2019 were 240 (mean: 1.12) and 398 (mean: 1.85), respectively (Figure 1). The characteristics of the citations are shown in Table 2. Of the 240 citations retrieved from the WoS, 77 (32.1%) were in ORL journals and 163 (67.9%) were in other journals. Fifty-eight out of 240 (24.2%) citations were made by a group with a Turkish first author and 182 (75.8%) were made by foreign authors (Table 4). The total numbers of the citations according to the WoS indexes including the SCI-E, the ESCI and other indexes were 169 (70.4%), 67 (27.9%) and four (1.7%), respectively (Figure 2). While of the 58 citations performed by Turkish authors, 30 (51.7%) were in journals indexed in SCI-E, 28 (48.3%) were in journals indexed in ESCI. Also, nine of all citations (3.8%) were performed in other articles published in the TAO.

The EIFs that were manually calculated for TAO for each year from 2017 to 2021 are shown in Table 5. The highest EIF was found in 2020.

Discussion

The publish or perish concept is still valid in the academia, however, the growing body of scientific works is imposing to expand the area of publication (12). As a result, for-profit journals have increased their share in the domain and become a significant rival for not-for-profit journals. Community-based journals, including those published by scientific societies, however, offer clear benefits to the scientific community in that they are peer-approved and certified, accessible and supportive, agile and responsive, relevant, recognized, and targeted (13). As a follower of these universal concepts, the TAO has been endeavoring to improve the Turkish ORL literature nearly for 60 years, representing a substantial part of the Turkish Otorhinolaryngology-Head and Neck Surgery Society. To support the propitious attempts of the Turkish Otorhinolaryngology-Head and Neck Surgery Society aimed at serving for the development of community-based publishing in the Turkish ORL literature, we intended to carry out the present bibliographic study on the last decade of the TAO.

To our knowledge, the presented study is the first to conduct the sequential analysis of a Turkish ORL journal. According to the results of the presented study, there was a remarkable increase in the number of citations in the period 2015–2019 compared to 2010–2014. There also was a remarkable increase in the EIF values for 2019 and 2020. Citations as a classical bibliometric tool still hold their position as a significant indicator for the scientific influence of a journal, despite its substantial drawbacks (3). Citations majorly measure the impact of a researcher as it pertains to other researchers without examining knowledge translation and reader uptake. Hence, its bibliometric significance seems to be an ongoing controversy and promotes the quest for alternative bibliometric indicators such as Altmetrics which measure the attention gathered by a work on the Web (14). However, it should be emphasized that new bibliometric indicators have also some disadvantages such as lack of standardization or control of scientific community. Therefore, editorial boards should primarily focus on the scientific merit of the submitted manuscripts and bibliometric indicators should be used as auxiliary tools for improving the scientific quality of the journal.

In the presented study, head and neck (28.5%) and rhinology (25%) were found to have the leading number of publications among the study topics in 2010 to 2014, whereas general ORL (28.1%) and head and neck (22.9%) had the highest number of publications in 2015–2019. However, in the context of the number of the citations, publications regarding otology (0.29) and rhinology (0.22) had the highest mean value of citations among the study topics in 2010–2014, whereas pediatric ORL (1.23) and otology (1.2) were the leading topics in terms of mean citation values in 2015–2019. In 2002, Fenton et al. (15) analyzed ORL citation classics from 1900 to 1999 and reported that otology/lateral skull base surgery (48.75%) and head neck surgery (33.75%) were the most frequent topics in the list. More recently, Coelho et al. (16) revisited the citation classics in ORL journals indexed in the WoS in 2014 and found that otology (51.7%) and head neck surgery (37.8%) were the leading topics. It should, however, be noted that the studies of Fenton et al. (15) and Coelho et al. (16) did not involve pediatric ORL as a topic. The results of the presented study showed that the citation trend shifted to pediatric ORL and otology in the TAO during 2015–2019. Like the presented study, Erdağ et al. (17) reported that otology and pediatric ORL were the most cited topics among the 100 most cited Turkish papers in the ORL journals of the WoS.

Reviews and original investigations were the most cited study types in the presented study with mean numbers of citations of 1.45 and 1.29 in 2015–2019, respectively. Original investigations and case reports, on the other hand, had the highest mean values of citations (0.25 and 0.18, respectively) in 2010–2014. As the second most published study type in the TAO in 2015–2019, case reports had 0.73 citation per study. Although the historical origin of the case reports dates to the BC era (18), the contribution of the case reports to the development of science have become a matter of debate especially in the last 20 years (19). Concerns related to the low evidence levels and the citation potentials of case reports are likely to be the most important factor for the editors’ reluctance to publish a case report instead of more solid options, such as original investigations, reviews or meta-analyses, for improving the impact factor of a journal. Nevertheless, the benefits of case reports, such as informing the scientific community about new diseases or treatment methods should not be ignored. Furthermore, case reports have also become a significant contributor to initiate and/or improve the preparation of a scientific article for recruiters (20, 21). In the presented study, in contrast to the common beliefs concerning the low citation potential of a case report, we found that case reports published in the TAO in the years 2015 to 2019 had received considerable number of citations (46 citations for 63 article) which may support the ongoing significance of the case reports in the scientific publication.

Although there is no clear value regarding the exact rate accepted as a national bias of citation, improper national citations for improving a journal’s impact factor may yield to ethical misconduct like those experienced in some Brazilian journals in 2012 (22). Also, encouraging the citation of a journal’s own papers to artificially boost the impact factor of a journal may be considered as an ethical misconduct. However, citations to high-quality studies, whether national or self-citation to the journal’s own papers, should not be subjected to ethical consideration, therefore, researchers should not hesitate, due to ethical concerns, to cite these papers. Erdağ et al. (10) reviewed the references of studies published in the 2015 issues of four Turkey-based ORL journals and found that 460 of 2,708 references (16.98%) had a Turkish first author. The authors also determined that 85 articles published in ORL journals indexed by SCI in the same period had a Turkish first author. In these articles, 271 of total 2,252 (12.03%) references were from Turkey and 18 of these were published in journals from Turkey. The data provided by Erdağ et al. (10) suggest that Turkish authors rather tend to cite national studies, both in their studies published in SCI-indexed ORL journals or national ORL journals. In the presented study, 58 out of 240 (24.2%) citations retrieved from the WoS databases for the articles published in the TAO in 2015–2019 were by Turkish authors, whereas 182 (75.8%) were by foreign authors. It is also worth noting that 30 out of 58 citations (51.7%) made by Turkish authors were in journals indexed in the SCI-E, whereas 28 (48.3%) were in journals indexed in the ESCI. Interestingly, there were only nine citations (3.8%) to TAO’s own papers.

The presented study has several limitations. First, citation for an article is a dynamic and ongoing process and databases themselves have inherent limitations that may cause potential errors. Second, the bibliometric analysis of the citations regarding the period 2010–2014 was done only through Google Scholar databases since the articles published in the TAO were not listed in the WoS databases in this period. Twenty-two out of 214 articles published in the years 2010 to 2014 were not included in Google Scholar, therefore the citation analysis of these articles could not be done. Third, the TAO had no official journal citation report created by Clarivate Analytics, therefore, impact factors values were calculated manually.

Conclusion

Although the increase in the number of citations and in impact factor values cannot be appreciated as a single indicator for the success of a journal, the results of the presented study revealed a promising advancement in the scientific quality of the TAO, driven by the inclusion of the journal in the national and international indexes, the change in the language of the journal to English and the well-orchestrated editorial efforts.

Main Points

• Turkish Archives of Otorhinolaryngology was first indexed in TÜBİTAK ULAKBIM TR, Emerging Sources Citation Index and PubMed Central after its publication language was changed to English in 2015.

• While the most frequent study type was case report and most frequent topic was head and neck in 2010 to 2014, these were original investigation and general otorhinolaryngology, respectively, in 2015 to 2019.

• According to Google Scholar and WoS databases, there was a remarkable increase in the total number of citations in 2015–2019.

• The presented study showed a promising advancement in the scientific quality of the TAO, driven by the inclusion of the journal to national and international indexes and by changing the language of the journal to English, as well as the well-orchestrated editorial efforts.

Ethics Committee Approval: This study does not require the ethics committee approval due to the fact that the study is a citation analysis.

Informed Consent: This study does not require informed consent due to the fact that the study is a citation analysis.

Peer-review: Externally peer-reviewed.

Financial Disclosure The authors declared that this study has received no financial support.