Abstract

Objective

Principles of transparency and best practice in scholarly publishing is one of the important standards for the functioning and publishing quality of peer-reviewed scientific journals. The aim of this study is to evaluate Turkish otorhinolaryngology, head and neck surgery (ORL-HNS) journals according to these principles and to point out the areas that need improvement.

Methods

This descriptive study is based on the evaluation of website contents of eight Turkish ORL-HNS journals according to the 16 principles of transparency criteria. The number of scientific papers published in 2020 and 2021 were retrieved from the respective websites of the journals. The impact factors were calculated by analyzing the citations in 2022 via Google Scholar. The probable relationship between impact factor and compliance with transparency principles was investigated. Impact factor and transparency principles were studied to draw attention to the international standards which can contribute to journals for international scholarly publishing.

Results

The journals highly comply with website publishing, ethics, access, and ownership criteria; however, most of them do not comply with advisory council, advertising, other income, and business practices criteria. While the first three journals with the highest impact factors comply with 12 to 14 of the 16 criteria, the last three comply with five to 12.

Conclusion

The journals with high transparency criteria scores and high impact factors suggest that these criteria are important in terms of the reliability and validity of the information, and citation. Moreover, the websites of Turkish scientific ORL-HNS journals were seen to need improvement according to the transparency criteria, especially regarding financial issues such as business, financial status, and advertising.

Introduction

Peer-reviewed scientific journals have made great contributions to the development of science. There are many scientific journals in many fields of expertise where scientific research results are revised and published with peer evaluation. Criteria for the functioning of these journals have been developed and published by various international organizations. One of these resources, the principles of transparency and best practice in scholarly publishing was developed and first published in 2013 and revised in 2022 by the Committee on Publication Ethics (COPE), the Directory of Open Access Journals (DOAJ), the Open Access Scholarly Publishing Association (OASPA), and the World Association of Medical Editors (WAME). The principles of transparency consist of 16 criteria related to publishing standards, such as name, website, publication calendar, archive, and license, which should be stated on the website of the journals (1).

Journals that meet the principles of transparency can be indexed by DOAJ. Especially in recent years, due to the increasing number of predatory journals, authors have begun to be more careful about not publishing in predatory journals. WAME developed a guide to identify predatory journals (2). One of the most important criteria of this guide for choosing a journal to submit a manuscript is whether the journal is indexed by DOAJ. Therefore, it has become important for journals to comply with the transparency principles developed by these four important international publishing organizations under the leadership of DOAJ. Studies have shown that this compatibility is related to publishing quality (3).

Regarding the advancements about scientific publishing, the publishing quality of journals in the field of otorhinolaryngology, head and neck surgery (ORL-HNS) continues to improve in Türkiye. A study examining case reports in Turkish ORL-HNS journals, mentioned that case reports that adopted various principles such as complying with ethical standards and being easily accessible would be very supportive of moving Turkish journals to more preferred indices (4). In our study, the compliance of ORL-HNS journals published in Türkiye was studied based on the principles of transparency. The aim was to investigate and analyze whether the journals complied with these principles and thereby inform journals by pointing out the aspects that needed improvement. Also, principles that are more likely to be incompatible with this field were examined and a possible relationship between journal citation numbers and the index of compatibility with transparency principles was evaluated to draw attention to global standards that could be useful in international scholarly publishing.

Methods

Data Collection

COPE, DOAJ, OASPA, and WAME are important scientific organizations. In 2013, they published 16 criteria under the title Principles of Transparency and Best Practice in Scholarly Publishing. These criteria were revised, and the 4^th version was published in 2022. The 16 criteria are the name of the journal, its website, publishing schedule, archive, copyright, licensing, publication ethics and related editorial policies, peer review, access, ownership and management, an advisory body, editorial team-contact information, author fees, other revenue, advertising, and direct marketing. In addition, these 16 criteria are grouped under four main headings: journal content, journal practices, organization, and business practices (1).

In this sub-analysis, the statuses of eight ORL-HNS journals published and registered in Türkiye were scored as 1 for compliance and 0 for non-compliance with a range of minimum 0 and a maximum 16 points, as shown in Table 1 and 2. Accordingly, the compatibility scores of the journals with the transparency criteria were calculated. The conditions required for the conformity with the criteria are explained in Table 1. Journals that met more than half of the conditions for a criterion were considered to meet that criterion.

The Turkish Archives of Otorhinolaryngology, the European Journal of Rhinology and Allergy, The Turkish Journal of Ear, Nose and Throat, the KBB Forum Electronic Journal of Otolaryngology-Head and Neck Surgery, the Journal of Ear Nose Throat and Head Neck Surgery, the Praxis of Otorhinolaryngology, the ENT Updates, The Journal of International Advanced Otology were included in the analysis which was carried out by examining the websites of the journals (Table 2). The analyses were conducted after each journal’s website was evaluated by two researchers independently to prevent bias. The aim was to assess which transparency principles Turkish ORL-HNS journals complied with and which aspects needed to be improved.

In addition, the number of scientific articles published by the journals in 2020 and 2021 was collected by searching the journals’ website archives, and the number of citations received by the relevant publications in 2022 was calculated using Google Scholar (5). First, the presence of a relationship between the calculated 2022 impact factor and the journal’s compliance with transparency principles was calculated by dividing the number of citations a journal’s 2020 and 2021 articles received in 2022 into the total number of citable articles published in that journal in the same two years. Journals for which a relationship was found were further evaluated through statistical analysis to identify with which transparency criteria/transparency criteria subheadings it was more closely related. As indicated in the COPE guidelines, ethics committee approval is not required for secondary research in which publicly available data is accessed (7).

Statistical Analysis

The possibility of a relationship between the impact factor in 2022 and compliance with the transparency principles was analyzed with Spearman non-parametric correlation tests due to the small number of samples using the IBM SPSS Statistics for Windows, Version 28.0. (Armonk, NY: IBM Corp) (6). The average, minimum, maximum, mode, median, standard deviation, and percentiles were calculated.

Results

During the study period between August and September 2023, we could access all the current websites of the eight Turkish ORL-HNS journals in the study. The evaluation of those journals according to the 16 transparency criteria is shown in Tables 2 and 3. The journals complied with an average of 11.25 criteria out of 16. It was seen that there was compatibility with a minimum of five and a maximum of 14 criteria (Table 3).

According to the calculated 2022 Impact factor, the first three journals out of eight were The Journal of International Advanced Otology, the Turkish Archives of Otorhinolaryngology, and the European Journal of Rhinology and Allergy. These journals complied with 13, 13 and 12 criteria, respectively, out of the 16 transparency criteria. The last three journals in the ranking are the Praxis of Otorhinolaryngology, the KBB Forum Electronic Journal of Otolaryngology-Head and Neck Surgery, and The Turkish Journal of Ear, Nose and Throat. These journals were evaluated as complying with 8, 5 and 12 criteria, respectively. Due to insufficient sample size, the relationship between these two concepts could not be analyzed properly (Table 4).

Discussion

The principles of transparency and best practice in scholarly publishing, developed and published by COPE, DOAJ, OASPA and WAME, is an international publishing resource which defines a crucial standard-namely, the transparency principles-in scholarly journal publishing (1). It is critical for journals to organize their websites according to those 16 transparency principles for being indexed by major international indices such as DOAJ. Furthermore, the editors, their teams and journal managers are not only members of a journal, but also readers of their and many other journals. This study can enlighten them to improve their journals according to the up-to-date principles. Also, by learning these principles, readers who are also potential researchers can evaluate journals on their own and choose them wisely to publish their research which requires great effort.

In the presented study, we evaluated the adherence of Turkish ORL-HNS journals to the principles of transparency and best practice in scholarly publishing. The main objective of our study is to identify the aspects to be improved and to aid these journals in achieving better compliance with international standards.

Our findings revealed that while all journals comply with the principles for website, publication ethics and related publication policies, access, ownership, and management, some fall short of the name, archive, copyright, license, peer review, editorial team, and contact information criteria. The reason for falling short of the principle of name could be because the name of the journal can be confused with those of other journals, potentially misleading authors and readers about the scope and origin of the journal.

In terms of archive criteria, the main reason for the incompatibility is that the lack of explanation of system that preserves the contents of the journal for a long time digitally on the website. Failure of a separate statement of the copyright conditions of the website and printed publications caused the journals to be found incompatible with the copyright principle. Absence of a statement about the license terms of the published articles (including HTML and PDF formats) and not sharing the license policies of the third parties where the publications are stored are among the reasons for incompatibility with this criterion. Some journals were not found to comply with this criterion of peer review because they did not clarify on their website the particular conditions of their peer review process, announcement of the result whether by wet signature or anonymously and whether referees recommended by the author were in charge during the evaluation.

Absence of up-to-date information about the editorial team and the deficiencies in the contact information of the editorial office are the main items that can be considered disqualifying for the editorial team and contact information criteria.

Most of evaluated journals (6/8) did not share the contact information of the advisory council, how the income-expense tally is kept, what the pricing and distribution policy is if the printed journal is published, what type of advertisements are accepted under what conditions and by whom, and this led to incompatibility with the advisory council, other income, direct marketing, and advertising principles (Table 2).

According to the 2022 Google Scholar Impact Factor calculated from the verified citations in Google Scholar, the first three among the eight journals are The Journal of International Advanced Otology, the Turkish Archives of Otorhinolaryngology and the European Journal of Rhinology and Allergy. The last three journals in the ranking are the Praxis of Otorhinolaryngology, the KBB Forum Electronic Journal of Otolaryngology-Head and Neck Surgery, and the Turkish Journal of Ear, Nose and Throat, starting from the last (Table 4). While the first three journals comply with 12 to 13 principles, the last three journals comply with only 5 to 12 principles, suggesting that compliance with such international principles may be important for journals to be cited and included in important international indices such as Web of Science (WoS) and Scopus, which often seek citation analyses during the journal evaluation process. The first three journals with high compatibility rates to the transparency principles are indexed either in WoS or Scopus. However, only one of the last three journals is indexed in one of the indices; The Turkish Journal of Ear, Nose and Throat is indexed in Scopus and has the highest compliance rate among them. As a result of the analysis, the relationship of the calculated 2022 impact factor with the criteria covering the journal content was determined. This may suggest that qualitative publications can be effectively included in international indexes and get citations. However, the relationship between these two concepts could not be adequately analyzed due to insufficient sample size, which constitutes the main limitation of the presented study. Furthermore, the original impact factors are calculated from WoS citations in WoS journals and very few Turkish ORL-HNS journals are indexed in WoS, so a direct hypothesis on the relationship between the compliance and the impact factor could not be built.

Conclusion

Based on the findings of the presented study, in which we analyzed Turkish ORL-HNS journals in terms of compliance with one of the most important international scholarly publishing standards, the principles of transparency and best practice in scholarly publishing, we suggest the Turkish ORL-HNS research community to study the missing and inapplicable criteria, and carry out the necessary studies, especially on journal websites, to improve compliance with the referred principles.

All ORL-HNS scientific journals in Türkiye will benefit from considering the fulfilment of the 16 transparency criteria to improve their publishing quality to gain presence on international platforms and receive citations. One of the issues that should be emphasized is the need to regulate primarily the sharing of details of the advisory bodies, other expenses, advertising, and other marketing content, with which fulfillment is complex. In addition, considering the criteria of the three journals with the highest impact factors among the evaluated journals, one of the important messages of this study is that the relationship between the impact factor and compliance with the transparency criteria should be investigated with a larger sample.