Reviewing articles as a way of professional evaluation of scientific texts: organizational and ethical aspects

ARTICLE INFO Svitlana Fiialka, Olga Trishchuk and Nadija Figol (2020). Reviewing articles as a way of professional evaluation of scientific texts: organizational and ethical aspects. Knowledge and Performance Management, 4(1), 26-36. doi:10.21511/kpm.04(1).2020.03 DOI http://dx.doi.org/10.21511/kpm.04(1).2020.03 RELEASED ON Monday, 28 December 2020 RECEIVED ON Tuesday, 20 October 2020 ACCEPTED ON Wednesday, 23 December 2020


INTRODUCTION
Scientific journals play the most significant role in the production and dissemination of new knowledge. The main element of the journal is a scientific article created by the author (researcher), but the decision on its quality and the possibility of inclusion to the journal (publication) is made by the editor. In the practice of scientific communication, the editor does not make decisions on the article alone, but relies on the reviewers' recommendations. However, these recommendations can be contradictory: even if four reviewers are involved in the review of the manuscript, four different solutions are possible: publish, publish with minor modifications, publish if revised, and reject (Donmoyer, 1996). It is complicated by the fact that the editors of scientific journals constantly have to find answers to the questions "Where to get reviewers?", "By what criteria should they be selected?", "Can the author suggest a reviewer for his article?", "How can an editor persuade the reviewers to evaluate the manuscript and encourage them to continue collaborating?", "How to prevent conflicts of interest?", "How to verify the impartiality?". Increasingly, scholarly articles appear at the boundaries of disciplines, so another problem arises: "Who can give expert judgment to such research?" (Fox, 2017).
Many journals, especially in developing countries, do not carry out the peer review process properly. Editors and/or reviewers apply lower

LITERATURE REVIEW
Reviewing has served as a tool for quality control of scientific papers since the second half of the 1600s (Zuckerman, 1971), when scientific societies appeared. One of the first steps towards integrity in science was the evaluation of the quality of papers by the members of a prestigious organization, such as the Royal Society. Until the middle of XVIII century such a phenomenon has become more formalized. The President of the Royal Society, Earl of Macclesfield, reminded scholars that external confirmation was the only way to uphold high standards of science, and set up a peer review committee. However, even then, the system gave the first failures: external reviewers of the Journal of the Royal Society rejected E. Jenner's paper on vaccination, that was later published and radically changed the course of medicine and the whole history of mankind (Booth, 1982).
An important impetus for the scientific understanding of the review process was the Chicago International Congress (1990), which presented research on peer review, followed by an article in the Journal of the American Medical Association (Pubmed, 1997). By the way, much of the literature that studies peer review is specifically about the medicine (Das, 2016), perhaps because spreading false facts in this field can be extremely dangerous (Curtin et al., 2017).
Over time, the nature of peer review has changed slightly (Walker, & Rocha da Silva, 2015). It was still perceived as a significant contribution to sci-ence: on the one hand, to help the editorial board to accept or reject the manuscript, and on the other, to help the authors improve the submitted manuscripts (van Rooyen et al., 1999;Tennant, 2018).
However, the organizational aspect of peer review has dramatically transformed. In particular, there are numerous tools for finding reviewers: Reviewer Locator tool, Reviewer Discovery from Aries Systems and Reviewer Finder from Elsevier's EVISE system. Through Publons profiles, researchers are able to track others researchers and present their own peer review experience. In Publons, scientists can list all the journals in which they were reviewers. This resource maintains anonymity, since only the journal name and year are indicated in the profile, so it is not possible to identify who reviewed the article. Also, in the Publons settings, a scientist can share an article or review to discuss it publicly.
In addition to these changes, the number of manuscripts requiring peer review has increased exponentially, notably due to increased pressure on scientists and a simultaneous increase in the number of conferences and scientific journals. However, only 20% of researchers are responsible for more than 90% of reviews (Kovanis et al., 2016), and 40% of reviewers have never undergone specific peer review training (Vesper, 2018).
As the number of peer-reviewed journals has grown by 3.5% annually over the last 200 years, it is becoming increasingly difficult for editors to find a reviewer (Villar, 2019). According to the ed-itor of the European Journal of Hospital Pharmacy P. Wiffen, in 2013-2014 the editorial staff appealed to seven or more potential reviewers to obtain consent for the manuscript evaluation. In 2017-2018 there were already 15 potential reviewers. Therefore, a significant number of articles have ceased to be reviewed at all: if the editorial team considers the article inappropriate for publication due to the inconsistency with the journal's aim and scope, lack of novelty, conclusions based on limited quantitative data, such article is rejected without review (Wiffen, 2018). This is usually what reputable journals like "Science" and "Nature" do. In the same way, editors accept, without peer review, manuscripts that are considered to be of exceptional quality, and also remove from the list of potential reviewers persons who are unreliable or dishonest (D'Andrea, & O'Dwyer, 2017).
M. Willis analyzed the responses to the reviewers' invitations and concluded that the main reason for the reviewers' refusal was lack of time (Willis, 2016). M. Breuning, J. Backstrom, J. Brennon, B. Gross, and M. Widmeier interviewed reviewers for reasons why they refused to review the manuscripts. Among the reasons, the respondents mentioned: overload of main work; too many peer review proposals, incompetence on the subject matter of the article being reviewed; vacation; administrative responsibilities; conflict of interests; personal problems (such as illness). Female reviewers have more often stated that they are too busy, have too many peer review proposals, or are on vacation. Men are more likely to report lack of experience or administrative responsibilities (Breuning et al., 2015). Therefore, with the increase in publication activity, reviewers are increasingly selected not by competence but by punctuality (Kurdi, 2015): "good" reviewers may be overloaded by requests, and in fact become "unreliable" reviewers (Greenstein & Biglieri, 2009). It is therefore essential that authors submit articles only after taking into account comments from colleagues and making sure that they can do nothing to improve their work (Kohli, 2011).
The reviewing may seem one of the least gratifying tasks. Reviews are usually not paid, published or cited. Even authors, in most cases, never know who reviewed their article. In addition, reviewers lose as they prepare reviews rather than write their own research papers (García et al., 2017). The only reward may be the personal satisfaction of the reviewer for helping another member of the scientific community improve their work or save time for readers who might spend time reading a poor-quality article. However, if the article is good, then the reviewer becomes one of the first to see new research that helps him formulate his own ideas and improve his own research skills and academic writing.
Some publishers still offer rewards to reviewers. For example, reviewers of Elsevier's journals may receive discounts on Elsevier's services (http:// www.reviewerrecognition.elsevier.com).
Some journals (such as the University of California Press) use the authors' publication fees to pay for the reviewers' work (Willis, 2016).
When it comes to reviewing policy, it can either be "blind" when trying to hide the identity of reviewers, authors, or both, or non-anonymous when the reviewer and author know each other's identities (Haffar et al., 2019).
Open peer review is the least used approach: more than 50% of researchers rarely accept invitations to be peer reviewers if their names are open for authors. Meanwhile, proponents of open peer review suggest that it encourages more honest and thoughtful reviews (Jubb, 2016). However, an open system is associated with a higher rate of refusal to re-review and increase the time of writing new reviews (Haffar et al., 2019).
To facilitate the process of selecting reviewers for editors, many journals ask authors to nominate reviewers when submitting a manuscript Moore et al., 2011). The reviewers offered by the author accept review requests more often than reviewers selected by the editor and are more often from the author's country (Liang, 2018). Such reviewers typically make less critical reviews than those appointed by editors (Haffar et al., 2019), and are more likely to recommend articles for publication (Schroter et  Some studies show that the source of reviewers' search did not affect the quality of the review (Rivara et al., 2007;Schroter et al., 2006;Wager et al., 2006). However, in our opinion, to ask the authors for a list of potential reviewers would be appropriate only to create a journal database for further reviewing of related materials.
Blind reviewing is seen as a way of eliminating or reducing the bias associated with the author and the institution he/she represents (van Rooyen et al., 1998). It can be single-blind, double blind, triple blind or quadruple blind. The main advantage of blind peer review is that reviewers are able to express their opinions freely without fear of conflict (O'Connor et al., 2017). So, blind reviewing helps to protect the manuscript from biased evaluation, but does not save authors from the unethical reviewers' behavior (Rowland, 2002;Chung et al., 2015) like "borrowing" ideas from a peer-reviewed manuscript (Resnik et al., 2008;Mack, 2015). In addition, it is possible that the reviewer can guess who has written the article (Chung et al., 2015;Rowland, 2002). The identity of famous authors is often recognized by their style or theoretical orientation. In some areas, the "invisible colleague" may be known to the reviewer, since scientists are aware of the research conducted by their colleagues (Crane, 1967). Moreover, the author himself can unambiguously reveal his identity in the manuscript (Krinsky, 1999).
In the case of single-blind peer review, the author's identity is known to the reviewer and the reviewer remains "blind" (Chung et al., 2015). Some scholars have suggested that a reviewer's knowledge of whose work he or she reviews enables him/her to understand it better (Jubb, 2016). However, it has been confirmed that, with one-sided blind peer review, it is much more common for reviewers to accept articles written by famous authors, authors from prestigious universities, and innovative companies (Tomkins et al., 2017).
With double blind peer review, both reviewers and authors are unknown to each other during the manuscript evaluation process (Chung et al., 2015). It has been proven that double-blind peer review is more common in the social sciences and humanities (Jubb, 2016).
In the triple blind review model, apart from hiding the reviewer's identity from the author and vice versa, the authors are unknown to the editors when submitting the publication for review. The editorial board manages the manuscript without knowing the names and institutions of the authors. All communication is possible through the journal's website without disclosing any author-related identity (da Silva & Dobránszki, 2015).
In the case of quadruple blind peer review, a professional association (for example, the ASA Publications Committee) appoints an anonymous editor and his or her identity remains confidential throughout the editorial process (da Silva et al., 2018).
Despite a large number of peer review practices, biases are possible at all stages of the editorial process. Before reviewing, these biases are related to the pre-evaluation of the articles by the editor and making decisions based on the author's country, institution or authority. Editors' selection of reviewers can also be biased. Reviewing can include the dogma of reviewers, prejudices related to the differences between the author's paradigm and the views of the reviewer. This may delay the publication or writing of the review that diminishes or disproves the author's results. Therefore, the review process is far from perfection, and all its models have advantages and disadvantages (Haffar et al., 2019).
However, speaking of biases, one cannot ignore the so-called "good biases" that are based on the importance and originality of the articles reviewed. Hypothetically more experienced researchers do better research, write better manuscripts and, accordingly, are better reviewers.
In 2018, Publons interviewed 11,800 reviewers in the natural sciences, engineering and medicine. The study found disproportionate representation of reviewers from different regions: US scientists produce 32.9% of all reviews and 25.4% of all scientific articles, while 8.8% of reviews come from China, while Chinese scientists produce 13.8% of scientific articles. This disproportion harms re-searchers and science: fewer peer review invitations mean fewer opportunities to get up-to-date with scientific advances, find out what editors expect, and engage with editors. Interestingly, 38% of all reviews were written by reviewers from the same region as journal editors (Publons, 2018).
Prejudice can also be caused by a conflict of interest (a set of circumstances in which primary professional interest is influenced by individual secondary interests) (Thompson, 1993): researchers affiliated with Company X are asked to review studies showing that a drug manufactured by a company outperforms products of competitors; a researcher from company Y, which produces competitive products, is asked to review the same study; a biased editor manages the review process; researchers are asked to review the article submitted by a person with whom he or she has a professional relationship, such as a colleague at his institute, an employee, or a former student or advisor; the journal editor himself submits his own article for review (Resnik & Elmore, 2018).
Another problem is the bias of confirmation: on the one hand, reviewers like when their own opinions are confirmed, on the other hand, they are more enthusiastic about finding flaws in articles that contradict with their findings. That is, some reviewers only mention the weaknesses of the article (and only by chance, if at all, mention the strengths); others try not to criticize at all. Ideally, a reviewer should avoid both of these extremes and try to create a balanced picture (Bannister & Janssen, 2018).
Turner and Hanel identify two types of reviewers: demanding and selfish. The demanding reviewers constantly raise the requirements for the manuscripts they approve, while the selfish tend to accept lower quality manuscripts. Demanding reviewers tend to write negative reviews because the standards of such reviewers are too high. In contrast, the selfish reviewers reject any manuscript that exceeds the quality of their own articles because quality manuscripts, in their view, can compete with their own publications (Thurner & Hanel, 2011). In addition, it has been stated that selfish reviewers who perceive the author as their competitor may intentionally impair the quality of the peer-reviewed publication (D'Andrea & O'Dwyer, 2017).
In most cases, two demanding reviewers do not differ in their decisions, and demanding and selfish reviewers can, so differences in reviews can indicate the selfishness of at least one of the reviewers (D'Andrea & O'Dwyer, 2017). Knobloch-Westerwick, Glynn and Huge have explored the "Matilda effect" when works that are first identified by male authors are rated higher than those where the first female authors, especially in scientific fields where men dominate (Knobloch-Westerwick et al., 2013). Merton analyzed the Matthew effect when the works of well-known researchers in the field usually received higher marks than articles of similar level, but less well-known authors (Merton, 1968). Blank found a correlation between the ratings of articles and the reputation of the institutions they represent (Blank, 1991).
The purpose of the paper is to summarize Ukrainian experience in the organizational and ethical aspects, problems and perspectives of peer-reviewing scientific articles on the basis of analysis of literature and survey of the scientists.

DATA AND METHODS
To address the diversity of scholars' views on peer review, the paper provides a narrative review of the literature. The articles were searched in Web of Science Core Collection (43 articles), Scopus (27 articles), and additionally in Google Scholar (65 articles) by the keywords "reviewing", "reviewer's duties", "types of reviewing" and more. In addition, articles from the references were selected for analysis. Articles from journals which do not have a review procedure, whose editorial staff does not correspond to the subject matter of the publication, which does not cite articles from journals belonging to leading international science databases, etc., were removed from the search (17 articles from Google Scholar).
So, from October 2019 to January 2020 on the pages of Facebook groups "Ukrainian Scientific Journals" (Ukrainian Scientific Journals, 2020), "Ukrainian Scientists Worldwide" (Ukrainian Scientists Worldwide, 2020), "Pseudoscience News in Ukraine" (Pseudoscience News in Ukraine, 2020), "Scientific Conferences and Publications" (Scientific Conferences and Publications, 2020), "Education. Science. Technology. Innovations" (Education. Science. Technology. Innovations, 2020), "Academic Virtue and Plagiarism" (Academic Virtue and Plagiarism, 2020), "Higher School and Science of Ukraine: Disintegration or Blossoming?" (Higher School and Science of Ukraine: Disintegration or Blossoming, 2020) a survey of Ukrainian scientists on peer review was conducted. The survey was tested on the Printing and Publishing Institute of the National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute before launch (7 professors tested the survey). The results of the survey were analyzed during April-June 2020.
In total, 390 researchers from different fields of science participated in the survey. The questionnaire contained 13 questions, including 5 open-ended ones. Responding to 8 questions, researchers could choose answers, of which on 5 questions respondents could choose more than one answer.

RESULTS
Summarizing the theoretical findings of the study, we conducted a survey of Ukrainian reviewers on their attitude to reviewing, motivation, reviewing models they prefer, cases of conflict of interest, difficulties in engaging with authors and editors.
It was found that 174 reviewers (39.5%) have no experience of reviewing foreign scientific publications. Others reviewed for foreign journals, primarily in the fields of physics, chemistry, medicine, mathematics and information technology: When asked about the peer review model that the reviewers worked on, the respondents provided the following answers (respondents had multiple choice options): double-blind peer review, when both reviewers and authors were unknown to each other during the manuscript evaluation process, -198 responses; single-blind peer review, where authors' surnames are available in the review material, -180 responses; open peer review (author knows who is the reviewer and reviewer knows who is the author) -90 responses; triple blind review, when both reviewers and authors are unknown to one another, and authors are unknown to editors of the journal -30 responses. 18 respondents just confirmed the review, prepared in advance by the author of the article (Figure 1). However, expressing their attitude towards peer review, respondents gave slightly different estimates: only 34 respondents (8.7%) favored open peer review, 168 (43.1%) single-blind, 147 (37.7%) double blind, and triple blind -36 (9.2%). In addition, 5 respondents (1.3%) indicated that they would confirm a review prepared by the author himself ( Figure 2). 295 respondents (75.6%) indicated that they had not disclosed conflicts of interest during the review process: they were asked to sign the review based on friendly relations; the boss requested a positive review of the article of a certain person; the editor requested a positive review and more.
Respondents' answers on the motivation of the reviewers were follows (respondents had multiple choice): self-development, opportunity to get new knowledge -356; the struggle for the "purity" of science -204; it is prestigious -96; ability to improve own academic writing -94; one more "point of activity" for reporting -80 ( Figure 3).
Among the reasons why reviewers refuse to review articles, on the first place is the inconsistency of the article in the research profile of the reviewer, and hence the lack of knowledge for proper review (294 responses). Other reasons are the lack of time (91) and poor quality of publication when the reviewer does not see the point in wasting time (31).
Among the reasons for the rejection of the articles, the respondents indicated: low scientific value, lack of novelty (278); plagiarism (104); falsification (91); inconsistency with the aim and scope of the journal (84); insufficient involvement of scientific literature (68); structure of the article (60); language and style (44). It is suspicious that 32 respondents never reject articles regardless of their quality (Figure 4).
Responding to fact-checking questions, 132 reviewers (33.8%) indicated that they necessarily check bibliographic references and facts in scientific articles; 163 (41.8%) do so only when in doubt; 19 (4.9%) -only when they are most interested in revealing the truth; 28 check bibliograph- For reporting Improving academic writing skills It is prestigious The struggle for the "purity" of science Self-development ic references (7.2%). 48 respondents (12.3%) never checked the facts and references ( Figure 5).
246 respondents (63.1%) had problems interacting with authors and editors. Concerning such difficulties, some authors have provided their own comments, which can be summarized as follows: the author does not want to listen to the recommendations and actively proves that only his or her version is correct; lack of time; ignor-ing comments; pressure in the review process to provide a positive review; authors' claims of "rigor" of the review; editorial requirements to compare articles with others in their databases; the authors' misunderstanding of the benefits of submitting articles through an electronic journal system, such as OJS; the editors publish poor articles with negative review; plagiarism; the need to work with overloads; automation of sending reviews; opponents of certain scientific areas (such as GMOs or vaccinations) may manipulate comments in reviews; the policy of editors who suggest to review articles on topics in which the reviewer is not an expert; editors do not review the articles for compliance with the requirements of their journals and immediately forward the article to the reviewers. The conclusion depends only on the quality of the article

CONCLUSION
The results of the analysis of the scientific literature indicate that the necessary conditions for quality review are as follows: the reviewer reveals any potential conflict of interest; refuses to review if it considers itself incompetent in the topic of the article; draws attention to the originality and scientific value of the article; identifies both the strengths and weaknesses of the manuscript; provides constructive suggestions on how to improve the article; demonstrates a positive collegial tone of communication rather than a negative "restaurant critic".
Reviewing can serve a variety of purposes: to test the reliability of the study (whether the study was conducted due to standards and whether the findings are reliable); to evaluate the originality, relevance of the article for the audience; to evaluate the fit between the article and the mission and the goals of the journal; to help the authors. Reviewers are expected to evaluate the article impartially, and not use research results until they are published.
Because reviews are usually only seen by editors and authors, this can lead to the following issues: reviewers can be rude or biased; authors may not adequately respond to grounded criticism; editors may ignore the decision of the author or reviewer, and journals may charge for publishing articles without proper peer review. These issues could be overcome by publishing reviews in open access. It would allow scholars to learn how to write articles and also to collect data such as reviewers' biases or the effectiveness of reviewers, and readers would receive further confirmation of the quality of the scientific results outlined in the article, as well as verify the transparency of the editorial decisions.
Reviewing policies can influence the behavior and strategies of authors' publications, making journals more or less attractive, depending on the type of review and the level of transparency of the editorial process. Further research may relate to new peer review practices, such as publicly available review; cascading review; use of independent peer review platforms.