By Agnese Benzonelli, Associate Editor for Archaeometallurgy
Earlier this year, a fascinating and illuminating discussion was shared among the Arch-met mailing list in response to the question: “why does Nature get archaeometallurgy so wrong?”.
While the subject and examples given were, at least initially, focused on archaeometallurgy, the topic is of wider interest to archaeological science and archaeology more broadly. There were over fifty responses from scholars from all over the world in just a couple of days. I will summarise the most relevant points. The entire discussion can be found at the following link: https://www.jiscmail.ac.uk/cgi-bin/wa-jisc.exe?A1=ind2102&L=ARCH-METALS
Recently, an increase in poor-quality papers being published in high-impact journals has resulted in a widespread malaise in the archaeometallurgy community. Nature, Science, and their online-only open-access spinoffs – Scientific Reports and Science Advances, publish research that includes methodological gaps, a lack of essential scientific analysis, and incorrect interpretation of datasets, all of which lead to erroneous conclusions. The question is: why?
The underlying issue is still the failure of most of the scientific community to regard archaeometallurgy as a scientific discipline in its own right. This leads to incompetence among the board of managing editors, which results in an inadequate selection of competent reviewers.
One of the posts revealed an insightful analysis of the editorial boards of the aforementioned journals, where it was evident that the core problem is structural. When the board is divided into categories of competences, such as in Nature and in Scientific Reports, it is appalling to see that there are none in the humanities and their related sciences. The nearest category to Social Sciences is Psychiatry. Otherwise, in the more scientific disciplines, there is Earth and Environmental Sciences, however, there is no one in the discipline with expertise in archaeometry among almost one hundred people. As such, archaeometallurgy and archaeometry papers are often handled by biologists.
These editors then fail to find competent reviewers and, after the quick analysis of the editorial boards, it is clear why this happens. Given the overlap between science and humanities, which is typical of archaeometallurgy, non-specialised reviewers from one or the other side fail to provide a complete report, lacking either the interpretation of scientific data, or the scientific method that supports that interpretation. When competent reviewers are selected, they report to be extremely frustrated by finding their reviews, objections, corrections, comments and recommendations ignored, and the manuscripts published as they are, or with only minimal changes. This lack of consideration of the reviewer’ report, even when it offers detailed reasons why, seems without reason, and undermines the core concept of “peer review”.
It is clear that these journals are not designed to publish on archaeometry and archaeological science, nor considering, evaluating and giving the right credit to important research. Moreover, these journals tend to favour storytelling and sensationalism, even at the cost of scientific accuracy and the quality of the published word. Therefore, the few archaeometry papers published in these journals are those with eye-catching words, such as “first” or “earliest”, which compels authors to make highly-exaggerated claims to bolster the significance of their findings.
So, why do we all try at some point to submit our research to these journals? Part of the reason is the relentless pressure felt by universities to publish in the big journals with high Impact Factors that guarantee high citation numbers. While many archaeologists tend to avoid these journals, favouring a more targeted journal, and audience, the credit gained from both academics and their institutions, in “hitting the jackpot” by being published in them is substantial. For example, a researcher claimed that the citation rate of one of his papers published in Science resulted in higher than the rest of his portfolio combined, which shows the impact of the journal citation score. This is true especially for people without an established reputation, who still consider it worthwhile to try their luck, regardless of the extremely low acceptance rate. It was reported that in the UK, the Archaeology REF panel specifically ignored citation statistics and journal status in their most recent evaluation of research outputs.
The discussion then moved on to the second significant theme of the thread: the review process. There are too many manuscripts, and a limited (and exhausted) pool of willing and - more importantly - competent reviewers. More and more manuscripts are submitted to journals, which in turn are expanding to cover the demand. Considering focused journals, JAS, passed from 712 pages in 1990 to 5,128 pages in 2014, before splitting into two journals which together, in 2018, published 7,536 pages. Archaeometry went from four to twelve issues per year, and new journals (Archaeological and Anthropological Sciences, Star, etc.) emerged over just a couple of decades.
Around a third of the papers received are sent for review, which involves two to three reviewers for each paper. Half of the papers sent to review are published in the journal. Essentially, for each published paper, the author(s) should pay back into the system four to six peer reviews.
So, why are so many manuscripts produced? One reason is that, since the 1980s, universities and research institutes perform annual performance reviews, which put a lot of pressure on authors, regardless whether they’re established academics or new PhD students. If you want to keep your job, get promoted or secure an interview, you need to be published.
Since 2000 there has been a revolution in analytical techniques (geophysical, chemical, isotopic, genetic, statistical, graphical, etc.), which has resulted in much more data, and therefore papers, as well as the growth and recognition of archaeological science. EU funding programmes, including FP6, FP7, Horizon 2020 and Horizon Europe, generated tens of millions for archaeological science. This increased the number of projects, courses and researchers. The rapid growth is an indication of an active, healthy field, which is still expanding and offering many research opportunities. It is not without fault, however, including an increase in flawed papers being published, which has serious consequences.
First, it results in an increase in emerging scholars over experienced ones. While the former are the pressured authors who generate new papers, the latter are the competent reviewers, with stable jobs, who review the research thanks to their own grants. This influx of papers puts an incredible pressure on the reviewers, already overwhelmed by their own research, teaching and paper-writing, as well as often writing reviews for funding agencies. Simply checking if the manuscript is worth accepting at the initial stage, which often leads to declining the invitation, is very time-consuming. A review is even more time-consuming, especially for those who work freelance, and do not have salaried academic positions.
Comments
Post a Comment