EST symposium on publishing in TS
Hogeschool Gent, 23 September 2005
PEER REVIEWING
Aline Remael
Chief Editor
Linguistica Antverpiensia NS
- The Players
| Working definition
Reviewing a manuscript submitted for publication means assessing its strong and weak points with a view to evaluating whether or not it is publishable, as it is, or after some rewriting has been done.
|
Table 1
Who are the persons involved in this process and what is at stake for them? Peer reviewing is “not only an essential part of the editorial process, but also a learning experience for contributors,” said the call for papers for the Ghent EST symposium. In other words, peer reviewing concerns the editors of a journal, and it has an impact on its contributors, but in the decision-making process the referees carry part of the responsibility, and the readers are among the beneficiaries.
Let us consider the role and interests of the different players in greater detail.
The editors:
Peer reviewing is important for the editors of a journal because ideally it is a way for them to help guarantee they will be publishing quality, state-of-the-art contributions, written by and for specialists in the field. Anonymous peer reviewing is therefore a requirement for the journal’s credibility and a way to assure itself a reading public.
The contributors:
Peer reviewing is important for the contributors because it ought to guarantee that their papers appear in a journal of quality, among articles by peers working in the same or related fields.
Peer reviewing is also a built-in control mechanism for the contributors as much as for the editors: once an article has been accepted, they know, again ideally, that they are on the right track . Referees’ comments should therefore first and foremost be an indication for both the editors and the authors as to the pertinence and academic level of the writing.
The referees:
Referees bring their academic expertise to bear on the manuscript of one of their colleagues, taking part in the decision whether or not to publish. They therefore assume a great responsibility. That there must always be more than one referee is self-evident, three seems like a good number to guarantee a balanced judgement and to avoid stalemates.
The readers:
The most obvious readers of academic TS journals are specialists and students in TS, or an area in TS, but they can also be academics from related fields of study, translators, or simply people with an interest in TS. Still, the core reading public will probably consist of TS scholars and students. They constitute the academic community that benefits most directly from quality journals. However, academic institutions are also an interested party. They can get part of the credit if their personnel is involved, and they can use high-level publications to keep curricula up to scratch. This last issue works both ways: editors must be aware of the direction or directions in which the professional field is heading when determining the course of their journal, including reviewing policy.
In what follows, the editors and the referees will be the focus of the attention, but the journal’s readers should always be at the back of our minds.
- Issues determining peer reviewing
The table below offers a tentative list of issues determining peer reviewing. Some of them seem self-evident, but as soon as one tries to fill in these rather abstract or general concepts, problems arise. The very notion of ‘quality’ can give rise to almost endless discussions. This means I will not be supplying all the answers and that the ones I do give are hypothetical, they serve as a basis for discussion.
|
Table 2
2.1 What is a quality article?
In order to decide what the task of a referee is, editors must have a good idea of the type of articles they want. The following requirements seem essential.
|
Table 3
- One that deals with a relevant TS topic: confirming & renewing.
There is no need to reinvent the wheel. Placing one’s own contribution in the relevant literature, reflected in the bibliography, is certainly desirable, but a balance must be kept between showing one knows the literature and referring to what is important for the article at hand. On the one hand, a good academic article is ‘context confirming’ in this sense: it defines its niche within existing and ongoing research. On the other hand, it is also ‘renewing’: it contributes new insights, a new methodology, new examples, …It does not necessarily have to do all of these things. An article does have to be well-documented both with regard to its foundation in existing research and its innovative features.
- One that starts from a well-formulated hypothesis or research question.
A hypothesis and clearly formulated research question give direction to an article. Without such a starting point an article risks going around in circles, getting bogged down in side issues or ending in the middle of nowhere without reaching any form of conclusion.
- One that tackles its research topic with a clear and appropriate methodology
Not only the topic and research question but also the method or approach authors have used for their research must be stated explicitly. This will help readers to situate the research within the broader field of TS and give them ‘tools’ with which to evaluate the soundness of the results. The methodology must be applied consistently; i.e. the article should not expound the methodology first and then forget about it. Findings must be documented with more than sporadic examples (statistically relevant data, references to other verifiable pieces of research, clear statements of boundaries, …)
- One that aims at clarity, is well-structured
Starting from a well-formulated research question the article proceeds step by step, building on findings and providing verifiable data. It does not digress, and it does not resort to footnotes to include side issues that are only tangentially relevant. An excessive use of footnotes or endnotes is not commendable.
- One that respects relevant academic writing traditions and the style sheet of
the journal
Academic writing is characterized by a certain degree of formality, the use of specialized terminology (and definitions), subdivisions into sections, a judicious use of quotes and footnotes or endnotes, references that are complete, the use of figures and tables where so required. Besides, journals provide more or less detailed style sheets that must be respected.
- One that is well-written and keeps its reading public in mind.
There is nothing wrong with academic writing that is accessible. Still, depending on the reading public of a particular journal, the writing style may have to be adapted, e.g., in its degree of formality. Some journals are meant for a broader public, some for a very specialized subgroup.
2.2 What are referees expected to comment on?
|
Table 4
All the issues mentioned in the previous table (n°3) are certainly worth considering: a relevant TS topic, a well-formulated hypothesis or research question, documentation, clear and appropriate methodology, clarity and logic in structure, academic writing, compliance with the style sheet. But also the additional ones given in table 4 above are relevant.
However, referees may not be expected to comment on all of these items. The final decision is made by the editors or possibly the guest editor(s) of a special issue in consultation with the editor(s). The editors of a TS journal are also specialists in the field, possibly to varying degrees, that is depending on the degree of specialization aimed at in the case of a theme issue, for instance. In some cases the tasks will therefore have to be divided.
In other words, what referees have to comment on will vary from journal to journal or even from issue to issue. These variables make it impossible to give a clear-cut blanket answer to the question raised in 2.2.
The problem is also related to the multidisciplinary nature of translation, or the ‘interdisciplinary’ nature of TS and the different make-up of the many existing TS journals, as well as their reading public. In other words, before we can turn to the question of what peer referees are expected to comment on, we must consider a few of the other issues from table 2.
2.3 Are there any specific challenges for TS journals and do all TS journals have the same requirements?
Translation Studies is an interdiscipline. Or at least, it has strong interdisciplinary features. This not only means that TS expertise can encompass a variety of research areas, it also means that in many cases the way forward in TS research is that of research projects to which various specialists bring their own expertise.
The content of articles may therefore have to be evaluated by specialists working in a specific subfield, sometimes by experts from related disciplines. Evaluating the writing style and structure may then fall to the editors. Indeed, the supposed target readers may be a determining factor in the evaluation of style or even the degree of specialization required, and this is a judgement the editors may be better off making (cf. instructions, below).
All sciences have their sub-fields, and in that respect TS is no different from, say, medicine, which obviously has areas of specialization. If a TS journal aims to cover the whole spectrum of translation research it must have a pool of specialists to draw on. It is self-evident that one and the same scholar will not be able to evaluate manuscripts related to technical
translation, audiovisual translation, literary translation, theatrical translation, localisation, etc. Besides, translation studies is characterised by a number of different theoretical approaches
and methodologies as well, e.g. the application of methods from corpus linguistics, combined with insights from Descriptive TS to give but one example.
I would therefore say that this interdisciplinary nature is indeed a factor each TS journal needs to take into account when deciding what its referees will be expected to evaluate, but also which referee one wants, and – possibly – how many. Some TS journals may specialize on the basis of subject areas, others on the basis of theoretical approaches to translation, still others may wish to reflect the state of the art, giving a voice to innovative trends in the field and/or to young scholars, even crossing over into related fields. Then again, a so-called ‘general’ TS journal may publish special issues, and work with guest editors (which Linguistica Antverpiensia NS does), editors who are themselves specialists in the sub-field to which a particular volume or issue is devoted.
Finally, English is the dominant publishing language in academia, but some TS journals are published in other languages and some publish multilingual issues (for instance, again, Linguistica Antverpiensia NS). Personally, I feel it is up to the authors to ensure they fully master the language they write in, and if they do not, to make sure their article is read by a native speaker before submission. Still, since this is not always what happens in reality, multilingual journals will sometimes have to ensure that either one of their editors or one of their referees, or possibly both can also evaluate the language in which the paper is written.
The way the editorial board and possibly the advisory board of a journal are constituted may to some extent determine the journal’s reviewing policy:
– the board may have been composed so as to cover as many subfields as possible,
– the board may have been composed on the basis of publishing experience,
– the board may have been composed on the basis of all of this, plus their professional contacts, etc.
Referees may be members of the board, members of the advisory board, if there is one, or external experts constituting a kind of ‘reader database’ that the editorial board or advisory board can draw on. When a particular issue on a specialized topic is edited by one or more guest editors, they too may have a pool of contacts to draw on.
The two extremes may be on the one hand, a closed circle of editors/referees linked to a journal and/or research paradigm, which may hold the danger of predictability as to what kind of paper will/will not be accepted; and on the other, a flexible pool of readers, which may be more difficult to control, in the sense of ensuring referee- and hence article-quality, but may yield more innovative papers.
To conclude, the degree of specialisation of the journal and of the editorial and/or advisory boards will be one factor determining what is expected of the referees.
2.4 Who is a ‘quality’ referee and what are the selection criteria?
It goes without saying that any referee should have proven expertise in TS, or one/several of its research areas. This means they should have respectable CV’s, publication lists and demonstrable experience with evaluating papers. However, this may not be enough. As I have pointed out before, any journal must be forward looking and occasionally that might mean taking minimal risks.
To my mind, referees must therefore also be sufficiently flexible to evaluate and accept approaches and viewpoints that differ from their own. Personally, I do not believe in closed-circuit or strict paradigm-based reviewing. In any case, all reader reports end up with the editorial board who can then discuss them and decide, possibly asking for extra comments on the basis of the reports received, especially if they are not in full agreement. I have never had to deal with reports that were at opposite ends of the evaluation spectrum.
This approach to refereeing implies that referees must have sufficient insight into the requirements of sound research on a more general level, including its creative aspects, and that they must be able to work in a team, i.e. to liaise with the editors and/or editorial board. This also means they must respect deadlines, just as much as the contributors themselves.
Last but not least, in order to facilitate comparison and decision-making, it is of the essence that referees formulate concise, precise and founded comments on the basis of the guidelines they have been given. Which brings me to the next topic.
2.5 Can referees be given guidelines and can referees be refereed?
Reviewing a manuscript means assessing its strong and weak points with a view to evaluating whether or not it is publishable, as it is, or following some degree of rewriting (cf. my working definition and table 4).
In order to bring this task to a good end, both the editorial board and its referees must be absolutely clear about what is expected of them. When I act as a referee I greatly appreciate being given precise but concise instructions. Without instructions the range of issues one might wish or have to comment on is endless. Besides, external referees may not be fully informed about the internal board’s publishing policies. The editors or editorial board must therefore draw up their guidelines on the basis of the journal’s editorial policy and their knowledge of its reading public. Even new referees will supposedly be acquainted with the journal already, but supplying them with a written version of the editorial statement or policy is a good start. If articles have been sent in response to a call for papers, forwarding this call to the referees is also advisable.
The refereeing guidelines themselves may vary from issue to issue, and guest-editor to guest-editor, provided that they do not deviate from the journal’s basic ‘mission’ and that they are drawn up in consultation with the board or the chief editor representing the board. Good guidelines will prevent situations in which the board find themselves with comments from different referees that are difficult to compare. They will preclude comments that are too personal or based on a personal preference for particular writing styles and/or approaches. They will also restrain referees who might otherwise rewrite the article or comment on it in such a manner that the rewritten text would become the article they might have written.
In a nutshell, the referees must be told what issues to comment on: the overall intrinsic quality of the article, its potential, its publishability, the degree or kind of rewriting required (and its feasibility), its academic writing style, bibliography etc. keeping in mind that authors can and often are requested to rewrite sections, render some passages more concisely or expand others.
Content and form are notoriously hard to disentangle, since the relevance of the topic is connected with the way it is treated. Referees may therefore be expected to compare the text with others they have read in the field, considering both content, presentation and the level of academic writing, but language and editing details are definitely the domain of the editors (cf. above). If too many articles have been received, and difficult decisions must be made, a referee might be ask to draw up a well-argued order of preference for the editors’ consideration.
The final responsibility for accepting or refusing a contribution obviously resides with the editorial board and (guest) editors. The more explicit the refereeing guidelines and expectations, the better referees themselves can be evaluated and their judgements ‘assessed’. Does this mean that a referee’s reliability and thoroughness can only be determined in retrospect? Not necessarily. Assessing the referees may not have to wait until after the reviewing process is completed: communication helps. Referees can also be asked to clarify some of their comments. Not only the quality of contributions, but also the quality and efficiency of the refereeing process can be improved along the way. The issue is not really one of ‘control’ or ‘assessment’ but of designing a smooth working procedure.
Guidelines and a clear starting point (editorial statement, call for papers) will help prevent problems of consensus. What is more, complete consensus may not be required, but a majority consensus or a consensus about the degree of rewriting needed can certainly be achieved in this way. Referees who do not respect guidelines or deadlines may simply not be asked to act as referees again, but on the other hand, everyone in academia has myriads of academic and administrative tasks to attend to so a degree of flexibility on the part of the board is usually advisable.
2.6 How do contributors react and do they have a say? What can they learn?
Generally speaking, contributors react very well and appreciate constructive criticism.
Only very occasionally have I had to intercede or explain what exactly was meant by a particular comment. It is very important for the editors, who are the link between the contributors and the referees to formulate criticism clearly and to motivate it.
If an author has taken the time and trouble to write an article, he or she can be expected to get serious comments, even if the article is rejected. Indeed, this may very well be because it does not fit into the volume rather than because of some intrinsic deficiencies. Authors should also receive editorial comments within a reasonable period of time. In fact, upon receipt of the article they should be told when to expect comments.
Besides, authors must be given some time to consider how to reformulate, expand or reduce, if that is what they are asked to do. As in the case of the refereeing process itself: clarity about what is expected remains the key for all parties concerned. If an author decides not to rewrite, the journal editors also like to be informed.
More generally speaking, good, constructive comments can offer young scholars an insight into the shortcomings of their manuscripts so that they can avoid typical pitfalls or remedy personal deficiencies in the future (cf. table 3). Having articles published is a learning process, having one’s articles refereed is part of this learning process, and of the publishing game.
Peer reviewing is a way to improve publications whether by established senior researchers or younger ones. It is not a way of keeping newcomers out of the publishing circuit. Quite the contrary is true:
|
peer reviewing is a form of quality control but also a inroad into academic writing
|
22/11/2005