The Peaks, Valleys, Perils, and Rewards of Peer Review: Part II. How to Read Peer Evaluations
Some Modest Advice for Authors and Evaluators
(for Part I: How to Write Peer Evaluations, click here)
The peer evaluations have arrived. It’s okay, deep breath. Go for a run, take a shower, pour yourself a scotch—it’s not going to be fun. Even if your paper has been accepted, This. Will. Not. Be. Fun. No matter how good the reports are, readers will find some fault with your carefully researched essay, toss some casual curse over your efforts, and you will obsess about these two lines, only and forever.
We at JQR believe peer review is a system that is truly good for scholarship, for mentoring, for the field. It is how the discipline keeps an eye on its young, how it prunes or weeds out ideas that are not yet or no longer bearing fruit. Of course it is also where petty grievances may be worked out, where bruised egos may strike back, where anonymity can bring out a reader's worst self. But to depict peer review only at its worst (small-minded, hide-bound, mean) does an injustice to the tremendous labor and good will that goes into the process, and might rather reflect how scholars on the receiving end tend to read their own reports (viz. disproportionate obsessing over a single negative comment).
Before you scroll down through your reader reports it might be helpful to know how at least this journal approaches the platonic eidos of the process.
Who is reading?
For any given article, the editors seek readers representing a range of positions. In a perfect case we secure three to five evaluations: a specialist or two of course, a senior person in the discipline who sees the broad horizon, and a scholar not directly in the paper’s field, but who may share a methodology or sensibility. This person stands in for our Scholar Plony—the JQR reader who is interested but not an expert; it means a great deal to us that this reader should see clearly what is at stake in the essay, why it matters and is interesting. This is how ideas migrate out of silos. Occasionally we also invite a smart junior person to read as a way both to give her some experience and to test her out for the future. Wouldn’t it be great if every submission received this range of evaluations? If all readers responded quickly to our invitations and brought their wisest, most measured, best selves to the task? (See part I.) But the editors have to work with what we’ve got, so if your piece has fewer readers—or if they seem flawed—it is likely not for lack of trying.
Read generously.
Just as we ask peer evaluators to read essays generously, so we encourage authors to read evaluations generously. You should not harrumph with disgust when a reader admits that the paper is outside of her expertise—this is by design; another letter is surely from an expert and we evaluate the lot together.
Also note, if a letter is personal or especially nasty we usually pass it on to you anyway, on the thinking that you may want to know such a reader is out there. Rest assured the editors also see, and discount, such vitriol. We treat letters tinged with inappropriate meanness with a grain of salt and we do not enjoy sharing them. (Again, see part I)
Keep in mind that the sins of an evaluator may be attributable to such innocent causes as a late night or the distracted reading of your article on a train ride home. Indeed, stepping back, a petty or superficial reader does still offer some useful perspective. No one will ever read your essay as carefully as you, and yet, if you are very lucky, the published essay will have a wide audience—one often made up in part of casual, superficial, and even sometimes petty readers. An evaluation that misses your point entirely may give you valuable information about the way in which many other people, too, will miss your point entirely.
What if the evaluators say different or contradictory things?
Just as the editors read letters for trends and shared observations, so should you. Try to triangulate from their criticisms rather than necessarily addressing them line by line. Do they think there is too much filler? That the bibliography is undernourished? That the theoretical excursus feels tacked on? These sorts of criticisms are fairly easy to address. However, two readers will rarely say the very same thing—and some may differ widely. Does one suggest you cut the opening section and the other say expand it? Does one think a reading is tendentious and the other think it brilliant? Here is where we trust that the paper is very much yours. Listen carefully to the advice, then choose for yourself which direction will both improve this paper and forward your larger research agenda. Make the letters work for you.
Note that when a paper is revised and resubmitted it goes back out for a second round of evaluation. At that stage the evaluators see all the original letters (anonymously) from the first go round, so they know that you were confronted with conflicting advice.
What does revise and resubmit really mean?
It means we want the essay to succeed, and think the author capable of making it publishable. In other words, it is worth your while to revise and resubmit, unless the reader suggestions truly push your project in a direction you are not ready, willing, or able to take it. If this is the case, know thyself: decline the offer, and submit the paper elsewhere.
Should you choose to revise, this means the conversation with the evaluators is open and ongoing. If you truly ignore the advice of a reader, she will appreciate hearing why you chose to do so. Address a memo explaining changes to the editors, who will send it to the reviewers.
Dealing with rejection
It may be helpful to know that we only take a small percentage of articles submitted, 12 to 18 percent or so. Compared to larger humanities journals these may be rather friendly odds, but ultimately the fact is we receive many more compelling essays than we have pages. We reject pieces for a whole host of reasons, some unrelated to the quality of the scholarship. Many papers are too narrow for our broad mandate and would fit better in a more specialized journal, others are too general. Sometimes we have a glut of strong submissions in one field, making it much harder to get a paper accepted in that subdiscipline than in others. Often a paper is damned good—but we simply have too much copy, and so must make hard choices among qualified work. In other words, sometimes it’s really not you—it is us.
Giving credit
Some criticisms are weightier for us than others, and among those we take especially seriously is that the author has not engaged important existing work on the essay’s topic. While this does not mean we want to see full bibliographies in footnotes (we don’t!), we want to know the author knows who is working in closely aligned topics and be conversant with that scholarship. The editors are aware that criticism about citations can also be cover for grouchy evaluators who do not see their own work sufficiently acknowledged (sometimes with good reason, sometimes not). Pro-tip: Grumpy readers may become pussycats with the addition of brief reference to their work (and they nearly always give themselves away). This is just the working pragmatics of the business, and giving lots of credit to lots of people is simply a good way to be, in scholarship as in life.
Nostra culpa
Editors make mistakes. JQR has the audacity to cover all fields of Jewish studies; peer review allows us to do so. We do our best to assess each submission rigorously and fairly, but over the years we must have erred many times in judgement—both in rejecting and accepting. We will do so again.
And here we return to our mantra: read generously.
Okay, you are ready for the long scroll.
About the Author
