William Thomson A Guide for the Young Economist Cambridge: MIT Press, 2001 Writing Referee Reports You may have been asked by your adviser or another faculty member in your department to referee a paper for a journal. You agonized over the job, not knowing exactly what was expected of you. As a young assistant professor and scholar you are also very likely to receive other requests to review manuscripts submitted for journal or book publication. It is not, however, a skill taught in any of the classes you took. Even if you have already submitted one of your own papers to a journal and received reports on it, they probably will provide you a very incomplete guide. The purpose of this chapter, therefore, is to help you evaluate others' work and produce a useful critique. Your two main goals will be to assess the manuscript's suitability for publication and advise the author about improving his or her work.1 1 Components of a Report Your report should consist of the following components, listed here in order of increasing specificity and decreasing importance. They pertain first to the substance of the contribution and then the quality of its exposition. 1. Summary of the paper 2. Overall evaluation of the paper with your recommendation about the publication decision 3. Comments about the model and the results 1. Hamermesh (1992) concludes his useful article about getting one's own work published with some good advice on refereeing. You will, of course, have other occasions to evaluate others' work. Eckel (1999) provides very useful guidance on reviewing grant proposals. I suggest that you read her essay not only when writing such a report but also when drafting your own proposals. 104 Chapter 3 4. Assessment of the exposition a. The structure of the paper b. Secondary aspects of the exposition c. Details of presentation I take up each of these items in the following subsections. 1.1 Summary I recommend beginning your report with a summary of the article even though, as an associate editor or as an author, I have rarely found such summaries useful. Indeed, they often amount to little more than a restatement of the abstract and tend to be used to pad out reports. However, if the summary is written in your own words (instead of being lifted from the paper), it can be helpful in several ways. • It makes the report a self-contained document—that is, one that can be read on its own. • It reassures the associate editor and, then, the author that you have read and understood the paper. • The effort required to describe the paper—I repeat in your own words—will be useful to you in forming your opinion. Summarizing will help you understand the nature of the author's contribution tetter. In the process, you may discover that you disagree with the author's view of certain aspects of the work and want to recommend highlighting the significance of a particular assumption or providing a different interpretation of the findings. If so, you are no longer summarizing the work and should make that clear, perhaps saying something like: "Although the author presents this paper as a contribution to the theory of strategic games, in my view the main result is Theorem 2, which has important implications for the theory of implementation. I suggest that the author present the study as a contribution to that literature." 2.2 Overall Evaluation Your overall evaluation of the paper should be based on an assessment of the following points: • Its significance for the field, Writing Referee Reports • Its appropriateness for the particular journal to which It has submitted. One criterion is the paper's originality: Is the contribution to the literature substantial enough? Another is subject matter: Is the topic pertinent to the journal's statement of purpose? (There should be a good match between them.) Your evaluation should conclude with a recommendation about publication. Your advice can be one of the following: • Publish it; and, if the journal has a notes section, recommend whether the paper should be a regular article or a note; • Encourage the author to resubmit, reserving your opinion about publication until the author has addressed the points that concern you; • Reject it. Even if your recommend publication, you will undoubtedly have comments and suggestions for improvements, and you may feel that some changes are necessary. Be explicit about which ones you see as essential. If you favor inviting resubmission, again, be explicit about the improvements you require to endorse publication. But be realistic. Don't accept the paper on condition that the author accomplish some unlikely feat of generalization. If acceptance would require too much improvement, it is safer to recommend rejection. Being vague or unreasonably demanding in your requests will put you in an awkward position in the second round, when you will recognize that you were not precise enough or were asking for more than could reasonably be delivered; you will then have to decide whether the improvements go far enough in the direction you indicated to justify publication. You also complicate the associate editor's task, making it harder to argue the case for rejection should the editor decide that the paper has not passed the threshold for acceptance. 1.3 Comments on Model and Results You should reflect on the interest of the model and the significance of the results. Are the reasons for undertaking the study compelling? Is the model specified so as to capture the essential aspects of the phenomenon the author is attempting to explain? Are the assumptions economically relevant and the conclusions interesting? 106 Chapter 3 You should also think about the mathematics used. Are the results correct as stated? Could they be strengthened? Could their proofs be simplified? Why did the author rely on sophisticated techniques when all earlier studies used elementary mathematics? Is the data used to prove the thesis well chosen for that purpose? Have the appropriate significance tests been run? You should not necessarily expect to find the answers to all possible questions of this sort, but if the paper does not contain the information needed to clarify most of them, request that the author provide it at the next round—either in the reply to the referees or in the paper itself. You may end up deciding that some of the material in the reply is worth including in the paper or, conversely, that some developments inserted in the revision in response to your comments do not deserve to be published after all. As time passes, your thinking may evolve, or the revision may contain information that requires you to modify the opinion expressed on the original version. Be flexible and, certainly, acknowledge any misconceptions you may have had during the first round. If you think the paper is fundamentally flawed, you will find it difficult to motivate yourself to work through the proofs and you are not obliged to do so in such a case. On the other hand, you should have checked the proofs of a paper that you recommend for publication. Occasionally, this process takes considerable time, and it may be acceptable, even unavoidable, to skip some of it. Since a proof often includes several steps or cases with similar structures, you may look at only one of them at your first reading. If you find too many imperfections in the proofs you study in detail (missing quantifications, inequalities going the wrong way, and so on), you will grow suspicious of the entire work. You won't trust the author about the steps left to the reader with the claim that they are "easy" or "similar to proofs in an earlier paper" or "only involved tedious calculations" (the standard excuses). Insist that in addition to fixing all the errors you noted, the author provide complete arguments, either in the revision or in a reply to the referee. Reserve your judgment until then. If the author has stumbled too often, simply reject the paper. If you have discovered no flaws in the proofs that you did check, you will feel reasonably confident that the argument is correct as a whole, especially if it makes intuitive sense. Under these circumstances, you may be justified in omitting some proofs or some steps of proofs. If you do, though, inform the associate editor of how extensively you have checked the mathematics. Writing Referee Reports 107 1.4 Comments on the Exposition The author should have done everything possible to make the study as transparent as possible. But it is not sufficient that the exposition be clear. You should ask whether it could be even clearer. Nor is it sufficient that the paper be understandable by researchers in the same field. If it can be written so as to be accessible to a wider audience without any loss of substance, that is what the author should do. It will help you formulate your comments to think about papers you found particularly lucid or enjoyable and identify the reasons why you felt that way. A natural way to organize your comments on the quality of the exposition is to start with issues of overall structure and proceed to questions of details. • Comments about the structure of the paper. The structure of the work should be immediately clear. It is the frame that supports the whole thing. Ask yourself whether the paper is well organized. Is the progression from introduction to conclusion natural? Is this issue really central to the argument? Should this proof be relegated to an appendix? Would it be more effective to present this theorem as a lemma instead and this proposition as a corollary of the main theorem? • Comments about secondary aspects of the exposition. Address whether a step in a proof taken from some earlier article needs to be reproduced or whether a reference to the work suffices. Would numerical examples or figures be useful? Should more effort be devoted to placing the paper in the context of the literature on the topic? Problems that are not serious for you because you know the literature may in fact prevent others from understanding anything. You may read an ambiguous quantification as it was intended, but readers not familiar with the relevant literature might read it the wrong way; and for them the paper may make no sense. Make sure that every detail is handled correctly. Although the general inclination of referees is to ask for deletions, do not hesitate to ask for changes that may lengthen the paper if you feel they will make it easier to understand, even though they do not lead to more general results. If you recommend shortening the paper, be, once again, very precise; authors are always reluctant to eliminate anything. A request to reduce the length by half is not precise enough: list the specific cuts. And when evaluating the revision, don't be fooled 108 Chapter 3 Writing Referee Reports by changes in font size, margins, or spacing that give the appearance of compliance but don't actually shorten the work. • Comments about the details of the presentation. Tell the author whether a formula should be displayed on a separate line or a condition given a different name. Should the importance of a conclusion be emphasized by using a distinctive typeface (such as italics)? Should two paragraphs be merged? You may also add a list of the typographical errors you noticed. 2 Distinguish between Nonnegotiable Requests and Suggestions for Changes You may want to divide your requests for revisions into two parts. • Some requests for changes are nonnegotiable: The model should be coherent; there should be no errors in proofs; proper credit should be given to previous contributors. You have a right to demand that the author respect such universal principles of good writing as simplicity and unity. The structure of the paper should be clear and its language should be free of unnecessary technical jargon. In the revision, do not accept as an excuse for persisting in errors you pointed out, or in features of the paper to which you objected, that they are present, even common, in the earlier literature on the subject or in the work of such and such a well-known predecessor. We may temporarily accept the limitations of a model or of an approach because certain conceptual issues have not yet been satisfactorily resolved in the field, or because the right techniques have not been developed. That is a necessary precondition to progress. But committing errors that can be avoided, given the state of the art, hampers progress and is unacceptable. • Other suggestions for change are simply ideas for the author to think about. You leave them to the discretion of the author. You believe that they would improve the paper, but you also see why the author might disagree. You are aware of counterarguments to your proposals or of the costs of implementing them. An additional way of justifying the model's specification may lengthen an already-long introduction; presenting a proof for the K-person case instead of the two-person case may obscure an argument that is now very transparent; dropping certain regularity assumptions on preferences and technologies may prevent the use of elementary mathematical tools; and so on. Certain features of a paper may not be to your taste and yet be quite legitimate. In these cases you can only suggest changes and try to convince the author of your reasons for wanting them; you cannot insist on them. For instance, you may not care for the style in which the paper is written, but you can't force your own style on the author. You may have to accept a verbal or informal presentation of a proof if the author's goal is to make the argument easily accessible to the less mathematically oriented readers, even if your own preference is for a formal proof. However, suppose that this verbal proof is missing critical information, for example, that in an informal argument intended to provide the intuition of a proof, definitions are ambiguous or quantifications are not clear or no reference is made to a conceptually important assumption without which the formal proof would irrevocably fail. Then readers can only be fooled into believing that they understand the argument, and you should demand that these important elements be made explicit. It is probably best to append your requests for changes to each part of your report as enumerated in the preceding section. 3 Evaluating Revisions How do you go about assessing a revision? First, compare it to the earlier version, section by section and paragraph by paragraph. Check how each of your numbered recommendations and requests for changes has been implemented. If the paper has been significantly reorganized and the pagination changed, this will not be an easy exercise. In this case, an author's reply to the referees will be very helpful in guiding you through the changes. Request such a reply; authors rarely spontaneously supply one. If you asked for one and the author did not bother to send it, have the associate editor demand compliance. If the author has paid only lip service to your suggestions, write to the associate editor and point out the critical comments the author has ignored. Here, too, it is quite reasonable to request that the author comply before you study the paper. Unless the changes are very minor, you need to go over the whole thing again. New errors are often made in the process of correcting existing ones. Some notational conflicts may appear; or the sequencing of definitions and results may be disturbed in ways that have escaped the author. Besides, several months have probably passed, and you may have new points to make. Unless only a few problems remain, ask for another round of revisions. 110 Chapter 3 Writing Referee Reports 111 4 Length and Style of the Report I do not have a specific recommendation on how long your report should be. A review of a paper that suffers from a fundamental flaw may be very short, whereas comments on a paper you found exciting may take several pages. In that case, your assessment will probably be short, while your suggestions will constitute the bulk of the report. Nor do I have a particular recommendation on the related question of how many hours you should devote to the job: the time needed will vary considerably from paper to paper. An hour may suffice for one that is obviously below the line for acceptance, whereas you may need four to five hours for a potentially publishable piece for which you have to supply a long list of requests and suggestions for changes. A paper with long or difficult proofs may take fifteen to twenty hours if it appears to be an important contribution that you think will be very critical for you to understand well for your own research. In this last case, strictly speaking, most of the time you spend will not be required to do the actual report. Concerning the style of your report, my most important practical recommendation is to number the various recommendations and requests that you make. Don't lump several points together. If your request has two parts, call them Part 1 and Part 2. At the next round, the numbering will make it very easy to check out whether your suggestions have been taken into account. Sooner or later—and in fact sooner rather than later—you will receive a revision from an uncooperative author who has done the bare minimum to address your comments while claiming to deal with them thoroughly. By being precise in your demands, you will make it more difficult for authors to avoid making the changes you think are needed. Referees' reports are not intended for publication, so do not bother polishing your English. Do not worry about stylistic issues such as repetitions, inconsistencies of tense, and so on, which can be time-consuming to correct. Save the effort for your own papers. Your priority is to be clear and definite. By revising your report to achieve these goals, you will eliminate most of the stylistic problems anyway. 5 The Cover Letter Do you need a cover letter to the associate editor (apart from "Please find enclosed my report on so-and-so's paper. Sincerely")? Sometimes yes. A first (but rare) reason is that you may want to discuss concerns about a possible conflict of interest with your own work. Again, as discussed later, if you feel sufficiently strongly that there is such a conflict, you should decline the job at the outset. Another reason is that you have harsh things to say and you fear being identified. The difficulty of remaining anonymous is all the greater if you need to mention work of your own that the author has failed to take into account properly. Such situations are, of course, not rare, and they will become more and more frequent as your CV lengthens. As noted earlier, in many cases, the associate editor has called on you to referee a paper because you have contributed to the relevant literature. Keep in mind, though, that complete anonymity is impossible anyway and that one of the first things some authors try to do when receiving a report is to figure out who wrote it.2 It is something that you just have to accept. If some issue of integrity, such as plagiarism, has to be raised, the cover letter may be where you should do so. On these occasions, however, it might be a good idea to first seek the advice of your adviser, if you still are a graduate student, or of one of your senior colleagues, if you are a young assistant professor. Your overall assessment of the paper and your recommendation do not, however, belong in the cover letter. You may want to provide a short summary of your report, or restate there in a different way certain points that you make in the report. But I object to the explicit requests of some editors and to the policy of some journals that the recommendation about acceptance or rejection not appear in the report sent to the author but only in the cover letter.3 When a paper is turned down, the author is entitled to know the basis on which the decision was made. If you have not received an acknowledgment a few weeks after you sent your report, you may want to check with the associate editor that it was not lost in the mail—or in cyberspace. 6 General Recommendations In this section, I discuss the need to take a critical stance and the extent of your responsibility to the journal and the author. 2. One clue: the person most frequently cited in the report is usually the author of the report. 3. I have heard several reasons for this policy. One is that it allows referees to feel more comfortable expressing negative opinions—it protects them. A second is that it shields authors from the harsh things a referee may have to say. The editor's letter can tone down overly critical comments made by referees. 112 Chapter 3 Writing Referee Reports 113 6.1 Expressing Judgment Like many first-time referees, you may not feel confident about expressing subjective opinions on the suitability for publication of someone else's work. Nonetheless, you should not limit yourself to an enumeration of objective statements about the paper. Take a stance. The following points should help you do that. First, the associate editor will also look at the paper—and in some cases study it—and there may be other referees (although often there are not). Moreover, subjective judgment is an inevitable part of the evaluation process. Some referees, perhaps feeling uncomfortable about rejecting a paper for subjective reasons, end up making poorly substantiated arguments against objective features of the paper in order to support a negative recommendation. For example, they emphasize errors in a proof when its imperfections could be fixed. (Errors are rarely completely avoided, and in some cases they do invalidate a proof.) Or they assert that the author's result is a special case of someone else's earlier theorem, when it is not (although it may well be closely related to a known theorem). Altogether, such referees are seriously undermining the usefulness of their reports. If you believe that the paper is not significant enough for the journal, express that judgment as the reason for your advice to reject it. Imperfections in proofs do not necessarily disqualify a paper from eventual publication. If the results appear to be true and are interesting, simply point out these imperfections and ask that they be eliminated. Also, if the relation between the results reported in the paper and other studies is unclear, demand that it be clarified. By itself, the fact that the author may not have understood this relation well, or may not have described it accurately, is not sufficient grounds for rejection. What is very helpful to the associate editor, however, is for you to separate the statements of fact in your report from your expressions of judgment. Here is an illustration. The theorem as written is incorrect. It would be correct, however, if preferences were required to be strictly convex." [Here you are making a comment about an objective aspect of the paper whose validity is not a matter of judgment.] Unfortunately, when strict convexity is imposed, the enlargement of the class of economies for which the author shows existence of equilibria is not of sufficient interest to justify publication in this journal. [Now you are expressing your subjective judgment, with which other readers may disagree.] 6.2 When Is Withholding Judgment Appropriate? In some cases, the decision to publish or not will seem to be primarily a matter of general editorial policy. For instance, the paper is much longer than the articles commonly published in the journal. Or it deals with a subject that does not match well the journal's statement of purpose. Or it is written at a significantly higher or lower technical level than that of the journal's standard article. Perhaps it is more didactic in tone or purpose, or its contribution is principally conceptual, whereas the journal's emphasis is on techniques. Or vice versa. If so, raise these issues in your report and let the associate editor and the editor decide how to deal with them. In principle, they have sent you the paper because they do not object to considering it for their journal. But they may not, in fact, have looked at it in great detail. 6.3 The Referee's Responsibilities to the Journal and to the Author Your main responsibility is to help the journal decide whether or not to publish the paper. But you should also consider helping the author produce a better article. You can usually do that at a small cost because you have thought a lot about the article. Be generous with your advice. Even if you recommend rejection, your comments will help the author revise the paper for a different journal. Moreover, the other referees, and perhaps the associate editor, may disagree with you and favor publication; in that case, your comments will be helpful for this journal as well. Almost every paper contains something useful and publishable if properly reformulated and targeted to the right audience. Even if you feel sure the paper does not deserve to be published in the journal you are evaluating it for, why not let the author benefit from the efforts you expended in forming your opinion? Give your advice about the best means of bringing out its strong points for resubmission to a different journal. After all, you are probably one of the first readers (sometimes the only one) who has studied the paper so carefully. Admittedly, in some circumstances, it is difficult to motivate oneself to suggest improvements, especially when the author's objective seems to have been to violate all the standards of scholarship. (It does occur.) Being generous with your advice, however, does not mean correcting major flaws in the author's logic or providing the proof of a conjecture stated in the paper. Although some of your comments might lead to major improvements, it is not your responsibility to produce such help. You are not a coauthor. Conversely, very few papers are acceptable as originally submitted. Be tough. You do a disservice to the journal, and to the field (remember that it is in most cases your own field), by being too lenient. And you are not doing the author any favor by failing to mention all the problems you noticed. Moreover, it is easier if you are a little tougher than needed at the first round and slightly more permissive at subsequent rounds. After being too lenient on the first round you may discover in the revision issues you missed earlier that definitely have to be addressed before you can recommend publication. Being tough is not the same as being mean. There is no pleasant way to tell an author that his or her work should be rejected, but that is absolutely no reason to be insulting. Do not make disparaging comments about the author's intelligence. I have heard the argument that because in most cases a paper could be submitted to other journals, we need not worry too much about rejections that should have been acceptances. Certainly, we all make mistakes. Yet the argument comes dangerously close to condoning sloppy evaluations. Moreover, it is not really very convincing, given the hierarchical perceptions of different journals' prestige. In some areas, there are no more than three or four possible outlets for a given work, and they are rarely equivalent in terms of the visibility and status they would give the work or its author. Moreover, if you are the only referee for a paper, your opinion may carry a lot of weight. Finally, the author may have already submitted the paper two or three times. For a young person being considered for a promotion, an additional acceptance by a prestigious journal can be critical. Yet one more point: if you happen to meet the author in person, there is no need to mention that you were the referee. It goes without saying that you will rarely be tempted to do so if you recommended rejection. But if you wrote a positive report, you might. The only reason for revealing your identity in a personal conversation is the desire to ingratiate yourself. Don't. 7 Deciding Whether to Accept'a Refereeing Job Now that you know what is expected of you when you receive a paper to referee, you may wonder whether you should accept the job. In general you should. But there are several reasons why you may decline. • You lack the expertise or the interest needed for the assignment. Perhaps the associate editor has misjudged your area of specialization, and the subject of the paper is too far removed from what you know well. Refereeing a paper on a topic with which you are not familiar is a good opportunity to learn about a new area and you should consider seizing it, but be realistic. If the background reading necessary for you to properly evaluate the work is too extensive, you may not be able to gain the perspective on the subject required for a good report. Similarly, you should have some minimal interest in the literature to which the paper contributes. If you don't, you will find it difficult to motivate yourself to do the work, and your view of the field will be unfairly reflected in your opinion. • You fear a conflict of interest. That is another good reason to turn down a refereeing job. Conflict may arise for various reasons. You may be currently engaged in similar research and feel proprietary about the subject, or even about some specific results contained in the paper. Or you have had an article on the same topic rejected, which you think might make it difficult not to overreact in judging others' work. If you are concerned that your emotions will get in the way of a fair evaluation, decline the job. • You have previously evaluated the paper for another journal* To the extent that submission to a second journal is comparable to an appeal in the judicial system, it is crucial to a fair hearing to have a new judge. In most cases, there will be other competent people to evaluate the paper, and its fate should not be made to hinge on the taste of a single person. However, there are also good reasons why you may want to look at the paper again and send a report. - It has been revised, perhaps substantially.5 - Your opinion of the paper, or perhaps the field, has evolved. - The second journal differs significantly in style and reputation from the first. (In these cases, a different sort of report is called for. You cannot simply pull out the old one.) 4. This will not, of course, happen for a while, i 5. You will also receive resubmissions in which none of the comments you made on an earlier version has been taken into account and in which not even the typos that you ( painstakingly listed have been corrected. 116 Chapter 3 Writing Referee Reports 117 - The author chose this particular journal for a second attempt in response to a suggestion made in your first report and you feel a certain responsibility for having encouraged this submission. - You have a knowledge of the subject few others share, and the associate editor may want to hear your opinion anyway. One could argue, of course, that if so few people are qualified to referee a given work that different journal editors have to use the same referees, it probably isn't a significant contribution to the field. I do not really agree. An editor may feel that there are only a few individuals who can be trusted with writing a good report. That does not mean that, with time, the article might not gain readers and eventually have some impact.6 - If you initially recommended rejection of a paper mainly because it was not "to your taste," it may be more natural for you to decline the assignment than if your criticisms had to do with such issues as the correctness of the analysis or the quality of the scholarship. In these latter cases, a quick look at the paper will tell you whether the problems you noted in your first report have been addressed. If they have not, you will save everybody precious time by sending a revised report that takes into account whatever changes have been made in the paper. If you were in favor of publication but the paper was rejected anyway, you will certainly welcome the opportunity to have your opinion heard again, and few people would object. However, if you do accept a second refereeing job on the same paper, let the associate editor know that it is your second time. In your cover letter, explain your earlier involvement with the work. There are several ways in which editors can use your assessment in this situation. They can put it aside, use it informally as an additional input into their own opinion, or treat it as a regular report. Let the individual editor decide. • You are concerned about meeting the deadline suggested by the associate editor. Being occasionally late by two or three weeks is not a major problem though. In our discipline, the publishing process is rather slow—as you have probably already discovered when submitting your own work. On the other hand, being deliberately slow to avoid receiving additional assignments too soon is not the best use of your 6. Hamermesh (1992) disagrees with me on this issue. knowledge of game theory. Try to do a little better than the average referee; the associate editor and the author will be grateful.7 But if you have received so many refereeing requests that you risk being swamped—and this may happen sooner after you graduate than you expect—you certainly have the right to say no. In fact, you should. Do not let refereeing work hurt your own research. On occasion, you may accept an assignment but have to postpone your evaluation of a paper because the author did not include all the proofs, or the article is based on some earlier work that is unpublished or not readily available. Get the material you need from the library, a colleague, or the author's web page. In some (rare) cases, you may have to write to the associate editor and request that the author make certain items available to you. If you decide to decline an assignment, the sooner the better. So, quickly assess the paper when you receive it. Anywhere from a few minutes to half an hour should suffice to make up your mind. If you let it sit on your desk only to discover several weeks later that you have to turn down the job, you will have caused unnecessary delay. Or, out of guilt for this delay, you may do the work anyway. But if you had good reasons to decline it in the first place, they probably still apply and you will not write a good report. Acting quickly is also important if, as discussed earlier, you need additional material that may take time to obtain. You do not want to discover a whole two months after receiving the assignment that you absolutely have to consult a related discussion paper by the author or a paper published in a journal to which your library does not subscribe. 8 Benefits to You of Your Refereeing Work Take your refereeing jobs seriously. Refereeing appears to be a very unrewarding activity: essentially only one person, the associate editor, knows who produced this thoughtful report. However, the job is part of your service to the profession. It does have a cost, but your turn will come to be the beneficiary. And even from the selfish viewpoint of your own preferences, your efforts will not be in vain. By repeatedly doing a good job you are helping your reputation; editors talk to each other 7. Hamermesh (1994) is a good source of information on what the usual delays are. He also discusses in detail the sociology of refereeing. 118 Chapter 3 and to other members of the profession. The quality of refereeing is often mentioned in recommendation letters written on behalf of young researchers. Your work will eventually earn you a spot on a board of editors, giving you more of a chance to make your opinion count. Another benefit of refereeing is that it helps you keep up with the literature. Next to presenting a paper in a class, there is nothing like refereeing it to become really familiar with it. This in-depth work will be very useful to your own research. 9 References Eckel, Catherine. "Writing Reviews for the Economics Program at the National Science Foundation That Will Make Your Program Officer Love You." Committee on the Status of Women in the Economics Professions Newsletter (Winter 1999): 11-12. Hamermesh, Daniel S. "The Young Economist's Guide to Professional Etiquette." journal of Economic Perspectives 6 (1992): 169-79. -. "Facts and Myths about Refereeing." Journal of Economic Perspectives 8 (1994): 153-63.