Sunday, February 24, 2008

Good peer-reviews

"This isn't a scientific paper." Reading this not-so-ringing endorsement of my first attempt at a peer-reviewed paper was a great way to beat down my ego, but it didn't do much to make me a better paper writer.

"Why didn't the author's use the CMU code?" This part of a scathing review made me want to crawl through the Internet and strangle the reviewer. I had an entire section that detailed why I didn't use the CMU code. From the comments, it was clear he/she didn't read the paper.

In a peer-reviewed conference or journal, each paper is given to 3-5 people that rate the paper. The best rated papers make it in. This system is not perfect, but it is a good one.
Can you make it better? Yes. If you are invited to peer-review a paper here are some suggestions:
  • As you read through the paper, keep notes. I like reading the paper on one monitor, or on paper, and keeping notes in a simple text editor. Do this the first read of the paper.
  • For each note, refer to specific parts by section, paragraph, line, whatever.
  • Number each point
  • Point out things with the paper that you liked.
  • No matter how bad the paper is, find something nice.
  • Be specific. If there are common problems throughout the paper, find a specific example.
  • Make suggestions on what would work
  • If you don't have time to read the entire paper, don't do a review on it.
I've had several reviews that itemized very specific items with my papers that have been a great help. Because I really remember the bad reviews, I've tried hard to pattern any review I do after the good ones.

The goal of peer-reviewed publications should be to further humanity. So if reviewer feedback doesn't help educate the writer of the article, it is pointless. Great publications may come from those who originally submitted poor work. If constructive feedback is not given, the author may never become great.