The methods presented here aim at automating the reviewer-article assignment by using topic modeling. The system only needs the article’s abstracts and samples of abstracts written by the reviewers. This removes the biases introduced by either authors suggesting reviewers or reviewers bidding for their favorite articles to review.
Also, we have develop a new method for estimating the underlying score of an article based on judgments of a set of reviewers. Our method goes beyond a simple average of the reviewers’ scores, and it controls for the variances and biases of the reviewers. Sometimes reviewers are too harsh or too nice compared to other reviewers and the system regresses their scores to the mean, accordingly. Also, some reviewers are very variables in their reviews or others are very consistent with their peers, and our system weighs their scores differntly.
The web implementation of this research has been possible thanks to the help of Tulakan Ruangrong.