PaperRater.org is a web-based tool for open review and collaborative reading of scientific publications. It finds papers through The SAO/NASA Astrophysics Data System (ADS), i.e. publications in virtually all journals related to astronomy and physics, and follows new submissions to any category of arXiv.
The peer review process has a long-standing tradition in improving manuscript quality, in terms of correctness, reasoning, accuracy, balance… However, it is not infallible, mainly because a single referee can miss out on some mistakes or can introduce some of his own.
On the other hand, as students and researchers we all read papers daily, evaluate and judge them alone, with colleagues, and in journal clubs. Done by many people, some of which domain experts, this process is able to improve a paper’s quality beyond what a single referee could achieve. If the joint wisdom of the community could be bundled.
This is what PaperRater.org is all about: to augment and eventually replace the intransparent process of peer review as a lone quality measure for publications by a public one. We seek to achieve this goal by two means:
- giving ratings to papers
- leaving public comments
If used frequently, the first aspect provides a quantitative and reliable measure of each papers’ quality. This is in contrast to more indirect measures like citation counts, which tell you how often something has been cited, but not why.
Example: Paper B criticizes paper A for, well, whatever. Paper B would still have to cite paper A, and thus let paper A look better according to essentially all scores currently employed.
Additionally, and far more importantly, we can spark and organize discussions on paper contents, reasoning, presentation, and all other aspects, which make for an excellent article. These discussions happen anyway, in journal clubs and among colleagues, on a daily basis. Thus, there is a great deal of information in the community, shaped by the opinions and thought of established researchers, post-docs, and students, all having their take on the matter.
If we could tap in this collective knowledge, wouldn’t that help all of us, readers, authors, and referees alike?
There is a slew of papers every day. Some of which are important to you, many are not, or maybe just not right now.
We need tools for organizing the constant influx of papers, break it up, sort it, and bundle it into manageable pieces. This is not getting easier when your interests are diverse, as many of us have already found out.
At PaperRater.org, we believe that the community knowledge can help here, too. If there is a lot of talking on a specific paper, doesn’t it make this paper worth looking at? Also, we all have friends and collaborator, at the next desk or on a different content, who know what we are interested in. So we offer means for you to easily find out what’s hot and what you should better read.
Additionally, we want to be able to aggregate a self-defined stream of papers, e.g. from different arXiv categories. And we want to be able to find this one paper about topic X that we have read some time ago.
PaperRater.org will read from all sources relevant to us, currently this means arXiv and ADS, and make sure that the papers found with any of these sources are consistent and up-to-date. And, like many aggregators, PaperRater.org allows you to set your own tags, such that you can find the papers by your terms, literally. By adding tags, you also help others, since you bring order into to unstructured streams of papers much more specifically than even the more refined arXiv categories (e.g. astro-ph.HE) can. In contrast to the traditional, library-demanded rigid category scheme, tags are flexible and can easily follow any upcoming hot topic. And thus can you. This is another reason, why we believe in the power of the collective knowledge.
Over time, we’ll probably refine the ideas laid out above, extend them, and come up with better ones. Let’s see where this journey will leads us…
If you want to see how all of this works, have a look at the Getting started guide.