Follow
Area: Applications
As the field of NLP advances, more NLP approaches find their way into practical applications. Yet while NLP has been successful in helping humans to find and extract information, only few applications help us to consume, evaluate and aggregate information from interconnected, changing texts. The focus of this area is to develop novel applications that assist humans in solving complex real-life text-based tasks - from editorial support to machine-assisted reading via NLP-enhanced annotation.
Peer review is a cornerstone of academic quality control, yet the pressure to publish causes significant reviewing overload in many key scientific fields, jeopardizing scientific progress and undermining public trust in science. While NLP applications for analysis of scientific publications are abundant, the field of peer review analysis is just taking up the pace. Peer review is an excellent target for cross-document discourse analysis, and this area puts special focus on developing NLP applications for peer reviewing assistance.
Publications
What Can Natural Language Processing Do for Peer Review?
Ilia Kuznetsov, Osama Mohammed Afzal, Koen Dercksen, Nils Dycke, Alexander Goldberg, Tom Hope, Dirk Hovy, Jonathan K. Kummerfeld, Anne Lauscher, Kevin Leyton-Brown, Sheng Lu, Mausam, Margot Mieskes, Aurélie Névéol, Danish Pruthi, Lizhen Qu... [+8] (2024)
🔥
arXiv
[paper]
[repo]
[bibTex]
[plain]
Datasets and Code
A corpus of anonymised structured peer reviews collected during the ACL-2018 reviewing campaign. ACL-2018 employed a rich reviewing schema, with each review containing a wide range of textual, binary, ternary and numerical fields, including Strengths, Weaknesses, Summary, aspect scores, overall score and confidence scores. While openly publishing the textual data is not possible due to the ethical concerns, we make numerical data publicly available to support meta-scientific study of peer reviewing in the NLP community.
The source code for CARE: our new open-source Collaborative AI-Assisted Reading Environment. Explore the extensive documentation and try the public demo!
