6.1.2 Assessment Criteria
Two major criteria are central to the evaluation of manuscripts submitted for publication: importance and quality. Importance involves an assessment of whether the work
• Represents a scientific advance (recognizing that individual articles usually convey only small advances)
• Has clinical relevance (if the journal is to be read and the information used by practicing clinicians)
• Presents new information
• Will be of interest to readers
An additional component of importance is editorial priority, a composite judgment made by the editor regarding the value of a particular submission relative to other submissions under evaluation at the same time, weighed in the context of the articles that journal has recently published and has scheduled for publication. The reality of limited space may also be a consideration, even in the era of electronic publication. Cyberspace may appear infinite, but the attention span and patience of readers are not. Furthermore, the editorial processing requirements (see 6.2, Editorial Processing) for material to be published electronically may be very similar to those for print publication. Hence, concise submissions may be given higher priority than long ones (other factors being equal) because they take up a smaller proportion of a journal’s resources and total space allotment.
Evaluation of quality involves an assessment of how well a paper treats its topic, including how well the topic and the methods used to deal with that topic are described. For original research reports, assessment of quality involves consideration of whether
• The design and methods are appropriate to answer the stated research questions
• The research questions and the methods used to answer them are well described and rigorously conducted
• The data analysis is appropriate
• The conclusions are supported by the results
• Patients or research participants were treated ethically (see 5.8, Ethical and Legal Considerations, Protecting Research Participants' and Patients' Rights in Scientific Publication)
The quality of the writing may be a major factor in the assessment of editorials, commentaries, and other nonresearch submissions where the elegance and impact of the writing itself may constitute major reasons for publication. For research and review articles (see 1.0, Types of Articles), writing quality (especially clarity) may affect the reactions of editors and peer reviewers even though the importance and quality of the research should be the main focus for assessment. Writing quality can be improved by manuscript editing (see 6.2.1, Manuscript Editing), but only if the research is described with sufficient clarity to permit basic understanding.
The specific nature or direction of results should not be an issue in quality assessment. If a paper addresses an important question and uses high-quality methods to answer it, the results are worth publishing no matter what they are. Publication bias that results from a tendency for investigators not to submit, or editors not to accept, papers that do not report statistically significant “positive” results should be eliminated.2 A well-done study that shows that a particular intervention is ineffective is usually just as important as a study that reports a “positive” result.