The ICMJE recommendations

The most important guideline for writing a manuscript to be submitted to a medical scientific journal is the Manuscript Preparation and Submission recommendations from the International Committee of Medical Journal Editors (ICMJE). It can be found here (1).

From a statistical viewpoint, two of the recommendations are exceptionally useful. The first is: "Describe statistical methods with enough detail to enable a knowledgeable reader with access to the original data to judge its appropriateness for the study and to verify the reported results."

To be able to judge appropriateness and verify reported results is not only about naming the methods that have been used, it is also about why a particular method has been used, the investigators' intention. It may therefore be necessary to motivate the method choice by explaining the analysis strategy. Vague or otherwise unclear statements such as "independent samples t-test" and "as appropriate" should be avoided because several independent samples t-test have been developed, and unless it is explicitly described, it is impossible to know what an investigator considers to be appropriate. The investigator may be mistaken.

The second exceptionally useful recommendation is: "Link the conclusions with the goals of the study but avoid unqualified statements and conclusions not adequately supported by the data. In particular, distinguish between clinical and statistical significance."

Ideally, a report starts with a research question and ends with a conclusion from the investigation aimed at answering the research question. However, in practice, manuscripts are not always coherent. The different parts do not fit together and work toward the same goal. The statistical evaluation of results is not always clearly linked with the research question, and the conclusion is often just a repetition of the statistical results, with no clinical interpretation and no generalisation of the findings.

Statistics professionals have long criticized medical research reports for their overuse of p-values and statistical significance (2). To be able to interpret a research finding clinically, both the magnitude of estimated effects and their sampling uncertainty needs to be considered, and this requires interval estimation, i.e. calculation of both effect estimates and their confidence intervals. It is usually impossible, to make the same clinical interpretation from the corresponding p-values or statements about statistical significance, whether these measures have been correctly calculated or not.

The criticism is therefore not about poorly calculated statistics but poorly understood statistical inference.

References

1. International Committee of Medical Journal Editors. Recommendations for the Conduct, Reporting, Editing and Publication of Scholarly Work in Medical Journals [Internet]. [cited 2026 Sep 8]. Available from: https://www.icmje.org/recommendations/

2. Wasserstein, R. L., & Lazar, N. A. (2016). The ASA Statement on p-Values: Context, Process, and Purpose. The American Statistician, 70(2), 129–133. https://doi.org/10.1080/00031305.2016.1154108


 

Comments

Post a Comment

Popular posts from this blog

Randomisation and alternation in clinical trials

  In early clinical trials patients were allocated to treatment and control groups by alternation, i.e. patients were assigned to the treatment group and control group in a strict order. The advantage with this method was that it produced groups with equal size. The two groups would also be fairly compared if the order of the patients included in the trial was random. However, the group allocation for a patient was predictable, which could potentially introduce selection bias into the trial's outcome through differential invitations to participate. The only way to avoid this issue was, of course, to make the group assignment concealed, which necessitated a non-systematic group assignment sequence. D'Arcy Hart (1) describes the development with two trials from the 1940s. The first one investigating the effects of patulin (a mycotoxin) on the common cold in 1943-1944 with strict alternation. The second one, having a randomised group assignment, investigating the effects of strept...
Read more

Conditional and marginal models

  Statistical analyses of related data need to take the relations into account to avoid misleading results. Two main types of statistical models are used, conditional and marginal models (1). A conditional model can be fitted within the framework of generalized linear mixed models (GLMM) and a marginal model using generalized estimating equations (GEE). Analyses based on conditional and marginal models give answers to different questions. While a conditional model can be used to estimate the outcome within a subject or cluster after conditioning on covariates, a marginal model can estimate the average outcome for the population accounting group-specific effects. For example, in a longitudinal study comparing patients receiving diet recommendations (treated) with patients receiving exercise recommendations (controls), uncontrolled blood pressure at repeated visits can be modelled using both a conditional and marginal model. Neither of these two models is automatically more correct ...
Read more

Pretesting normality

One common misunderstanding demonstrated in medical publications is the testing of normal distribution as part of a decision process: If Shapiro-Wilk Test or Kolmogorov-Smirnov Test indicates that a variable has a statistically significant departure from normal distribution, group mean differences are tested using the Mann-Whitney Test instead of Student's t-test. This decision rule may sound rock solid, but it may be a serious mistake. First, the practice treats absence of evidence as evidence of absence. A statistically nonsignificant normality test is simply not evidence of a normal distribution. Second, for a two-sample t-test or a linear statistical model, the relevant distributional issue concerns the difference or model residual conditional on covariates, not whether each observed treatment group's raw outcome values are exactly normal. Third, with small sample sizes, where the normality assumption may be important, the normality test has low power to detect non-normalit...
Read more