Aleatoric and epistemic uncertainty

 Statistical inference is used to evaluate sampling uncertainty in medical research. The two most commonly used uncertainty measures are confidence intervals and p-values. However, it is often useful to distinguish between the uncertainty resulting from random variation (aleatoric uncertainty) and the uncertainty caused by incomplete knowledge (epistemic uncertainty).

A small p-value indicates disagreement between observed data and a tested null hypothesis, but it is not in itself a direct measure of either aleatoric or epistemic uncertainty. P-values do not, by themselves, separate random variability from the uncertainty about whether the statistical model and its assumptions are appropriate.

A confidence interval quantifies sampling uncertainty about an estimate under the assumed statistical model and study design. Its width is influenced by outcome variability and sample size, but it does not generally capture uncertainty from model misspecification, unmeasured confounding, or poor external validity. More participants reduce random imprecision, but they do not by themselves remove systematic bias. For example, the effects of epistemic issues such as selection bias, information bias, and confounding bias can, in an experimental study, be eliminated or reduced by subject randomization, concealed allocation to exposure groups, and blinding of participants, clinicians, and outcome assessors. Observational studies can provide valuable information but are generally unsuitable for confirmatory claims due to their more limited scope, because this limited scope is itself a source of epistemic rather than aleatoric uncertainty.

Confidence intervals and p-values address uncertainty conditional on a specified statistical model; they should not be interpreted as comprehensive measures of uncertainty in a research finding. Random variation contributes to imprecision, whereas incomplete knowledge and design limitations may introduce uncertainty or bias that confidence intervals and p-values do not capture. Sound interpretation of a research finding requires attention to effect size, interval estimates, study design, model assumptions, and plausible sources of systematic error.

Comments

Post a Comment

Popular posts from this blog

Randomisation and alternation in clinical trials

  In early clinical trials patients were allocated to treatment and control groups by alternation, i.e. patients were assigned to the treatment group and control group in a strict order. The advantage with this method was that it produced groups with equal size. The two groups would also be fairly compared if the order of the patients included in the trial was random. However, the group allocation for a patient was predictable, which could potentially introduce selection bias into the trial's outcome through differential invitations to participate. The only way to avoid this issue was, of course, to make the group assignment concealed, which necessitated a non-systematic group assignment sequence. D'Arcy Hart (1) describes the development with two trials from the 1940s. The first one investigating the effects of patulin (a mycotoxin) on the common cold in 1943-1944 with strict alternation. The second one, having a randomised group assignment, investigating the effects of strept...
Read more

Conditional and marginal models

  Statistical analyses of related data need to take the relations into account to avoid misleading results. Two main types of statistical models are used, conditional and marginal models (1). A conditional model can be fitted within the framework of generalized linear mixed models (GLMM) and a marginal model using generalized estimating equations (GEE). Analyses based on conditional and marginal models give answers to different questions. While a conditional model can be used to estimate the outcome within a subject or cluster after conditioning on covariates, a marginal model can estimate the average outcome for the population accounting group-specific effects. For example, in a longitudinal study comparing patients receiving diet recommendations (treated) with patients receiving exercise recommendations (controls), uncontrolled blood pressure at repeated visits can be modelled using both a conditional and marginal model. Neither of these two models is automatically more correct ...
Read more

Pretesting normality

One common misunderstanding demonstrated in medical publications is the testing of normal distribution as part of a decision process: If Shapiro-Wilk Test or Kolmogorov-Smirnov Test indicates that a variable has a statistically significant departure from normal distribution, group mean differences are tested using the Mann-Whitney Test instead of Student's t-test. This decision rule may sound rock solid, but it may be a serious mistake. First, the practice treats absence of evidence as evidence of absence. A statistically nonsignificant normality test is simply not evidence of a normal distribution. Second, for a two-sample t-test or a linear statistical model, the relevant distributional issue concerns the difference or model residual conditional on covariates, not whether each observed treatment group's raw outcome values are exactly normal. Third, with small sample sizes, where the normality assumption may be important, the normality test has low power to detect non-normalit...
Read more