Research Guides
Review a Box-Plot Outlier Before Removing the Record
A website benchmark contains one unusually slow observation. A chart places it beyond a whisker, and the next draft quietly removes it. The graph may become tidier, but the research has changed. Before excluding a point, separate the chart's labeling rule from evidence that the observation is invalid.
This guide provides a review record for website measurements. It is not a claim that Colquhoun.AI supplies box plots, statistical testing or automatic outlier removal.
1. Ask what the whiskers mean
The NIST box-plot reference describes both a minimum-to-maximum form and a variation using fences. The box spans the lower and upper quartiles, with a median marker. In the fenced variation, the interquartile range is the upper quartile minus the lower quartile; inner fences lie 1.5 times that range beyond the quartiles. Points outside the inner fences receive separate symbols.
Therefore, do not interpret a whisker without checking the chart's convention. Record the software's quartile method and boundary handling as well. A flagged point is a reason to investigate, not, by itself, evidence of a measurement error.
2. Work through a fictional flag
Suppose a fictional page-test report states a lower quartile of 1.2 seconds and an upper quartile of 1.8 seconds. Its interquartile range is 0.6 seconds. Using the stated 1.5-times rule gives an upper inner fence of 2.7 seconds. An observation of 4.6 seconds would sit beyond that fence.
That arithmetic does not explain the observation. It might represent a real slow experience, a different test environment or a faulty record. Calling it a failure of the website, or deleting it as a failure of the instrument, would both require additional evidence.
3. Trace the flagged row before editing
Keep the row identifier, measurement time, tested URL and available test settings. Compare the record with its original capture. Look for a unit mismatch, duplicated import, interrupted run or an explicitly documented protocol deviation.
Do not guess missing settings from neighboring rows. Mark them unknown. If the capture is no longer available, say that the explanation could not be verified. Preserve an untouched copy of the input so another reviewer can reproduce what was initially plotted.
4. Separate three decisions
- Retain: the observation is unusual, but there is no established reason to reject it.
- Correct: source evidence establishes a specific transcription or unit error; record the original and corrected values.
- Exclude from this comparison: a documented rule makes the observation ineligible; retain it in the audit record and explain the rule.
A different device or test condition may call for a separate group rather than deletion. Decide which population the comparison describes before choosing which records belong in it. Do not redefine eligibility merely because a point makes a preferred conclusion less attractive.
5. Copy this outlier-review record
- Dataset version, row identifier and measured unit.
- Chart convention, quartile method and calculated fence.
- Observed value and reason it was flagged.
- Original capture and relevant test conditions.
- Verified explanation, or clearly marked uncertainty.
- Retain, correct or exclude decision with reviewer and rationale.
- Before-and-after sample counts and effect on the reported conclusion.
If removing a record changes the conclusion, disclose that sensitivity rather than presenting only the cleaner chart. Give readers enough information to distinguish a robust finding from one dependent on a disputed exclusion.
6. Close with an evidence-based caption
A useful caption names the metric, population, sample size and whisker convention, then identifies any exclusions. Keep the detailed review record beside the research files. The goal is not to eliminate unusual results. It is to make every change to the evidence understandable and reproducible.