Colquhoun.AI
← All guides

Research Guides

Preserve Test Order Before Averaging Website Measurements

· Colquhoun.shop editorial

A website test summary reports one average for twenty measurements. That number can hide an important detail: perhaps the first ten runs behaved differently from the last ten. Preserve collection order before deciding that the observations describe one stable process.

This worksheet uses a run-sequence view to review repeated website measurements. It is an exploratory method, not a Colquhoun.shop monitoring feature or a substitute for a controlled performance experiment.

1. Keep the run index with the value

NIST describes a run-sequence plot as a measured value plotted against its observation index. It can help reveal shifts in location or variation and unusual observations. NIST: Run-Sequence Plot

Assign an identifier as each measurement is collected. Keep the actual timestamp separately. Run number 10 means the tenth observation, not necessarily ten minutes after the first. Uneven intervals should not disappear behind equally spaced index labels.

2. Define what counts as a comparable run

Record the metric, page, browser, device and test steps. Include relevant conditions such as cache state and whether the same interaction was performed. A sequence assembled from different tasks needs those differences explained before its shape can be interpreted.

Retain failed or missing attempts in the collection record. They may not supply a numeric point, but removing their identifiers can make the sequence look uninterrupted. Distinguish a missing observation from a measured zero.

3. Investigate a shift without naming its cause too soon

In a fictional test, ten initial runs cluster near 400 milliseconds and ten later runs cluster near 800 milliseconds. The overall average is near 600 milliseconds, but that summary alone does not communicate the two different portions of the sequence.

The reviewer checks the run notes and finds that the test setup changed between runs ten and eleven. That is a reason to investigate comparability, not immediate proof that the setup change caused the difference.

The report preserves both portions, documents the change and proposes a controlled follow-up with consistent settings. It does not erase the earlier runs or describe the combined average as the site's universal performance.

4. Do not sort away the question

A value-sorted view can help locate the largest measurements, but it answers a different question. Once sorted by magnitude, the display no longer shows the original collection sequence.

Keep a separate exploratory view if sorting is useful. Retain the run identifiers so a reviewer can return from an unusual value to its original position and notes. A convenient spreadsheet sort should not silently change what the horizontal axis represents.

5. Copy this sequence-review card

Annotate known events using the collection record, not a story invented after seeing the chart. Leave unexplained changes explicitly unexplained until more evidence is available.

A useful conclusion states what the sequence shows, which conditions were recorded and what remains uncertain. The chart helps decide whether a single summary is adequate; it does not independently establish a deployment effect, visitor experience or search-ranking outcome.

This guide describes the built-in Web Reader and Keyword Density behavior verified for this publication. Available data depends on the page response; these tools do not measure visits or search rankings.