Colquhoun.AI
← All guides

Research Guides

Name the Percentile Method Before Comparing Benchmarks

· Colquhoun.shop editorial

Two analysts calculate the 75th percentile from the same website measurements and report different answers. Before assuming that one copied the wrong data, check the percentile method. A label such as P75 does not fully describe the calculation used to produce a sample estimate.

This guide provides a percentile-reconciliation record for website research. It is not a recommendation to replace a platform's documented metric definition, and it does not claim that Colquhoun.AI offers statistical testing.

1. Record the estimator, not only the percentile

The NIST percentile reference explains that interpolation methods are not universally standardized. It describes several methods, including R6, which uses the ordered position p(N+1), and R7, which uses 1+p(N-1), with interpolation between observations where needed.

For a reproducible report, retain the percentile requested, the method used and the input population. A function name without its relevant options may be insufficient when a tool supports multiple estimators.

2. Reconcile a small fictional sample

Consider five fictional page measurements, already ordered: 1, 2, 3, 4 and 8 seconds. For the 75th percentile, p is 0.75 and N is 5.

Using the R6 position gives 0.75 multiplied by 6, or 4.5. Interpolating halfway between the fourth value, 4, and fifth value, 8, gives 6 seconds. Using the R7 position gives 1 plus 0.75 multiplied by 4, or 4, so the estimate is 4 seconds.

Neither calculation changed the five observations. The difference came from the stated method. This tiny sample is an arithmetic illustration, not a benchmark for any real site or a basis for a performance claim.

3. Verify the inputs before blaming the formula

Confirm that both calculations use the same eligible records, measurement unit and time window. Check missing-value handling and whether repeated measurements are retained, aggregated or filtered. Keep those decisions visible rather than hiding them behind a final exported number.

If the record sets differ, reconcile that first. Two equal row counts do not prove identical inputs. Compare stable record identifiers or the retained input files, using only data the reviewer is authorized to inspect.

4. Follow the metric's existing contract

If a platform publishes a documented percentile metric, preserve its definition when quoting it. A separately calculated value from a local export may describe a different sample or estimator. Label it as a separate analysis instead of presenting it as the platform's own result.

For an internal study, agree on a method before inspecting which choice produces the more attractive conclusion. If the method changes between reports, recalculate the comparable baseline where possible and disclose the change. Do not quietly splice unlike calculations into one trend.

5. Copy this percentile-reconciliation card

Keep the unrounded calculation for verification and round only the displayed result according to the report's stated convention. Display formatting should not obscure whether the underlying methods actually agree.

6. State what the reconciliation establishes

A successful check explains how the reported sample percentile was obtained. It does not establish that the sample represents all visitors, that a performance change was caused by a deployment or that a small-sample tail estimate is stable.

The useful outcome is an auditable calculation contract. Another reviewer should be able to reproduce the result and understand why a differently configured tool might produce another number.

This guide describes the built-in Web Reader and Keyword Density behavior verified for this publication. Available data depends on the page response; these tools do not measure visits or search rankings.