Research Guides
A Review Sample Is Not the Whole Customer Base
A competitor page can reveal useful complaints without telling you how common those complaints are among all customers. Before turning a handful of reviews into an advertising brief, record how the sample was selected.
This worksheet is a proposed editorial workflow, not a claim that Colquhoun's research tools are currently available.
Define the question before collecting
Write a narrow question: “Which setup problems appear repeatedly in the recent low-rated reviews we can inspect?” That is different from “Why do most customers dislike this product?” The second claim needs evidence the first sample cannot supply.
Record the product, platform, market, date range, rating filter, sort order, review count inspected and collection date. Record inaccessible pages or missing review text rather than silently treating them as empty.
Check what the page actually exposes
A page may initially show only part of its reviews. Google's documentation describes pagination, load-more controls and infinite scrolling as ways to display subsets of larger collections, including user reviews. It also explains that its crawler generally does not activate buttons to reveal more results. That is a warning to examine the collection method, not proof that every research tool sees the same subset. Google's incremental-loading guidance
Do not equate “20 reviews returned” with “20 reviews exist.” Check the visible total, current filter and whether more results can be reached through permitted methods. Respect access restrictions and service rules.
Count observations, not customers you never observed
Suppose you inspect 20 recent one- and two-star reviews. Six mention confusing setup, three mention billing, and two mention both. A defensible summary is: “Setup appeared in six of the 20 selected low-rated reviews.”
Do not write “30% of customers struggle with setup.” The sample intentionally excluded higher ratings and may exclude other platforms, languages or time periods. Categories may overlap, so their counts need not add up to 20.
Keep a short evidence table with review URL, date, rating, issue category and a restrained paraphrase. Preserve enough context to distinguish a complaint about an old version from one about the current product. Avoid copying unnecessary personal details.
Turn the finding into a testable brief
A repeated complaint can suggest a question for your own product: can a new user complete setup without assistance? Test that before promising an easier experience in an ad. Competitor complaints do not prove your product solves them.
Use this completion checklist:
- Selection rules are stated.
- Missing evidence is visible.
- Counts refer to inspected reviews, not all customers.
- Overlapping categories are disclosed.
- Current-product claims are independently checked.
- The proposed message has evidence behind it.
The result is a focused research brief with honest limits, not a representative customer survey or a guarantee of better advertising performance.