Colquhoun.AI
← All guides

Research Guides

Check Text Normalization Before Merging Keyword Rows

· Colquhoun.shop editorial

Two keyword rows can look identical on screen and still fail an exact text match. A quick cleanup can create the opposite problem: different terms become one row after an overly broad transformation. Before reporting duplicates in a keyword or page-title export, define what “the same text” means for the analysis.

This manual worksheet is for authorized research files. It does not describe a built-in Colquhoun.shop Unicode checker, and it does not establish how a search engine groups queries.

1. Keep the original string beside the comparison key

Save an untouched export. In a working copy, keep the source row ID, original text and a separate comparison field. Do not overwrite the evidence with the cleaned value. Record the export date and the reporting dimensions that distinguish rows, such as page, country or device.

A matching text field alone does not make two whole records duplicates. Two rows can contain the same query while describing different reporting scopes. Resolve that distinction before removing rows or adding their metrics.

2. Separate Unicode normalization from other editing

Unicode defines canonical and compatibility equivalence and four normalization forms. NFC can make canonically equivalent sequences comparable without applying compatibility folding. NFKC additionally folds compatibility distinctions; Unicode cautions against applying it blindly because some distinctions matter to meaning. Unicode Standard Annex #15: Unicode Normalization Forms

For a small test, compare café written with a single U+00E9 character against café written with U+0065 followed by U+0301. An NFC comparison key makes this pair match. Keep that experiment separate from lowercasing, removing accents, trimming spaces or rewriting punctuation. Those are additional decisions, not interchangeable names for normalization.

3. Build a tiny test set before processing the export

Include the accented pair above, an exact duplicate, a genuinely different phrase and any unusual product notation present in the research. Write the expected grouping beside each pair before running the transformation. If the tool cannot show its matching rules, mark the result as uncertain rather than treating its duplicate count as a verified finding.

For an invented product comparison, a model label containing a superscript may need to remain distinguishable from a plain digit. Decide based on the research purpose and source meaning, not on whichever cleanup option produces fewer rows.

4. Review the groups that changed

Count the rows before and after grouping, then inspect every merged group in a small dataset or a documented review sample in a large one. Preserve the original variants under their comparison key. An unexpected merge is a reason to revisit the rule, not evidence that the source text was wrong.

If metrics accompany the text, do not automatically sum them when keys match. First check whether the source rows represent disjoint observations for the same measure and period. Text equivalence is a string-processing result, not a guarantee that measurements can be combined.

5. Publish the rule with the finding

A useful conclusion is “These rows share an NFC comparison key under the documented method,” not “Google considers these keywords identical.” Keep the claim at the level the test actually supports, and retain enough original text for another reviewer to reproduce or challenge the grouping.

This guide describes the built-in Web Reader and Keyword Density behavior verified for this publication. Available data depends on the page response; these tools do not measure visits or search rankings.