Colquhoun.AI
← All guides

Research Guides

Preserve Large Research Identifiers Before Parsing and Joining

· Colquhoun.shop editorial

A research export can be valid JSON and still lose the distinction between two record identifiers when a downstream program reads it. If those identifiers connect citations, review records or source rows, the result can be a false match that looks like a successful import. Test identity preservation before joining or deduplicating the data.

This is a data-handoff worksheet, not a report of a defect in Colquhoun.shop. The example uses invented identifiers and a specific JavaScript parsing path; other parsers can behave differently.

1. Decide whether the field is a measurement or an identity

A count of visits is a quantity. A provider's record identifier is a reference, even when it contains only digits. Adding two identifiers has no useful research meaning. Keep that distinction in the data contract rather than allowing a spreadsheet or parser to infer it from appearance.

RFC 8259's number guidance permits implementations to limit numeric range and precision. It identifies integers from negative (2 raised to 53 minus 1) through positive (2 raised to 53 minus 1) as an interoperability range where implementations agree exactly. A syntactically valid JSON number outside that range is not a guarantee of exact preservation through every consumer.

2. Reproduce a collision before using real records

Use a synthetic fixture containing two number-valued fields: {"a":9007199254740992,"b":9007199254740993}. In a JavaScript test using ordinary JSON.parse and Number values, both parsed fields become 9007199254740992. The source text contains two distinct integers; that parsing path no longer preserves the difference.

Now test a string-valued fixture: {"a":"9007199254740992","b":"9007199254740993"}. Keep the fields as strings throughout the test. They remain distinct without requiring arithmetic on the identifiers. This is an example contract choice, not permission to change an external API's expected types without coordinating its producer and consumers.

The failure can occur before a join starts. Turning an already rounded value into a string afterward only preserves the wrong value. Recover the original source rather than trying to guess the missing digit.

3. Trace the identity through every handoff

Run both fixtures through the real pipeline used for the handoff. A unit test of the initial parser will miss a later conversion in an export helper. Include an identifier with leading zeros if the source contract permits them; that is a separate reason to avoid casually treating identifiers as quantities.

4. Repair the contract, not the observed result

For a system you control, agree on a lossless identifier representation with every consumer. If an external source requires numeric identifiers, use an appropriate lossless parsing path and test it end to end. Do not silently rewrite a provider's payload and then claim compatibility because one sample imported.

Keep arithmetic measurements separately typed and validated. Making every field a string can conceal other errors and does not explain how totals should be calculated.

5. Publish a narrow verification statement

A useful acceptance note names the fixture, the pipeline and the result: “The two synthetic identifiers remain distinct after parsing, joining and reopening the export in the tested consumer.” Retain the raw fixture with the test so another reviewer can reproduce it.

This establishes identity preservation for the tested path. It does not prove that the source records are authentic, that every join is logically correct or that all large-number formats are supported. Those are separate checks, which should not disappear behind a successful JSON parse.

This guide describes the built-in Web Reader and Keyword Density behavior verified for this publication. Available data depends on the page response; these tools do not measure visits or search rankings.