BigQuery introduces built-in identity columns for generated keys

Google Cloud launched identity columns in BigQuery on September 3. The announcement introduces columns that generate sequential 64-bit integer values, allowing the warehouse to take over a piece of key generation that ingestion systems often implement themselves.

Choose who controls the identifier

Google describes two modes. One keeps value generation under BigQuery’s control; the other supplies a generated value by default while permitting explicit values. The feature works with standard data manipulation workflows, including inserts and merges.

For a pipeline owner, the immediate question is whether a table needs an internal surrogate key or must preserve an identifier assigned upstream. Those are different responsibilities. A generated warehouse key can identify a stored row, while an upstream event identifier may be needed to recognize that the same business event arrived twice.

Keep retry semantics explicit

Before replacing application-side key generation, trace a record through an ordinary load, a retry, and a historical backfill. Decide what should remain stable across those paths and how downstream tables refer to the result.

Do not assume that receiving a fresh number makes a repeated ingestion harmless. The pipeline still needs a deliberate rule for matching repeated business records. Likewise, consumers should not infer business chronology from a generated identifier unless the application contract specifically establishes that relationship.

The two generation modes also create a migration choice. Preserving existing identifiers may matter during a staged cutover, but allowing explicit values requires clear ownership of the incoming data.

A useful pilot is a small dimension or staging table with known retry cases. Compare the old and proposed loads, check downstream joins, and document which identifier represents the source event. The feature can remove key-generation machinery once the team has established which responsibilities that machinery was serving.