DataFingerprinting
Version 1 · Document the DataFingerprinting library: purpose, algorithm, config, integration
DataFingerprinting
A class library that embeds a subtle, deterministic signature into selected numeric and text fields of outbound feeds, so unauthorised redistribution can be traced back to the customer it was delivered to. Structured data can't be watermarked like an image or PDF, so attribution is achieved through tiny, controlled value-level variation (e.g. a sectional time 36.53 → 36.54).
Three guarantees define its behaviour:
- Deterministic — same customer + same source value + same config always produces the same output, so an expected pattern can be re-derived and checked later.
- Subtle — only a small configurable fraction of values change (default ~2%), each by a tiny permitted amount, so the feed stays accurate and usable.
- Safe by default — unknown customer, missing/disabled policy, or a null value returns the original unchanged. The routine never throws for ordinary data.
Algorithm
For every candidate value, five steps run:
- Coordinate — the value's address in the feed:
meeting | race | runner | field | subkey(FingerprintCoordinate). - Canonical string — the customer's secret key is prepended and the coordinate serialised in one central place (
Hashing/CanonicalCoordinate.cs):key|meeting=…|race=…|runner=…|field=…|subkey=…(optional parts omitted). - Hash — SHA-256 of that string; first 8 bytes read little-endian as a
UInt64(Hashing/Sha256FingerprintHasher.cs). - Select —
bucket = hash % 100; the value is altered only whenbucket < Threshold(soThreshold = 2≈ 2% of values). - Apply — if selected, an independent slice of the same hash (
hash / 100) deterministically picks the permitted variation. Selection and choice are therefore independent.
Because only the secret key differs between customers, each customer alters a different small subset of values — a unique constellation that acts as the fingerprint. To trace a leak, re-run these steps for each customer key over the leaked data and see whose pattern matches.
Modes
Modes implement IFingerprintMode (a pure Apply); adding one is a drop-in.
| Mode | What it does | Example |
|---|---|---|
NumericOffsetMode |
Adds a permitted offset, rounds to precision (away-from-zero) | 36.53 → 36.54 |
VariantSelectionMode |
Swaps a text token for a permitted alternate | HANDICAP → HCP |
Configuration
Bound from the Fingerprinting section of appsettings.json. Policy config is non-secret; customer keys are secrets and must come from user secrets / environment / a secret store — never committed.
FingerprintKey— the secret mixed into every hash; never leaves the server.ApiKey— the public value the caller sends inX-Api-Key; resolved to a customer id.- Per policy:
Enabled(master on/off),Threshold(% of values altered),Offsets+Precision(NumericOffset),Alternates(VariantSelection).
Usage and integration
Registered once at startup with builder.Services.AddFingerprinting(builder.Configuration), which binds options and registers the hasher, both modes, and the service as singletons (stateless, singleton-safe).
The public entry point is a single generic method:
T Fingerprint<T>(string? customerId, string fieldType, T value, FingerprintCoordinate coordinate);
In API.DataRouter, ApiKeyCustomerMiddleware resolves the caller from the X-Api-Key header (non-blocking — an absent/unknown key just means no fingerprinting) and stashes the customer id in HttpContext.Items. Feed builders then call the service for each candidate value.
Currently wired only into the RaceResult feed (RaceResultService), for fields race_time_seconds, runner_time_seconds, and race & runner sectional_time. See [[api-datarouter]].
Testing
DataFingerprinting.Tests (xUnit) covers determinism, a golden hash vector (4633320248862227263UL) that locks the algorithm, ~2% distribution over large samples, customer isolation, all safe-default paths, and both modes.
Open items
- Wire the RaceCard and Trial feeds (same pattern as RaceResult).
- Move real
FingerprintKey/ApiKeyvalues out ofappsettings.jsoninto a secret store before deployment. - Per-field policies exist; per-customer policy overrides are not currently supported.