# UnambitiousFx > A collection of lightweight, modular .NET libraries designed to simplify web API development — with Native AOT support and zero unnecessary overhead. > > Version: Engram 1.x This file contains all documentation content in a single document following the llmstxt.org standard. ## Engram UnambitiousFx.Engram implements [RFC 8785](https://www.rfc-editor.org/rfc/rfc8785), the JSON Canonicalization Scheme (JCS), for .NET. It turns any JSON value into one deterministic text, and gives a SHA-256 fingerprint of that text. > An *engram* is the stable trace a memory leaves in the brain. Here, it is the trace a JSON document leaves: its > canonical form and its fingerprint. ### Why canonicalize? Two JSON texts can carry the same data and still differ byte for byte: ```json { "currency": "EUR", "amount": 12.50 } {"amount":12.5,"currency":"EUR"} ``` Member order, whitespace, number formatting (`12.50` and `12.5`, `1e3` and `1000`) and string escapes all change the bytes without changing the data. Hashing or signing those bytes directly gives different results for the same data. RFC 8785 defines one form that every conforming implementation produces, in any language. ### Key features - **RFC 8785 canonical form** of a `JsonElement`, a `JsonNode`, a JSON text (string or UTF-8 bytes), or a typed value serialized through its source-generated `JsonTypeInfo`. - **Content fingerprint**: the lowercase hex SHA-256 of the UTF-8 canonical form. - **ECMAScript number formatting**, as RFC 8785 requires: the shortest digits that round-trip. - **Strict**: input with no canonical form throws a `FormatException`. - **Dependency-free and NativeAOT-ready**, for `net8.0`, `net9.0`, `net10.0` and `net11.0`. ### Where next - [Getting Started](getting-started): install the package and canonicalize a document. - [Canonicalization](canonicalization): what the canonical form changes, and what it refuses. - [Fingerprints](fingerprints): idempotency keys, change detection, signatures and ETags. --- ## Getting Started ### Install ```bash dotnet add package UnambitiousFx.Engram ``` Engram targets `net8.0`, `net9.0`, `net10.0` and `net11.0` and has no dependency beyond the BCL. ### Canonicalize Everything is on one static class, `JsonCanonicalizer`, in the `UnambitiousFx.Engram` namespace. ```csharp using UnambitiousFx.Engram; var json = """{ "b": [1, 2.50], "a": "x" }"""; string canonical = JsonCanonicalizer.Canonicalize(json); // {"a":"x","b":[1,2.5]} ``` ### Fingerprint ```csharp string hash = JsonCanonicalizer.Hash(json); // 64 lowercase hexadecimal characters: the SHA-256 of the UTF-8 canonical form ``` Two texts with the same data have the same fingerprint, whatever their formatting. ### Inputs `Canonicalize`, `CanonicalizeToUtf8` and `Hash` all accept the same inputs: | Input | Example | |---|---| | JSON text | `JsonCanonicalizer.Hash("""{"a":1}""")` | | UTF-8 bytes | `JsonCanonicalizer.Hash(body.AsSpan())` | | `JsonElement` | `JsonCanonicalizer.Hash(document.RootElement)` | | `JsonNode?` | `JsonCanonicalizer.Hash(node)`; a `null` node is JSON `null` | | A typed value | `JsonCanonicalizer.Hash(order, AppJsonContext.Default.Order)` | The typed overload serializes through the source-generated `JsonTypeInfo` of a `JsonSerializerContext`, so it works under trimming and NativeAOT: ```csharp public record Order(string Currency, decimal Amount); [JsonSerializable(typeof(Order))] public partial class AppJsonContext : JsonSerializerContext; var fingerprint = JsonCanonicalizer.Hash(new Order("EUR", 12.50m), AppJsonContext.Default.Order); ``` :::note A `string` argument is parsed as JSON text. To canonicalize a JSON *string value*, pass a `JsonNode`: `JsonCanonicalizer.Canonicalize(JsonValue.Create("text"))` gives `"text"`. ::: ### Errors | Exception | When | |---|---| | `JsonException` | The text is not valid JSON. | | `FormatException` | The value has no RFC 8785 canonical form: a duplicate member name, a number outside the double range, a lone surrogate. | Catch them where the JSON enters your system, as you would for any parsing. --- ## Canonicalization `JsonCanonicalizer.Canonicalize` produces the RFC 8785 canonical form of a JSON value. Use `CanonicalizeToUtf8` when you need the bytes (to sign, to hash with another algorithm, to store): they are UTF-8 without a byte order mark. ### What changes | Rule | Input | Canonical | |---|---|---| | No whitespace outside strings | `{ "a" : [ 1 , 2 ] }` | `{"a":[1,2]}` | | Members sorted by UTF-16 code units, recursively | `{"b":1,"a":{"d":1,"c":2}}` | `{"a":{"c":2,"d":1},"b":1}` | | Numbers as ECMAScript prints them | `[4.50, 1E30, 2e-3, 100]` | `[4.5,1e+30,0.002,100]` | | Minimal string escaping | a `/` written `\/` | `/` | | Literals unchanged | `[null, true, false]` | `[null,true,false]` | Array order is data, so it is kept. #### Member order Members are sorted by their names' UTF-16 code units, not by Unicode code points or by culture. A character outside the Basic Multilingual Plane (an emoji, for example) is a surrogate pair starting at `0xD800`, so it sorts *before* a character such as `U+FFFF`. Digits are characters too: `"10"` sorts before `"2"`. #### Numbers JSON numbers become IEEE 754 doubles, then are written as ECMAScript's `Number.prototype.toString` writes them: - the shortest digits that read back as the same double (`333333333.33333329` becomes `333333333.3333333`); - no exponent from `1e-6` up to `1e21`, an exponent outside (`1e+21`, `1e-7`); - no trailing zeros, no `+` on positive numbers, `-0` written `0`. `JsonCanonicalizer.FormatNumber(double)` exposes this formatting on its own. :::caution Precision A canonical form is about JSON data, which RFC 8785 reads as doubles. Integers above 2^53 and decimals with more than about 15 significant digits lose precision. If an exact large number matters (an identifier, an amount in a currency with many decimals), send it as a JSON string. ::: #### Strings Only `"`, `\` and the control characters below `U+0020` are escaped; `\b`, `\t`, `\n`, `\f` and `\r` use their short form, the others use a lowercase six-character `\u` escape. Every other character, including non-ASCII ones, is written as itself. ### What is refused RFC 8785 has no canonical form for some valid JSON. Engram throws a `FormatException` rather than produce a form that another implementation would not reproduce: | Input | Why | |---|---| | `{"a":1,"a":2}` | Duplicate member names: which value would be canonical? | | `[1e400]` | The number does not fit in a double. | | A string or member name with a lone surrogate (an unpaired escape in the `D800`–`DFFF` range) | It cannot be encoded in UTF-8. | `NaN` and infinities cannot appear in JSON text; `FormatNumber` refuses them with the same exception. ### Conformance Every build runs the canonicalizer against: - the number samples of RFC 8785 Appendix B; - the input and output files of the [reference implementation](https://github.com/cyberphone/json-canonicalization), compared byte for byte. --- ## Fingerprints `JsonCanonicalizer.Hash` returns the lowercase hexadecimal SHA-256 of the UTF-8 canonical form: 64 characters. Two JSON values with the same data have the same fingerprint, however they were written. ```csharp JsonCanonicalizer.Hash("""{ "amount": 12.50, "currency": "EUR" }""") == JsonCanonicalizer.Hash("""{"currency":"EUR","amount":12.5}"""); // true ``` ### Idempotency and deduplication Store the fingerprint of what you processed; a second submission of the same data, however it was formatted, has the same key. ```csharp app.MapPost("/imports", async (HttpRequest request, IImportStore store) => { using var body = await JsonDocument.ParseAsync(request.Body); var key = JsonCanonicalizer.Hash(body.RootElement); if (await store.ExistsAsync(key)) { return Results.Ok(new { status = "already imported" }); } // process, then store.Add(key) }); ``` ### Change detection Keep the fingerprint of the last export, not the whole document; compare it with the fingerprint of the new one. ```csharp var current = JsonCanonicalizer.Hash(export, AppJsonContext.Default.Export); if (current != lastExportHash) { // the data changed since the last export } ``` ### ETags A fingerprint of the response body is a strong ETag that does not depend on when the data was written: ```csharp var etag = $"\"{JsonCanonicalizer.Hash(dto, AppJsonContext.Default.Dto)}\""; ``` ### Signatures and HMAC Sign the canonical **bytes**, not the serializer output: a verifier that parses and re-serializes the JSON still checks the same bytes. ```csharp byte[] payload = JsonCanonicalizer.CanonicalizeToUtf8(document.RootElement); byte[] mac = HMACSHA256.HashData(secret, payload); ``` ### What a fingerprint does not do - It identifies **JSON data**, not domain values: `12.5` and `12.50` are the same number, but `"12.50"` is a string and hashes differently. Normalize domain values before serializing if they must compare equal. - SHA-256 detects any change, but a bare fingerprint is not an authentication: anyone can compute it. Use an HMAC or a signature when the sender must be trusted. --- ## Benchmarks The suite in `benchmarks/EngramBenchmark` canonicalizes and fingerprints an object holding an array of `Items` records (a name, a decimal price, two tags, a boolean and an id), from a parsed `JsonElement` and from raw UTF-8 bytes. | Benchmark | What it measures | |---|---| | `Canonicalize` | `Canonicalize(JsonElement)`: the canonical text (baseline) | | `Hash` | `Hash(JsonElement)`: canonical form, UTF-8 encoding and SHA-256 | | `HashFromUtf8` | `Hash(ReadOnlySpan)`: the same, parsing included | ### Results BenchmarkDotNet 0.15.8, `ShortRun` (3 warmups, 3 iterations), .NET 10.0.14, Apple M5 Pro, macOS. | Method | Items | Mean | Allocated | |------------- |------:|-----------:|-----------:| | Canonicalize | 10 | 4.10 µs | 16.85 KB | | Hash | 10 | 4.50 µs | 17.77 KB | | HashFromUtf8 | 10 | 7.50 µs | 17.84 KB | | Canonicalize | 1000 | 446.07 µs | 1627.18 KB | | Hash | 1000 | 479.36 µs | 1699.59 KB | | HashFromUtf8 | 1000 | 773.53 µs | 1699.66 KB | Hashing adds about 10% to canonicalization; parsing the bytes adds about 70%. Time and allocations grow linearly with the document. The 1.0 implementation favours exactness over speed (it builds a `string` and sorts members in lists), so allocations are the first thing to improve. ### Running the benchmarks ```bash dotnet run --project benchmarks/EngramBenchmark -c Release # a quick, less precise run dotnet run --project benchmarks/EngramBenchmark -c Release -- --filter '*' --job short ``` --- ## Using these docs with AI These docs are built to be consumed by AI coding assistants, not just humans. ### 1. llms.txt (zero setup) Every build publishes machine-readable bundles following the [llmstxt.org](https://llmstxt.org) standard: | File | Contents | |------|----------| | [`/llms.txt`](https://unambitiousfx.com/engram/llms.txt) | Index of every doc section with links | | [`/llms-full.txt`](https://unambitiousfx.com/engram/llms-full.txt) | All documentation in one file | ### 2. Context7 MCP [Context7](https://context7.com) indexes these docs and serves them over MCP. Library ID: **`/UnambitiousFx/Engram`**. ### 3. Agent skill shipped in the package The `UnambitiousFx.Engram` package ships an [Agent Skill](https://agentskills.io), `unambitiousfx-engram`: a `SKILL.md` that tells coding agents how to canonicalize and fingerprint JSON correctly. `dotnet restore` places it under `~/.nuget/packages/unambitiousfx.engram//skills/`, but agents only load skills from their own folders. Set `UnambitiousFxSkillsDestination` to have the package copy its skill there on every build: ```xml title="Directory.Build.props" $(MSBuildThisFileDirectory).claude/skills ``` The copy always matches the referenced package version. Every UnambitiousFx package uses the same property, so one setting copies the skills of all of them.