CESR is formally specified in:

• IETF Draft: draft-ssmith-cesr (Composable Event Streaming Representation)
• Trust over IP Foundation: TSWG CESR Specification v0.9 Draft
• Related Specifications:
  • draft-pfeairheller-cesr-proof - CESR Proof Signatures extension
  • draft-ssmith-said - Self-Addressing Identifiers (uses CESR encoding)
  • draft-ssmith-keri - KERI protocol (primary consumer of CESR)

Version History and Evolution

CESR evolved from the need to support KERI's cryptographic event streaming requirements:

• Early Development (2020-2021): Initial design focused on Base64 text encoding with derivation codes
• Binary Domain Addition: Extended to support compact binary representation
• Composability Formalization: Mathematical proof of text-binary composability property
• Code Table Expansion: Multiple code tables for different primitive types and sizes
• CESR-Proof Extension (2022-2023): Added support for transposable signature attachments on self-addressing data
• Current Status (v0.9 Draft): Stable specification with multiple production implementations

Protocol Architecture

Three-Domain Model

CESR operates across three abstract domain representations:

1. Text (T) Domain

• Character Set: URL-safe Base64 (A-Z, a-z, 0-9, -, _) per RFC 4648
• Encoding: 6 bits per character
• Alignment: Primitives must be integer multiples of 4 characters (24 bits)
• Purpose: Human readability, text protocols, debugging
• Key Property: Stable type coding (type information never shares bits with length/value)

2. Binary (B) Domain

• Representation: Raw bytes (8 bits each)
• Alignment: Primitives must be integer multiples of 3 bytes (24 bits)
• Purpose: Compact transmission, efficient storage
• Key Property: Maintains composability with text domain

3. Raw (R) Domain

• Representation: Tuple (text_code, raw_binary)
• Purpose: Actual cryptographic operations
• Usage: Cryptographic libraries work with raw bytes
• Conversion: Requires transformation to/from T or B domains for transmission

24-Bit Alignment Constraint

The composability property requires strict 24-bit boundary alignment:

• 24 bits is the least common multiple of 6 (Base64 character width) and 8 (byte width)
• Text domain: 4 characters × 6 bits = 24 bits
• Binary domain: 3 bytes × 8 bits = 24 bits
• Consequence: Without alignment, conversions would create bit-level dependencies between adjacent primitives

Self-Framing Architecture

Every CESR primitive is self-framing - it contains all information needed to parse it without external delimiters:

1. Type information: Encoded in derivation code prefix
2. Size information: Either fixed (implicit in code) or variable (explicit in code)
3. Value: The actual cryptographic material or data

Example Text Domain Primitive:

E<43 Base64 characters>

• E = derivation code (Blake3-256 digest)
• Implies 32-byte raw value
• Total: 44 characters (aligned on 24-bit boundary)

Code Table Organization

CESR uses multiple code tables optimized for different requirements:

Fixed-Size Tables

Small Fixed Raw Size (1-character codes):

• Pad size 1 primitives
• Examples: A (random seed), B (Ed25519 public key), E (Blake3-256 digest)

Large Fixed Raw Size (2-character codes):

• Pad size 0 primitives
• Examples: 0B (Ed25519 signature), 0D (Blake3-512 digest)

Variable-Size Tables

Small Variable Raw Size (2-character codes):

• Length encoded in second character
• Supports up to 4095 quadlets (text) or triplets (binary)

Large Variable Raw Size (4-character codes):

• Length encoded in characters 2-4
• Supports up to 16,777,215 quadlets/triplets

Count Codes (Group Framing)

Purpose: Enable grouping of primitives for pipelining

Types:

• Primitive count codes: Specify number of primitives in group
• Byte/character count codes: Specify total size of group
• Nested group codes: Support hierarchical composition

Example:

-AAB<primitive1><primitive2>

• -AAB = count code indicating 2 primitives follow
• Enables extraction of group without parsing individual primitives

Hierarchical Composition

CESR supports hierarchical composition through:

1. Concatenation: Primitives can be concatenated in any order
2. Grouping: Count codes create logical groups
3. Nesting: Groups can contain other groups
4. Interleaving: CESR streams can interleave with JSON, CBOR, MGPK

Message Formats & Encoding

Primitive Structure

All CESR primitives follow this general structure:

[Derivation Code][Value]

Derivation Code Components:

• Type selector: Identifies primitive type
• Size information: Fixed (implicit) or variable (explicit)
• Pad size indicator: Determines alignment strategy

Text Domain Encoding

Pre-Padding Approach

CESR uses leading pad bytes (not trailing = characters) to achieve 24-bit alignment:

Algorithm:

1. Calculate pad size: ps = (3 - (N mod 3)) mod 3 where N = raw byte length
2. Prepend ps zero bytes to raw value
3. Convert to Base64
4. Replace first characters with derivation code

Example (1-byte value):

• Raw: 0x42
• Pad size: 2
• Pre-padded: 0x0000 42
• Base64: AABC
• With code M: MABC

Derivation Code Formats

1-Character Codes (pad size 1):

[Code][43 Base64 chars]

Example: BDKrJxkcR9m5u1xs33F5pxRJP6T7hJEbhpHrUtlDdhh0

2-Character Codes (pad size 0):

[Code1][Code2][42 Base64 chars]

Example: 0BDKrJxkcR9m5u1xs33F5pxRJP6T7hJEbhpHrUtlDdhh0

Variable-Length Codes:

[Code][Length][Value]

Example: 4B##<value> where ## encodes length in Base64

Binary Domain Encoding

Binary encoding follows similar principles but operates on bytes:

Structure:

[Binary Code][Raw Bytes]

Code Mapping:

• Text code E → Binary code 0x0C
• Text code 0B → Binary code 0x34 0x00

Alignment: Binary codes ensure 3-byte alignment through lead byte padding

Supported Primitive Types

Cryptographic Material

Seeds and Salts:

• A: 128-bit random seed
• 0A: 256-bit random salt

Public Keys:

• B: Ed25519 non-transferable prefix
• D: Ed25519 public verification key
• C: X25519 public encryption key
• 1AAA: ECDSA secp256k1 public key

Digests:

• E: Blake3-256 (32 bytes)
• F: Blake2b-256 (32 bytes)
• G: Blake2s-256 (32 bytes)
• H: SHA3-256 (32 bytes)
• I: SHA2-256 (32 bytes)
• 0D: Blake3-512 (64 bytes)

Signatures:

• 0B: Ed25519 signature (64 bytes)
• 0C: ECDSA secp256k1 signature
• Indexed signatures with dual-indexed codes

Data Types

Numbers:

• M: Short number (2 bytes)
• N: Big number (8 bytes)
• 0H: Long number (4 bytes)

Strings:

• 4B##: Variable-length byte string (small)
• 5B##: Variable-length byte string (lead size 1)
• 7AAB####: Variable-length byte string (big)

Timestamps and UUIDs:

• Encoded as variable-length strings
• ISO 8601 format for timestamps

Interleaved Serializations

CESR streams can interleave with other serialization formats:

Supported Formats:

• JSON (RFC 8259)
• CBOR (RFC 8949)
• MessagePack (MGPK)

Detection Mechanism:

• CESR uses version count codes to mark boundaries
• Sniffer component detects format transitions
• Regex matching locates version strings in non-CESR formats

Example Mixed Stream:

[CESR primitives][JSON object][CESR primitives][CBOR data]

Protocol Mechanics

Stream Processing Model

Cold Start and Re-synchronization

CESR provides cold start stream parsing capabilities:

Problem: After reboot or error, parser needs framing information

Solution:

1. Version count codes at stream boundaries
2. Group count codes enable skipping to next boundary
3. No buffer flushing required for recovery

Re-synchronization Process:

1. Detect format using sniffer
2. If non-CESR: extract length from version string, skip to boundary
3. If CESR: locate next count code, resume parsing

Pipelining and Multiplexing

Pipelining enables parallel processing:

1. Multiplexing: Combine multiple primitives into complex streams
2. De-multiplexing: Extract primitives from streams
3. Group extraction: Pull entire groups without parsing contents

Performance Benefits:

• Parallel processing: Multiple cores can process different groups
• Early rejection: Invalid signatures can drop messages without full parsing
• Efficient routing: Group codes enable stream routing without deep inspection

Composability Transformations

CESR defines six transformations between domains:

T(B) - Binary to Text:

1. Read binary code
2. Determine primitive type and length
3. Extract raw bytes
4. Apply Base64 encoding with appropriate padding
5. Prepend text derivation code

B(T) - Text to Binary:

1. Read text code
2. Determine primitive type and length
3. Extract Base64 characters
4. Decode to bytes
5. Remove lead padding
6. Prepend binary code

Round-Trip Property:

T(B(T(primitive))) = T(primitive)
B(T(B(primitive))) = B(primitive)

Attachment Mechanisms

CESR supports cryptographic attachments without wrapper envelopes:

Signature Attachments:

[Message Body][Count Code][Signatures]

Receipt Attachments:

[Key Event][Count Code][Witness Receipts]

CESR-Proof Attachments (from draft-pfeairheller-cesr-proof):

[SAD][Count Code][Path-Signature Pairs]

Security Properties

Cryptographic Agility

CESR provides algorithm agility through derivation codes:

Benefits:

1. Multiple algorithms: Support Ed25519, ECDSA, Ed448 simultaneously
2. Graceful migration: Add new algorithms without breaking existing code
3. Post-quantum readiness: Can incorporate quantum-resistant algorithms
4. Algorithm identification: Every primitive self-identifies its algorithm

Security Consideration: Derivation codes must be carefully managed to prevent downgrade attacks

Integrity Properties

Self-Framing Integrity:

• Malformed primitives are immediately detectable
• Parsing errors don't propagate to adjacent primitives
• Stream corruption is localized

Composability Integrity:

• Round-trip conversion preserves all information
• No bit-level dependencies between primitives
• Group boundaries are cryptographically verifiable

Threat Model

Protected Against:

1. Primitive injection: Self-framing prevents injection between primitives
2. Length extension: Fixed/explicit lengths prevent extension attacks
3. Type confusion: Derivation codes prevent type misinterpretation
4. Replay attacks: (Handled by higher-level protocols using CESR)

Not Protected Against (requires higher-level protocols):

1. Signature verification: CESR encodes signatures but doesn't verify them
2. Key management: CESR represents keys but doesn't manage them
3. Replay protection: Requires timestamp/nonce mechanisms in protocols

Attack Resistance

Malformed Stream Attacks:

• Detection: Invalid codes immediately detected
• Recovery: Cold start re-synchronization without data loss
• Isolation: Errors don't cascade to valid primitives

Resource Exhaustion:

• Length limits: Variable-length primitives have maximum sizes
• Group limits: Count codes prevent unbounded group sizes
• Early rejection: Invalid primitives rejected before full processing

Interoperability

KERI Protocol Integration

CESR is the native encoding for KERI:

Key Event Messages:

{
  "v": "KERI10JSON00011c_",
  "t": "icp",
  "d": "EH7Oq9oxCgYa-nnNLvwhp9sFZpALILlRYyB-6n4WDi7w",
  "i": "EH7Oq9oxCgYa-nnNLvwhp9sFZpALILlRYyB-6n4WDi7w",
  "s": "0",
  "kt": "1",
  "k": ["DSuhyBcPZEZLK-fcw5tzHn2N46wRCG_ZOoeKtWTOunRA"],
  "n": ["EPYuj8mq_PYYsoBKkzX1kxSPGYBWaIya3slgCOyOtlqU"]
}

Signature Attachments (CESR-encoded):

-AABAA1o61PgMhwhi89FES_vwYeSbbWnVuELV_jv7Yv6f5zNiOLnj1ZZa4MW2c6Z_vZDt55QUnLaiaikE-d_ApsFEgCA

ACDC Integration

ACDCs use CESR for:

SAIDs (Self-Addressing Identifiers):

• All ACDC SAIDs are CESR-encoded digests
• Example: EAdXt3gIXOf2BBWNHdSXCJnFJL5OuQPyM5K0neuniccM

Compact Disclosure:

• Replace full field maps with SAIDs
• Maintains verifiability while reducing size

CESR-Proof Signatures:

• Transposable signature attachments
• Support nested partial signatures
• Enable selective disclosure

SAID Protocol Dependency

SAID (Self-Addressing Identifier) protocol uses CESR:

SAID Generation:

1. Replace SAID field with dummy characters (#)
2. Compute digest
3. CESR-encode digest
4. Replace dummy with CESR-encoded SAID

Verification:

1. Extract SAID
2. Replace with dummy
3. Recompute digest
4. Compare CESR-encoded digests

OOBI Protocol Integration

OOBI (Out-Of-Band Introduction) uses CESR for:

AID Encoding: All AIDs in OOBIs are CESR-encoded URL Construction: CESR primitives embedded in URLs Discovery: CESR enables compact AID representation

Implementation Considerations

Code Table Management

Challenge: Multiple code tables with version evolution

Approach:

1. Version count codes: Specify active code table version
2. Default tables: Parsers start with default version
3. Dynamic loading: Load new tables when version codes encountered
4. Backward compatibility: Support multiple versions simultaneously

Performance Optimization

Text Domain:

• Lookup tables: Pre-compute code-to-type mappings
• String operations: Minimize string copying
• Batch conversion: Convert groups en masse when possible

Binary Domain:

• Zero-copy parsing: Parse directly from byte buffers
• Alignment: Ensure proper memory alignment for performance
• SIMD: Use SIMD instructions for Base64 encoding/decoding

Stream Processing:

• Buffering: Minimize buffer allocations
• Pipelining: Process groups in parallel
• Early rejection: Validate codes before extracting values

Common Implementation Challenges

24-Bit Alignment:

• Issue: Ensuring all primitives align on 24-bit boundaries
• Solution: Careful pad size calculation and validation

Code Table Synchronization:

• Issue: Sender and receiver must use same code tables
• Solution: Version count codes and explicit version negotiation

Interleaved Format Detection:

• Issue: Distinguishing CESR from JSON/CBOR/MGPK
• Solution: Sniffer component with format-specific detection logic

Error Recovery:

• Issue: Recovering from malformed streams
• Solution: Cold start re-synchronization using count codes

Testing Strategies

Round-Trip Testing:

for each primitive:
  text = T(R(primitive))
  binary = B(text)
  recovered = R(B(binary))
  assert primitive == recovered

Composability Testing:

primitives = [p1, p2, p3]
text_concat = cat([T(p) for p in primitives])
binary_group = B(text_concat)
recovered_text = T(binary_group)
assert text_concat == recovered_text

Interoperability Testing:

• Test against reference implementations
• Validate against test vectors
• Cross-language compatibility checks

Library Design Patterns

Primitive Classes:

• Diger: Digest primitive with verification
• Verfer: Public key primitive with signature verification
• Signer: Private key primitive with signing
• Siger: Indexed signature primitive
• Cigar: Non-indexed signature primitive
• Salter: Seed/salt primitive with key generation

Common Interface:

.qb64()   # Qualified Base64 text
.qb64b()  # Qualified Base64 bytes
.qb2()    # Qualified binary
.code()   # Derivation code
.raw()    # Raw bytes

Parser Architecture:

• Sniffer: Format detection
• Parside: Stream parsing with count codes
• Cesride: Primitive parsing
• Streamer: Stream composition

Platform-Specific Considerations

JavaScript/TypeScript:

• Use Buffer for binary operations
• Handle UTF-8 encoding carefully
• Consider WebAssembly for performance

Python:

• Use bytes for binary domain
• Leverage base64 standard library
• Consider Cython for hot paths

Rust:

• Zero-copy parsing with byte slices
• Strong typing for primitive variants
• Efficient memory management

Go:

• Use byte slices for efficiency
• Leverage standard encoding/base64
• Consider goroutines for parallel processing

Library Architecture Recommendations

Primitive Classes: Implement typed classes for each primitive category (Diger, Verfer, Signer, etc.) with common interface:

• .qb64() - Qualified Base64 text
• .qb2() - Qualified binary
• .code() - Derivation code
• .raw() - Raw bytes

Parser Components:

• Sniffer: Format detection for interleaved streams
• Parside: Stream parsing with count code support
• Cesride: Individual primitive parsing
• Streamer: Stream composition and attachment handling