Structural Organization

Group-framing-codes follow CESR's general primitive structure but with specialized semantics:

Text Domain Structure:

[Type Code][Count Value]

• Type Code: One or more Base64 URL-safe characters indicating the group type and count encoding scheme
• Count Value: Encoded integer specifying the number of elements in the group
• Total length must align on 24-bit boundaries (multiples of 4 Base64 characters)

Binary Domain Structure:

[Type Bytes][Count Bytes]

• Type Bytes: Binary representation of the group type
• Count Bytes: Binary-encoded count value
• Total length must align on 24-bit boundaries (multiples of 3 bytes)

The group-framing-code is prepended to the group it describes, making the entire structure self-framing. A parser encountering a group-framing-code knows:

1. The type of group (fixed-size primitives, variable-size primitives, nested groups, etc.)
2. The exact number of elements to extract
3. How to parse the subsequent stream content

Key Components

Group-framing-codes consist of three logical components:

1. Group Type Indicator

The type indicator specifies what kind of elements the group contains:

• Primitive Groups: Groups of cryptographic primitives (signatures, digests, keys)
• Nested Groups: Groups containing other groups (hierarchical composition)
• Mixed Groups: Groups with heterogeneous element types
• Special Groups: Application-specific grouping semantics

The type indicator is encoded in the leading character(s) of the code, using CESR's code table system.

2. Count Encoding

The count value can be encoded in multiple ways depending on the expected group size:

• Small Count Codes: For groups with few elements (typically < 64), the count is encoded in a single character
• Large Count Codes: For groups with many elements, the count uses multiple characters
• Variable-Length Count Codes: For groups where size is not known in advance, special encoding schemes allow variable-length count representation

The count encoding must maintain CESR's 24-bit alignment constraint. For example, if the raw count requires 2 bytes, it must be padded to 3 bytes (24 bits) before Base64 encoding to 4 characters in the text domain.

3. Alignment Padding

To satisfy CESR's composability requirements, group-framing-codes include implicit padding:

• Lead Bytes: Pre-padding bytes added before encoding to ensure 24-bit alignment
• Pad Size Calculation: ps = (3 - (N mod 3)) mod 3 where N is the raw count byte length
• Stable Encoding: The padding ensures the count value occupies the same character positions regardless of domain conversion

Data Format

Field Definitions

Group-framing-codes encode the following logical fields:

Type Field:

• Purpose: Identifies the group type and count encoding scheme
• Size: 1-4 characters (text domain) or 1-3 bytes (binary domain)
• Encoding: CESR code table lookup
• Examples: -A (small count group), -0A (large count group)

Count Field:

• Purpose: Specifies the number of elements in the group
• Size: Variable, depending on count magnitude and encoding scheme
• Encoding: Base64 (text domain) or raw binary (binary domain)
• Range: 0 to implementation-defined maximum (typically 2^24 - 1)

Implicit Pad Field:

• Purpose: Ensures 24-bit alignment for composability
• Size: 0-2 lead bytes
• Encoding: Zero-valued bytes prepended before Base64 conversion
• Visibility: Encoded into type/count characters, not separately visible

Encoding Format

Group-framing-codes use CESR's dual-domain encoding:

Text Domain Encoding:

1. Calculate pad size: ps = (3 - (count_bytes mod 3)) mod 3
2. Prepend ps zero bytes to raw count value
3. Base64 encode the padded count
4. Prepend type code characters
5. Result is multiple of 4 characters (24 bits)

Example for count value 5 (1 byte):

Raw count: 0x05 (1 byte)
Pad size: (3 - (1 mod 3)) mod 3 = 2
Padded: 0x000005 (3 bytes)
Base64: AAAF (4 characters)
With type code '-A': -AAAAF (6 characters total)

Binary Domain Encoding:

1. Calculate pad size (same formula)
2. Prepend pad bytes to raw count
3. Prepend type code bytes
4. Result is multiple of 3 bytes (24 bits)

Example for same count value 5:

Raw count: 0x05 (1 byte)
Pad size: 2
Padded: 0x000005 (3 bytes)
With type code 0xD0: 0xD0000005 (4 bytes total, but type code may be 1 byte)

Raw Domain Representation:

In the raw domain, group-framing-codes are represented as tuples:

(type_code_text, raw_count_integer)

Example:

("-A", 5)

This representation separates metadata (type code) from the actual count value used in parsing logic.

Size and Constraints

Alignment Constraint:

All group-framing-codes must satisfy the 24-bit alignment requirement:

• Text domain: Length must be 4k characters where k is a positive integer
• Binary domain: Length must be 3k bytes where k is a positive integer
• This ensures composability—concatenated groups can be converted between domains without bit-level dependencies

Count Range Constraints:

The maximum count value depends on the encoding scheme:

• Small count codes: Typically 0-63 (6 bits)
• Large count codes: Typically 0-16,777,215 (24 bits)
• Variable-length codes: Implementation-defined maximum

The CESR specification allows for multiple count code types to optimize for different use cases—small groups use compact codes, large groups use extended codes.

Nesting Depth Constraints:

While CESR supports hierarchical composition, implementations may impose practical limits:

• Parser stack depth: Recursive descent parsers have finite stack space
• Performance considerations: Deep nesting increases parsing overhead
• Application semantics: Most use cases require only 2-3 nesting levels

Operations

Creation and Initialization

Encoding a Group-Framing-Code:

The process of creating a group-framing-code involves:

1. Determine Group Type: Select the appropriate type code based on the elements being grouped (e.g., signature group, digest group, nested group)

2. Count Elements: Calculate the number of primitives or sub-groups in the group

3. Calculate Pad Size: Apply the formula ps = (3 - (count_bytes mod 3)) mod 3

4. Encode Count Value:

   • Prepend ps zero bytes to the raw count
   • Base64 encode (text domain) or use raw bytes (binary domain)

5. Prepend Type Code: Add the type code characters/bytes to the encoded count

6. Verify Alignment: Ensure the total length is a multiple of 4 characters (text) or 3 bytes (binary)

Example Creation Process:

# Pseudo-code for creating a group-framing-code
def create_group_code(group_type, element_count):
    # Determine count byte length
    count_bytes = element_count.to_bytes((element_count.bit_length() + 7) // 8, 'big')
    
    # Calculate pad size
    pad_size = (3 - (len(count_bytes) % 3)) % 3
    
    # Prepend pad bytes
    padded_count = b'\x00' * pad_size + count_bytes
    
    # Base64 encode for text domain
    encoded_count = base64.urlsafe_b64encode(padded_count).decode('ascii')
    
    # Prepend type code
    group_code = group_type + encoded_count
    
    return group_code

Updates and Modifications

Group-framing-codes are immutable once created. Modifying a group requires:

1. Parse Existing Group: Extract the current group-framing-code and elements
2. Modify Elements: Add, remove, or change elements in the group
3. Recalculate Count: Determine the new element count
4. Create New Code: Generate a new group-framing-code with the updated count
5. Reconstruct Stream: Concatenate the new code with the modified elements

This immutability is essential for CESR's cryptographic integrity—changing a group-framing-code invalidates any signatures or digests computed over the stream.

Incremental Group Building:

For applications that build groups incrementally:

1. Accumulate Elements: Collect primitives in a buffer
2. Finalize Count: Once all elements are known, calculate the final count
3. Prepend Code: Add the group-framing-code to the beginning of the buffered elements
4. Stream Output: Emit the complete group to the output stream

This approach avoids the need to reserve space for the count or perform stream rewrites.

Verification and Validation

Parsing a Group-Framing-Code:

The Parside parser processes group-framing-codes through these steps:

1. Read Type Code: Extract the leading character(s) to determine the group type
2. Lookup Code Table: Use the type code to find the count encoding scheme
3. Extract Count Value: Read the appropriate number of characters/bytes for the count
4. Decode Count: Base64 decode (text domain) or interpret raw bytes (binary domain)
5. Remove Padding: Strip the lead pad bytes to obtain the raw count integer
6. Validate Alignment: Verify the code length is a multiple of 4 (text) or 3 (binary)

Validation Checks:

Robust parsers perform several validation checks:

• Type Code Validity: Ensure the type code exists in the current CESR version code table
• Count Range: Verify the count is within the valid range for the code type
• Alignment: Confirm 24-bit boundary alignment
• Stream Length: Check that the stream contains at least count elements after the code
• Nesting Consistency: For nested groups, verify that sub-group counts sum correctly

Error Handling:

Parsing failures can occur due to:

• Malformed Codes: Invalid type codes or corrupted count values
• Truncated Streams: Stream ends before all counted elements are present
• Alignment Violations: Codes that don't satisfy 24-bit boundaries
• Version Mismatches: Type codes not defined in the active CESR version

Parsers should implement graceful error handling, potentially using escrow mechanisms to buffer incomplete groups until more data arrives.

Usage Context

In Which Protocols

Group-framing-codes are used throughout the KERI ecosystem:

1. KERI Event Messages

Key Event Logs (KELs) use group-framing-codes to organize:

• Signature Groups: Multiple signatures on an establishment event
• Witness Receipt Groups: Collections of witness signatures
• Seal Groups: Multiple seals anchoring external data

Example KEL structure:

[Event Data]
[Group Code: 3 signatures]
  [Signature 1]
  [Signature 2]
  [Signature 3]
[Group Code: 2 seals]
  [Seal 1]
  [Seal 2]

2. ACDC Credentials

Authentic Chained Data Containers (ACDCs) use group-framing-codes for:

• Attribute Groups: Collections of attributes in compact disclosure
• Edge Groups: Multiple edges in a credential graph
• Signature Groups: Multi-signature proofs on credentials

3. IPEX Protocol

Issuance and Presentation Exchange (IPEX) uses group-framing-codes to bundle:

• Credential Chains: Groups of chained ACDCs in presentation exchanges
• Disclosure Groups: Multiple graduated disclosures in a single exchange

4. Transaction Event Logs (TELs)

TELs use group-framing-codes for:

• Registry Event Groups: Collections of issuance and revocation events
• Backer Groups: Multiple backers for a registry

Lifecycle Management

Creation Phase:

1. Group Assembly: Application logic determines which primitives to group
2. Code Generation: Create the appropriate group-framing-code
3. Stream Construction: Concatenate code + elements
4. Signature/Digest: Compute cryptographic commitments over the complete group

Transmission Phase:

1. Domain Selection: Choose text or binary domain based on transport protocol
2. Serialization: Convert the group to the selected domain
3. Multiplexing: Combine with other groups in a pipelined stream
4. Transmission: Send over network (TCP, UDP, HTTP, etc.)

Reception Phase:

1. Stream Sniffing: Use sniffable property to detect group boundaries
2. Code Parsing: Extract and decode the group-framing-code
3. Element Extraction: Read exactly count elements from the stream
4. Validation: Verify signatures, digests, and structural integrity

Processing Phase:

1. De-multiplexing: Separate groups from the stream
2. Element Iteration: Process each primitive in the group
3. Nested Recursion: For hierarchical groups, recursively process sub-groups
4. Application Logic: Execute protocol-specific operations on the group contents

Related Structures

Group-framing-codes interact with several related CESR structures:

1. Primitive Codes

Individual primitives have their own framing codes that specify type and size. Group-framing-codes organize these primitives into higher-level structures.

2. Count Codes

The terms "group-framing-code" and "count code" are often used interchangeably. Technically, count codes are a subset of framing codes specifically designed for grouping.

3. Version Codes

CESR version codes specify which code table to use when parsing. Group-framing-codes are interpreted according to the active version.

4. Opcode Structures

Opcodes are a proposed extension to CESR that would enable stack-based virtual machine operations on groups. Group-framing-codes would serve as operands for these opcodes.

5. Attachment Structures

In KERI, group-framing-codes are often used in attachment sections of messages, where they organize signatures, receipts, and other supplementary data.

Advanced Topics

Hierarchical Composition

One of the most powerful features of group-framing-codes is support for nested groups:

[Outer Group Code: 2 groups]
  [Inner Group Code: 3 signatures]
    [Signature 1]
    [Signature 2]
    [Signature 3]
  [Inner Group Code: 2 digests]
    [Digest 1]
    [Digest 2]

This hierarchical structure enables:

• Logical Organization: Group related primitives at multiple levels
• Efficient Parsing: Process entire sub-groups atomically
• Flexible Schemas: Adapt group structures to application needs

The Parside parser handles nested groups through recursive descent:

1. Parse outer group code
2. For each element in the outer group:
   • If element is a primitive, extract it
   • If element is a group code, recursively parse the sub-group
3. Return to outer group context after sub-group completion

Pipelining and Multiplexing

Group-framing-codes enable efficient pipelining:

Multiplexing:

[Stream]
  [Group A: 5 elements]
  [Group B: 3 elements]
  [Group C: 7 elements]

A multiplexer can combine multiple groups into a single stream, with each group maintaining its self-framing property.

De-multiplexing:

A de-multiplexer reads the stream sequentially:

1. Parse Group A code → extract 5 elements → route to handler A
2. Parse Group B code → extract 3 elements → route to handler B
3. Parse Group C code → extract 7 elements → route to handler C

This architecture supports high-bandwidth applications where multiple data flows are interleaved in a single stream.

Cold Start Problem Solution

The cold start problem occurs when a parser begins reading a stream mid-transmission without knowing where primitive boundaries are. Group-framing-codes solve this through the sniffable property:

1. Unique Bit Patterns: Group codes have distinctive leading bits
2. Self-Synchronization: Parser can scan for group code patterns
3. Boundary Detection: Once a group code is found, the parser knows exactly how many elements follow
4. Stream Recovery: Parser can synchronize to the next group boundary after errors

This makes CESR streams robust to transmission errors and partial data reception.

Performance Considerations

Parsing Efficiency:

• Single-Pass Parsing: Group-framing-codes enable one-pass stream processing
• No Backtracking: Parser never needs to re-read data
• Predictable Memory: Count values allow pre-allocation of buffers

Encoding Overhead:

• Small Groups: 4-6 characters (text) or 3-4 bytes (binary) overhead
• Large Groups: Overhead increases with count magnitude but remains logarithmic
• Amortization: For large groups, per-element overhead is negligible

Domain Conversion:

• Batch Conversion: Groups can be converted en masse between text and binary
• Composability: No need to parse individual elements during conversion
• Alignment: 24-bit boundaries ensure efficient conversion without bit-shifting

Implementation Guidance

Parser Implementation

Implementing a group-framing-code parser requires:

1. Code Table Management: Load and maintain CESR code tables for the active version
2. Type Dispatch: Map type codes to parsing functions
3. Count Extraction: Implement Base64 decoding and pad removal
4. Element Iteration: Loop exactly count times to extract elements
5. Recursion Handling: Support nested group parsing with stack management
6. Error Recovery: Implement graceful handling of malformed codes

Encoder Implementation

Implementing a group-framing-code encoder requires:

1. Type Selection: Choose appropriate group type for the use case
2. Count Calculation: Determine the number of elements
3. Pad Calculation: Apply the pad size formula
4. Encoding: Base64 encode (text) or raw bytes (binary)
5. Concatenation: Prepend code to elements
6. Validation: Verify alignment and structural correctness

Testing Strategies

Unit Tests:

• Test encoding/decoding round-trips for various count values
• Verify alignment for all pad sizes (0, 1, 2)
• Test boundary conditions (count = 0, count = max)
• Validate error handling for malformed codes

Integration Tests:

• Test nested group parsing
• Verify multiplexing/de-multiplexing
• Test domain conversion (text ↔ binary)
• Validate cold start recovery

Performance Tests:

• Benchmark parsing speed for large groups
• Measure memory usage for nested structures
• Test stream processing throughput

Conclusion

Group-framing-codes are a foundational element of CESR's architecture, enabling self-framing grouping, hierarchical composition, and efficient stream processing. Their design reflects careful consideration of composability constraints, parsing efficiency, and protocol extensibility. Understanding group-framing-codes is essential for implementing KERI protocols and building applications on the CESR encoding framework.

Implementations should impose reasonable nesting depth limits to prevent stack overflow attacks.

Error Handling and Recovery

Robust implementations must handle:

• Malformed codes: Invalid type codes or corrupted count values
• Truncated streams: Stream ends before all counted elements are present
• Alignment violations: Codes that don't satisfy 24-bit boundaries
• Version mismatches: Type codes not defined in the active CESR version

The escrow mechanism can buffer incomplete groups until more data arrives, enabling graceful handling of partial stream reception.

Performance Optimization

Single-Pass Parsing:

Group-framing-codes enable one-pass stream processing without backtracking. Implementations should:

• Pre-allocate buffers based on count values
• Avoid re-reading data
• Process elements sequentially

Batch Domain Conversion:

The composability property allows en masse conversion between text and binary domains:

# Convert entire group without parsing elements
def convert_group_to_binary(text_group):
    # Base64 decode the entire group
    binary_group = base64.urlsafe_b64decode(text_group)
    return binary_group

This is significantly faster than converting individual primitives.

Testing Requirements

Alignment Tests:

Test all three pad sizes (0, 1, 2) to ensure correct alignment:

def test_alignment():
    for count in [1, 2, 3, 4, 5, 100, 1000]:
        code = create_group_code(count)
        assert len(code) % 4 == 0, f"Text code not aligned: {code}"

Round-Trip Tests:

Verify composability through round-trip conversion:

def test_round_trip():
    text_group = create_text_group()
    binary_group = text_to_binary(text_group)
    recovered_text = binary_to_text(binary_group)
    assert text_group == recovered_text

Nested Group Tests:

Test hierarchical composition with multiple nesting levels:

def test_nested_groups():
    outer_group = create_group([
        create_group([primitive1, primitive2]),
        create_group([primitive3, primitive4, primitive5])
    ])
    parsed = parse_group(outer_group)
    assert len(parsed['elements']) == 2
    assert len(parsed['elements'][0]['elements']) == 2
    assert len(parsed['elements'][1]['elements']) == 3

Security Considerations

Count Validation:

Always validate that count values are within reasonable bounds:

MAX_GROUP_SIZE = 1000000  # Implementation-defined limit

if count > MAX_GROUP_SIZE:
    raise ValueError(f"Group count {count} exceeds maximum {MAX_GROUP_SIZE}")

Nesting Depth Limits:

Enforce maximum nesting depth to prevent stack overflow:

MAX_NESTING_DEPTH = 10

def parse_group(stream, position, depth=0):
    if depth > MAX_NESTING_DEPTH:
        raise ValueError(f"Nesting depth {depth} exceeds maximum {MAX_NESTING_DEPTH}")
    # ... parsing logic

Stream Length Validation:

Verify that the stream contains enough data for the counted elements:

expected_length = position + (count * element_size)
if len(stream) < expected_length:
    raise ValueError(f"Stream truncated: expected {expected_length} bytes, got {len(stream)}")

Interoperability

Implementations in different languages must produce identical encodings for the same logical groups. This requires:

• Consistent pad size calculation
• Identical Base64 encoding (URL-safe variant, no padding characters)
• Same type code assignments
• Compatible version handling

Cross-implementation test suites should verify interoperability by exchanging encoded groups and validating parsing results.