• The multicodec table: A registry mapping integer codes to format names
• Encoding rules: How to construct and parse multicodec prefixes
• Assignment policies: How new codes are allocated

The specification does not have traditional version numbers; instead, the multicodec table evolves through additions of new format codes while maintaining backward compatibility.

Relationship to KERI/ACDC Ecosystem

While multicodec is not a KERI-specific protocol, it shares conceptual similarities with CESR (Composable Event Streaming Representation), KERI's native encoding system. Both protocols address the need for self-describing cryptographic primitives, but they differ in scope and design:

• Multicodec: General-purpose format identification for any binary data
• CESR: Specialized encoding for cryptographic primitives with text-binary composability

KERI implementations may encounter multicodec-encoded data when interoperating with systems using IPFS (InterPlanetary File System) or other multiformats-based protocols, particularly when dealing with Content Identifiers (CIDs).

Protocol Architecture

Layering Model

Multicodec operates as a prefix layer that wraps existing encoding formats. The architecture follows a simple two-component model:

[Multicodec Prefix][Encoded Data]

The prefix provides metadata about the encoded data that follows. This creates a clear separation of concerns:

• Prefix layer: Format identification (multicodec's responsibility)
• Data layer: Actual content in its native encoding (format-specific)

Component Organization

The protocol consists of three logical components:

1. Multicodec Table: A centralized registry mapping codes to formats
2. Encoding Rules: Specification for constructing prefixes
3. Parsing Rules: Specification for extracting format information

Multicodec Table Structure

The multicodec table is the authoritative registry of format codes. Each entry contains:

• Code: An integer identifier (encoded as varint)
• Name: Human-readable format name (e.g., "ed25519-pub", "sha2-256")
• Tag: Category classification (e.g., "key", "multihash", "multiaddr")
• Status: "permanent" or "draft"
• Description: Explanation of the format

Example entries from the table:

Code    Name              Tag         Status
0xed    ed25519-pub       key         permanent
0x12    sha2-256          multihash   permanent
0x1200  sha2-512          multihash   permanent

Data Flow

Encoding Flow

1. Format Selection: Encoder determines the format of data to be encoded
2. Code Lookup: Encoder looks up the multicodec code for that format in the table
3. Varint Encoding: Code is encoded as an unsigned varint
4. Prefix Construction: Varint becomes the prefix
5. Concatenation: Prefix is prepended to the raw data

Decoding Flow

1. Prefix Extraction: Decoder reads the varint prefix from the byte stream
2. Code Lookup: Decoder looks up the code in the multicodec table
3. Format Identification: Decoder determines the format from the table entry
4. Data Parsing: Decoder parses the remaining bytes according to the identified format

State Management

Multicodec is a stateless protocol. Each encoded value is self-contained and can be parsed independently without maintaining session state or context from previous operations. This statelessness provides:

• Simplicity: No state synchronization between encoder and decoder
• Parallelizability: Multiple values can be processed concurrently
• Robustness: No cascading failures from state corruption

Message Formats & Encoding

Prefix Structure

The multicodec prefix consists of a single component: a variable-length integer (varint) encoding the format code.

Varint Encoding

Multicodec uses unsigned varint encoding, a variable-length encoding scheme where:

• Small integers use fewer bytes
• Large integers use more bytes
• The encoding is self-delimiting (parsers can determine length while reading)

Varint encoding rules:

1. Each byte encodes 7 bits of the integer
2. The most significant bit (MSB) of each byte is a continuation flag:
   • 1: More bytes follow
   • 0: This is the last byte
3. Bytes are ordered least significant first (little-endian)

Example: Encoding the value 300

300 in binary: 100101100

Split into 7-bit groups (right to left):
  0000010  0101100

Add continuation bits (1 for first byte, 0 for last):
  10101100  00000010

Hexadecimal: 0xAC 0x02

So the multicodec prefix for code 300 would be the two bytes [0xAC, 0x02].

Complete Encoding Format

A complete multicodec-encoded value has the structure:

[varint(code)][raw_data]

Where:

• varint(code): Variable-length integer encoding of the format code
• raw_data: The actual data in its native format

Example: Ed25519 Public Key

An Ed25519 public key (32 bytes) with multicodec prefix:

Code for ed25519-pub: 0xed (237 decimal)
Varint encoding of 237: 0xED 0x01

Complete encoding:
[0xED, 0x01][32 bytes of public key]

Field Definitions

Format Code Field

• Type: Unsigned integer
• Encoding: Varint (variable length, 1-9 bytes)
• Semantics: Index into the multicodec table identifying the format
• Range: 0 to 2^64-1 (practical limit of varint encoding)

Data Field

• Type: Byte sequence
• Encoding: Format-specific (determined by the code)
• Semantics: The actual content being encoded
• Length: Format-dependent (may be fixed or variable)

Encoding Schemes for Different Data Types

Multicodec is format-agnostic and can prefix any binary data. Common use cases include:

Cryptographic Keys

Public and private keys in various formats:

• Ed25519 public key (code 0xed): 32-byte key
• RSA public key (code 0x1205): DER-encoded key
• Secp256k1 public key (code 0xe7): 33-byte compressed key

Hash Digests

Cryptographic hash outputs (often used in multihash format):

• SHA2-256 (code 0x12): 32-byte digest
• SHA2-512 (code 0x13): 64-byte digest
• Blake3 (code 0x1e): Variable-length digest

Content Identifiers

Multicodec is a component of CID (Content Identifier) used in IPFS:

CID = [multibase][cid-version][multicodec][multihash]

The multicodec component identifies the content type (e.g., "dag-pb" for Protocol Buffers DAG, "raw" for raw binary).

Protocol Mechanics

Encoding Process

The encoding process is straightforward:

Step 1: Determine Format

The encoder must know the format of the data being encoded. This is typically determined by:

• The data source (e.g., key generation algorithm)
• Application logic (e.g., hash function selection)
• User configuration

Step 2: Lookup Code

The encoder looks up the appropriate multicodec code in the table. For example:

• Ed25519 public key → code 0xed
• SHA2-256 digest → code 0x12

Step 3: Encode Varint

The code is encoded as an unsigned varint:

def encode_varint(n):
    bytes = []
    while n > 0x7f:
        bytes.append((n & 0x7f) | 0x80)
        n >>= 7
    bytes.append(n & 0x7f)
    return bytes

Step 4: Concatenate

The varint prefix is prepended to the raw data:

encoded = varint_bytes + raw_data_bytes

Decoding Process

Decoding reverses the encoding process:

Step 1: Extract Varint

The decoder reads bytes from the stream until it encounters a byte with MSB = 0:

def decode_varint(stream):
    n = 0
    shift = 0
    while True:
        byte = stream.read(1)[0]
        n |= (byte & 0x7f) << shift
        if (byte & 0x80) == 0:
            break
        shift += 7
    return n

Step 2: Lookup Format

The decoder looks up the code in the multicodec table to determine the format.

Step 3: Parse Data

The decoder parses the remaining bytes according to the identified format. This may involve:

• Reading a fixed number of bytes (for fixed-length formats)
• Reading a length prefix (for variable-length formats)
• Parsing structured data (for complex formats)

Message Exchange Patterns

Multicodec does not define message exchange patterns—it is a data encoding protocol, not a communication protocol. However, it is commonly used in contexts where self-describing data is transmitted:

• Request-response protocols: Multicodec-encoded keys or identifiers in API requests
• Content-addressed storage: Multicodec-encoded CIDs in IPFS
• Cryptographic protocols: Multicodec-encoded public keys in handshakes

Timing and Ordering

Multicodec has no timing or ordering requirements. Each encoded value is independent and can be processed in any order.

Security Properties

Threat Model

Multicodec's threat model is limited because it is a format identification protocol, not a security protocol. However, certain security considerations apply:

Threats In Scope

1. Format confusion attacks: Attacker provides data with incorrect multicodec prefix
2. Varint overflow attacks: Attacker provides maliciously large varint values
3. Table poisoning: Attacker attempts to register malicious format codes

Threats Out of Scope

1. Cryptographic attacks: Multicodec does not provide confidentiality, integrity, or authenticity
2. Data validation: Multicodec does not validate that data conforms to the claimed format
3. Access control: Multicodec does not restrict who can encode or decode data

Security Guarantees

Multicodec provides format identification, not security guarantees. Specifically:

• No integrity protection: Multicodec does not detect data corruption or tampering
• No authenticity: Multicodec does not verify the source of data
• No confidentiality: Multicodec does not encrypt data

Applications requiring these properties must layer additional protocols on top of multicodec, such as:

• Digital signatures for authenticity (e.g., KERI signatures)
• Message Authentication Codes (MACs) for integrity
• Encryption for confidentiality

Attack Resistance

Format Confusion Resistance

Multicodec provides weak resistance to format confusion attacks. An attacker can prepend an incorrect multicodec prefix to data, causing the decoder to misinterpret the format. However:

• Detection: Format-specific validation can detect mismatches (e.g., an Ed25519 key must be exactly 32 bytes)
• Mitigation: Applications should validate data after decoding

Varint Overflow Resistance

Multicodec implementations must guard against varint overflow attacks where an attacker provides a varint that:

• Encodes an extremely large value (e.g., 2^64)
• Uses many continuation bytes (e.g., 100 bytes)

Mitigations:

• Length limits: Implementations should limit varint length (e.g., maximum 9 bytes for 64-bit values)
• Value limits: Implementations should reject codes outside the valid range

Table Poisoning Resistance

The multicodec table is maintained through a centralized governance process on GitHub. New codes are assigned through pull requests reviewed by maintainers. This provides:

• Moderate resistance: Malicious codes are unlikely to be accepted
• Transparency: All code assignments are publicly visible
• Reversibility: Malicious codes can be removed if discovered

However, the centralized governance model creates a single point of failure. If the GitHub repository or maintainer accounts are compromised, attackers could inject malicious codes.

Interoperability

Dependencies on Other Protocols

Multicodec is a foundational protocol with minimal dependencies:

• Varint encoding: Relies on unsigned varint specification
• Format specifications: Depends on specifications for the formats it identifies (e.g., Ed25519, SHA2-256)

Multicodec does not depend on KERI, ACDC, or other identity protocols.

Integration with KERI/ACDC Ecosystem

While multicodec is not part of the core KERI specification, it may be encountered in KERI implementations when:

Interoperating with IPFS-based Systems

Some KERI implementations may store KELs (Key Event Logs) or ACDCs in IPFS, which uses multicodec-encoded CIDs for content addressing. In this context:

• KEL storage: KEL events stored in IPFS are referenced by CIDs containing multicodec prefixes
• ACDC storage: ACDC credentials stored in IPFS use CIDs for SAIDs (Self-Addressing Identifiers)

Bridging to Non-CESR Systems

KERI uses CESR for encoding cryptographic primitives, which provides similar self-describing properties to multicodec but with additional features (text-binary composability, count codes, etc.). When bridging to systems that use multicodec:

• Key format conversion: CESR-encoded keys may need conversion to multicodec-encoded keys
• Identifier mapping: CESR AIDs may need mapping to multicodec-encoded identifiers

Comparison with CESR

Both multicodec and CESR solve the problem of self-describing cryptographic primitives, but with different design priorities:

When to use multicodec:

• Interoperating with IPFS or multiformats-based systems
• Minimal overhead is critical
• Human readability is not required

When to use CESR:

• Building KERI-based identity systems
• Text-binary composability is required
• Human readability is valuable (debugging, auditing)
• Streaming and pipelining are important

Integration Points

Multicodec can integrate with KERI/ACDC systems at several points:

1. Storage layer: Using IPFS for KEL/ACDC storage
2. Identifier bridges: Mapping between CESR AIDs and multicodec-encoded identifiers
3. Key exchange: Converting keys between CESR and multicodec formats for interoperability

Implementation Considerations

Varint Implementation

Implementing varint encoding/decoding correctly is critical:

Common pitfalls:

• Integer overflow: Failing to check for overflow when decoding large varints
• Infinite loops: Not detecting malformed varints with all continuation bits set
• Endianness confusion: Varint is little-endian, but some implementations assume big-endian

Best practices:

• Use well-tested varint libraries (e.g., protobuf libraries include varint implementations)
• Implement length limits (e.g., maximum 9 bytes for 64-bit values)
• Validate that decoded values are within expected ranges

Table Management

Implementations must decide how to manage the multicodec table:

Options:

1. Embedded table: Include a snapshot of the table in the application
2. Dynamic loading: Fetch the table from GitHub at runtime
3. Hybrid: Embed a base table and update dynamically

Trade-offs:

• Embedded: Fast, reliable, but may become outdated
• Dynamic: Always current, but requires network access and trust in GitHub
• Hybrid: Balanced, but more complex

Format Validation

Multicodec only identifies formats—it does not validate that data conforms to the format. Implementations should:

1. Decode the multicodec prefix to identify the format
2. Validate the data according to format-specific rules
3. Reject invalid data with clear error messages

For example, when decoding an Ed25519 public key:

def decode_ed25519_pubkey(data):
    # Step 1: Decode multicodec prefix
    code = decode_varint(data)
    if code != 0xed:
        raise ValueError(f"Expected Ed25519 (0xed), got {code}")
    
    # Step 2: Extract key bytes
    key_bytes = data[varint_length:]
    
    # Step 3: Validate length
    if len(key_bytes) != 32:
        raise ValueError(f"Ed25519 key must be 32 bytes, got {len(key_bytes)}")
    
    return key_bytes

Performance Considerations

Multicodec has minimal performance impact:

• Encoding overhead: 1-9 bytes per value (typically 1-2 bytes)
• Decoding overhead: One varint decode operation (typically < 10 CPU cycles)
• Memory overhead: Negligible (no state maintained)

For high-performance applications:

• Cache table lookups: Avoid repeated table lookups for the same code
• Optimize varint operations: Use bit manipulation instead of loops
• Batch processing: Process multiple values together to amortize overhead

Error Handling

Implementations should handle errors gracefully:

Error conditions:

1. Unknown code: Code not in the multicodec table
2. Malformed varint: Invalid varint encoding
3. Truncated data: Data shorter than expected for the format
4. Invalid data: Data does not conform to the format

Error handling strategies:

• Fail fast: Reject invalid data immediately
• Provide context: Include the code, format name, and data length in error messages
• Log errors: Record errors for debugging and monitoring

Testing

Thorough testing is essential:

Test cases:

1. Valid encodings: Encode and decode known-good data
2. Invalid codes: Attempt to decode unknown codes
3. Malformed varints: Provide varints with invalid continuation bits
4. Edge cases: Test boundary values (e.g., code 0, maximum code)
5. Format validation: Provide data that doesn't match the claimed format

Test vectors:

The multicodec repository includes test vectors for common formats. Implementations should pass all test vectors to ensure correctness.

Interoperability Testing

When integrating multicodec with other systems:

1. Cross-implementation testing: Verify that data encoded by one implementation can be decoded by another
2. Format compatibility: Ensure that format-specific data is interpreted consistently
3. Table synchronization: Verify that all implementations use the same multicodec table version

Conclusion

Multicodec provides a simple, effective solution to the problem of format identification in binary data. By prepending a compact, self-describing prefix to data, multicodec enables automatic format detection without external metadata or negotiation.

While multicodec is not part of the core KERI specification, it shares conceptual similarities with CESR and may be encountered when KERI systems interoperate with IPFS-based storage or other multiformats-based protocols. Understanding multicodec helps KERI developers navigate the broader ecosystem of self-describing data formats and make informed decisions about when to use multicodec versus CESR.

The protocol's simplicity is both a strength and a limitation. Multicodec excels at format identification but provides no security guarantees. Applications requiring integrity, authenticity, or confidentiality must layer additional protocols on top of multicodec, such as digital signatures, MACs, or encryption. In the KERI ecosystem, these security properties are provided by KERI's key event logs, CESR's cryptographic primitives, and ACDC's verifiable credentials.

Testing Requirements

Thorough testing is essential:

• Valid encodings: Test encode/decode round-trips for all supported formats
• Invalid codes: Verify rejection of unknown codes
• Malformed varints: Test handling of invalid varint encodings
• Edge cases: Test boundary values (code 0, maximum code, maximum varint length)
• Cross-implementation: Verify interoperability with other multicodec implementations
• Test vectors: Use official test vectors from the multicodec repository