Comprehensive Explanation

duplicity-detection

Process Definition

Duplicity detection is KERI's fundamental security mechanism for identifying external inconsistency in key event logs. Unlike internal inconsistency (which makes a log unverifiable), duplicity occurs when two or more versions of a KEL exist for the same AID where each version is internally consistent but mutually incompatible with other versions.

The process accomplishes three critical objectives:

1. Detection of malicious controllers: Identifies when a controller attempts to maintain multiple versions of their event history to deceive different validators
2. Protection against key compromise exploitation: Detects attempts to use compromised keys to create conflicting rotation events
3. Establishment of verifiable trust: Enables validators to make binary trust decisions based on cryptographic evidence rather than reputation

Duplicity detection is used continuously throughout the lifecycle of any KERI identifier, operating as an ambient verification mechanism where any party, anywhere, at any time can detect inconsistent behavior. The key participants include:

• Controllers: The entities managing AIDs who may (maliciously or through compromise) create duplicitous events
• Witnesses: Designated entities that observe and receipt key events, providing the first line of duplicity detection
• Watchers: Independent observers that maintain copies of KELs and compare them for consistency
• Validators: Entities making trust decisions based on duplicity evidence
• Judges and Jurors: Components that evaluate duplicity evidence and determine reconcilability

Process Flow

Step 1: Event Publication and First-Seen Recording

When a controller creates a key event (inception, rotation, or interaction), the event is published to designated witnesses. Each witness that receives the event:

1. Validates the event against KERI protocol rules
2. Checks sequence number to determine if it fits the available tail position in their KEL
3. Records as first-seen if valid and properly sequenced
4. Creates a receipt (signed acknowledgment) of the event
5. Promulgates receipts to other witnesses in the witness pool

The "first seen, always seen, never unseen" principle is critical: once a witness accepts an event for a given sequence number, that event becomes permanent in that witness's view. Any subsequent event claiming the same sequence number will be rejected and flagged as potentially duplicitous.

Step 2: Witness Agreement via KAACE

Witnesses coordinate through KAACE (KERI's Agreement Algorithm for Control Establishment) to reach consensus:

1. Receipt exchange: Each witness shares its receipt with all other witnesses
2. Consistency verification: Witnesses compare the events they've observed
3. Threshold satisfaction: Agreement is achieved when the threshold of accountable duplicity (TOAD) is met
4. State convergence: Witnesses that agree on events form a consistent view of the KEL

If witnesses observe different events for the same sequence number, this immediately signals potential duplicity. The witnesses do not attempt to resolve which version is "correct" - they simply record what they observed and make this evidence available.

Step 3: Watcher Network Monitoring

Watchers operate independently of the controller's designated witnesses, providing distributed duplicity detection:

1. Promiscuous observation: Watchers observe KELs without being designated by controllers
2. Independent recording: Each watcher maintains its own copy of observed KELs
3. Cross-validation: Watchers compare their observations with other watchers
4. Duplicity evidence collection: When inconsistencies are detected, watchers preserve both versions as evidence

The watcher network creates ambient duplicity detection - the property that duplicitous behavior can be detected by anyone, anywhere, at any time. This is KERI's primary defense against eclipse attacks, where an attacker attempts to isolate a validator from the honest network.

Step 4: Validator Evaluation

When a validator needs to establish trust in an AID:

1. KEL retrieval: Obtains the KEL from witnesses or watchers
2. Internal consistency check: Verifies cryptographic integrity (signatures, digests, chaining)
3. Duplicity check: Queries multiple independent sources (witnesses, watchers) for the same KEL
4. Comparison: Identifies any discrepancies between versions
5. Evidence evaluation: If duplicity is detected, examines the conflicting versions

Step 5: Duplicity Classification

When duplicity is detected, it must be classified:

Reconcilable Duplicity: Occurs when one version can be definitively established as authoritative:

• One version has proper witness receipts while the other lacks them
• One version follows proper pre-rotation commitments while the other violates them
• The controller can demonstrate which version is legitimate through recovery procedures

Irreconcilable Duplicity: Occurs when no version can be established as authoritative:

• Multiple versions have equal witness support
• Both versions appear cryptographically valid
• No clear evidence indicates which version the controller intended

For irreconcilable duplicity, the validator typically rejects all versions and refuses to trust the AID.

Step 6: Recovery Process (for Reconcilable Duplicity)

When duplicity is detected but potentially reconcilable:

1. Judge invocation: Judges examine the KELs and DELs (Duplicitous Event Logs)
2. Jury evaluation: Jurors perform detailed duplicity detection on events and receipts
3. Superseding rules application: KERI's recovery protocol determines which version supersedes others
4. State correction: The authoritative KEL is established and propagated
5. Historical preservation: The duplicitous versions remain in DELs as evidence

The recovery process follows superseding rules defined in the KERI specification, which provide deterministic procedures for resolving conflicts.

Technical Requirements

Cryptographic Requirements

Non-repudiable Signatures: Every key event must be signed with non-repudiable digital signatures using the current authoritative keys. This ensures that duplicitous events provide cryptographic proof of misbehavior - the controller cannot deny having signed conflicting versions.

Hash Chaining: Each event must include:

• Previous event digest (p field): Creates backward chaining
• Self-addressing identifier (d field): Binds the event to its content
• Next key digest (n field): Creates forward commitment through pre-rotation

These cryptographic commitments make it impossible to alter past events without detection and prevent certain classes of duplicity attacks.

Witness Signatures: Witness receipts must be cryptographically signed by the witness's own AID, creating a verifiable chain of custody for event observation. The witness cannot later deny having observed a particular event.

Timing Considerations

First-Seen Immutability: The first-seen policy creates temporal ordering constraints:

• Events must be processed in sequence number order
• Once an event is accepted for a sequence position, that position is locked
• Out-of-order events may be placed in escrow but don't affect first-seen determination

Propagation Speed: Witness agreement and watcher propagation typically occur within microseconds across the network. This rapid convergence is critical for:

• Minimizing the window for duplicity attacks
• Enabling near-real-time duplicity detection
• Supporting interactive protocols that depend on current key state

Grace Periods: Some KERI implementations (particularly in vLEI credentials) use grace periods during which duplicity may be detected before final commitment. This provides a buffer for network propagation and witness coordination.

Error Handling

Escrow Mechanisms: Events that cannot be immediately validated are placed in escrow:

• Out-of-order events: Events with sequence numbers beyond the current tail
• Missing prior events: Events that reference unknown previous events
• Unverifiable signatures: Events with signatures that cannot be validated against current key state

Escrowed events are periodically re-evaluated as new information arrives. However, escrowed events do not affect first-seen determination - only events that pass validation and fit the sequence are considered first-seen.

Duplicity Evidence Preservation: When duplicity is detected:

• Both (or all) conflicting versions must be preserved in DELs
• The evidence must be indexed to the corresponding KERL events
• Cryptographic proofs (signatures, receipts) must be maintained
• The evidence must be made available to validators for independent verification

Network Partition Handling: During network partitions:

• Different network segments may observe different events
• Witnesses in each segment record their first-seen events
• When the partition heals, duplicity becomes evident
• Recovery procedures determine which version supersedes

Usage Patterns

Typical Scenarios

Scenario 1: Honest Controller, Functioning Network

In the normal case:

1. Controller creates event and publishes to all witnesses
2. All witnesses observe the same event
3. Witnesses reach agreement via KAACE
4. Watchers observe consistent KEL across the network
5. Validators retrieve consistent KEL from multiple sources
6. No duplicity detected - trust can be established

This is the expected operational mode for legitimate identifiers.

Scenario 2: Malicious Controller Attempting Duplicity

A malicious controller attempts to deceive validators:

1. Controller creates two different rotation events for sequence N
2. Sends version A to witnesses W1, W2
3. Sends version B to witnesses W3, W4
4. Witnesses record their first-seen versions
5. During receipt exchange, witnesses detect inconsistency
6. Watchers observing different witnesses see conflicting versions
7. Duplicity detected - both versions preserved in DELs
8. Validators querying multiple sources receive conflicting KELs
9. Validator rejects the AID as duplicitous

The malicious controller's attempt is cryptographically evident and cannot be hidden.

Scenario 3: Key Compromise with Delayed Detection

An attacker compromises a controller's signing keys:

1. Attacker creates duplicitous interaction events using compromised keys
2. Events may not be immediately detected if sent to different validators
3. Controller performs legitimate key rotation
4. During rotation, validators compare KEL histories
5. Duplicity discovered in pre-rotation interaction events
6. Recovery process initiated using judges and jury
7. Superseding rules determine authoritative version
8. Controller may recover control if rotation keys were not compromised

This scenario demonstrates KERI's live key compromise detection capability.

Scenario 4: Eclipse Attack Mitigation

An attacker attempts to isolate a validator:

1. Attacker controls all network connections to target validator
2. Attacker presents fabricated KEL to isolated validator
3. Validator's watchers maintain independent network connections
4. Watchers observe different KEL from honest network
5. Duplicity detected through watcher comparison
6. Validator recognizes isolation attempt
7. Validator rejects attacker's KEL

The watcher network provides eclipse attack resistance through distributed observation.

Best Practices

For Controllers:

1. Maintain witness diversity: Use witnesses from different organizations, geographic locations, and network providers to prevent coordinated attacks
2. Monitor your own KEL: Regularly query witnesses and watchers to detect unauthorized events
3. Protect rotation keys: Store pre-rotated keys with higher security than signing keys
4. Use appropriate TOAD: Set threshold of accountable duplicity based on security requirements and witness reliability
5. Implement key rotation schedules: Regular rotation limits exposure window for compromised keys

For Validators:

1. Query multiple sources: Always retrieve KELs from multiple independent witnesses and watchers
2. Implement watcher networks: Deploy or use existing watcher infrastructure for ambient duplicity detection
3. Verify witness receipts: Check that events have proper witness signatures meeting TOAD threshold
4. Preserve duplicity evidence: If duplicity is detected, maintain both versions for forensic analysis
5. Apply risk-based policies: Use different validation rigor based on transaction value and risk

For Witness Operators:

1. Maintain high availability: Witnesses must be reliably accessible for event publication and receipt queries
2. Implement first-seen strictly: Never accept conflicting events for the same sequence number
3. Participate in KAACE: Actively exchange receipts with other witnesses in the pool
4. Publish receipts promptly: Minimize latency between event observation and receipt publication
5. Monitor for attacks: Detect and report attempts to publish duplicitous events

For Watcher Operators:

1. Operate independently: Maintain infrastructure separate from controllers and witnesses
2. Observe promiscuously: Monitor KELs for any AIDs of interest without requiring designation
3. Preserve all versions: When duplicity is detected, maintain all conflicting versions as evidence
4. Provide query services: Enable validators to retrieve KELs and check for duplicity
5. Coordinate with other watchers: Share duplicity evidence across the watcher network

Integration Considerations

Application Layer Integration:

Applications using KERI must integrate duplicity detection into their trust decisions:

def validate_credential(credential, issuer_aid):
    # Retrieve KEL from multiple sources
    kel_witness1 = query_witness(witness1, issuer_aid)
    kel_witness2 = query_witness(witness2, issuer_aid)
    kel_watcher = query_watcher(watcher, issuer_aid)
    
    # Check for duplicity
    if not kels_consistent([kel_witness1, kel_witness2, kel_watcher]):
        log_duplicity_evidence(issuer_aid, [kel_witness1, kel_witness2, kel_watcher])
        raise DuplicityDetectedError(f"Duplicity detected for AID {issuer_aid}")
    
    # Verify credential signature against current key state
    key_state = derive_key_state(kel_witness1)
    if not verify_signature(credential, key_state.current_keys):
        raise InvalidSignatureError("Credential signature invalid")
    
    return True

Infrastructure Deployment:

Organizations deploying KERI infrastructure should:

1. Deploy witness pools: Operate witnesses for your own AIDs and potentially offer witnessing services
2. Deploy watcher networks: Maintain watchers for AIDs you interact with frequently
3. Implement DEL storage: Maintain duplicitous event logs for forensic analysis and evidence preservation
4. Integrate with monitoring: Alert on duplicity detection for critical identifiers
5. Establish recovery procedures: Define processes for handling detected duplicity

Performance Optimization:

Duplicity detection can be optimized through:

1. Caching: Cache KELs and key states with appropriate invalidation policies
2. Parallel queries: Query multiple witnesses/watchers concurrently
3. Incremental validation: Only validate new events rather than entire KEL on each check
4. Witness selection: Prioritize witnesses with low latency and high reliability
5. Watcher federation: Use federated watcher networks for geographic distribution

Privacy Considerations:

Duplicity detection has privacy implications:

1. KEL visibility: KELs are typically public or semi-public, revealing identifier activity
2. Watcher observation: Watchers can track which identifiers are being monitored
3. Correlation risks: Duplicity evidence may reveal relationships between identifiers
4. Mitigation strategies: Use separate AIDs for different contexts, implement selective disclosure

Relationship to KERI Security Model

Duplicity detection is the cornerstone of KERI's security architecture. Unlike blockchain systems that prevent duplicity through global consensus and total ordering, KERI makes duplicity evident rather than impossible. This design choice provides several advantages:

Scalability: No global consensus required - each AID's KEL can be validated independently

Portability: Identifiers are not locked to specific ledgers or consensus mechanisms

Flexibility: Different identifiers can use different witness configurations based on security needs

Accountability: Duplicitous behavior provides cryptographic proof of misbehavior

Recovery: Legitimate controllers can recover from key compromise through pre-rotation

The duplicity detection mechanism enables KERI to achieve end-verifiable security without requiring trust in infrastructure. Validators can independently verify identifier integrity by comparing KELs from multiple sources, making KERI suitable for zero-trust architectures where no party is implicitly trusted.

Advanced Topics

Duplicity in Delegated Identifiers

For delegated identifiers, duplicity detection extends to the delegation chain:

1. Delegator duplicity: If the delegating AID exhibits duplicity, all delegated AIDs are affected
2. Delegate duplicity: Delegates can exhibit duplicity independent of their delegator
3. Cooperative validation: Both delegator and delegate KELs must be validated
4. Cascading recovery: Delegator recovery may require delegate recovery

Duplicity in Multi-Sig Groups

For multi-signature group AIDs:

1. Threshold duplicity: Different subsets of signers may create conflicting events
2. Partial signatures: Events with insufficient signatures may appear duplicitous
3. Coordination failures: Network issues may cause apparent duplicity
4. Resolution complexity: Determining authoritative version requires examining all participants

Duplicity in Credential Registries

For TEL (Transaction Event Log) based credential registries:

1. Issuance duplicity: Multiple issuance events for the same credential
2. Revocation duplicity: Conflicting revocation states
3. Registry anchoring: TEL duplicity may indicate KEL duplicity in the issuer's AID
4. Credential validation: Validators must check both KEL and TEL for duplicity

Future Enhancements

Ongoing research and development in duplicity detection includes:

1. Machine learning: Automated pattern recognition for duplicity attacks
2. Reputation systems: Tracking witness and watcher reliability
3. Economic incentives: Reward mechanisms for duplicity detection
4. Privacy-preserving detection: Zero-knowledge proofs for duplicity evidence
5. Cross-protocol integration: Duplicity detection across different identifier systems