Skip to content

Core Message Bus Architecture

Overview

The Core Message Bus is the central nervous system of AICO, enabling modular, event-driven communication between all system components. It implements a publish-subscribe (pub/sub) pattern that allows modules to communicate without direct dependencies, supporting AICO's core principles of modularity, autonomy, and extensibility.

This architecture document describes the design, implementation, and integration patterns of AICO's central message bus system, which serves as the foundation for inter-module communication and coordination.

Design Principles

The Core Message Bus architecture is built on the following key principles:

1. Loose Coupling

Modules communicate exclusively through the message bus rather than direct method calls, enabling: - Independent development and testing of modules - Ability to replace or upgrade modules without affecting others - Simplified integration of new capabilities

2. Event-Driven Architecture

The system operates on an event-driven paradigm where: - Modules publish events (messages) when state changes occur - Interested modules subscribe to relevant topics - Processing occurs asynchronously and reactively

3. Standardized Communication

All messages follow a consistent envelope structure defined in Protocol Buffers:

message AicoMessage {
  MessageMetadata metadata = 1;
  oneof payload {
    EmotionState emotion_state = 2;
    ConversationMessage conversation_message = 3;
    // Other message types...
  }
}

message MessageMetadata {
  string message_id = 1;       // UUID string
  string timestamp = 2;        // ISO 8601 format
  string source = 3;           // Source module name
  string message_type = 4;     // topic.subtopic format
  string version = 5;          // Schema version
}

4. Topic-Based Routing

Messages are organized in a hierarchical topic structure: - Primary category (e.g., emotion, personality, agency) - Subcategory (e.g., state, expression, goals) - Action/type (e.g., current, update, request)

5. Versioned Message Formats

All message formats are explicitly versioned to enable: - Backward compatibility - Graceful evolution of the system - Support for multiple message format versions simultaneously

Technical Implementation

Message Bus Architecture

The Core Message Bus implements a hybrid broker pattern with the backend service acting as the central message coordinator:

Internal Communication (Backend Modules): - Protocol: ZeroMQ with Protocol Buffers - Transport: inproc:// for same-process modules, ipc:// for cross-process - Pattern: Pub/Sub with topic hierarchy - Host: Backend service runs central ZeroMQ broker on tcp://localhost:5555

External Communication (Subsystems): - Frontend (Flutter): WebSocket for real-time updates, REST API for commands - CLI (Python): ZeroMQ IPC with localhost REST fallback - Studio (React): REST API for admin operations, WebSocket for monitoring - Transport: All external clients connect to backend's API Gateway

Message Bus Technology

The Core Message Bus uses ZeroMQ as the standard internal messaging system:

  • High-performance, asynchronous messaging library
  • Lightweight and embedded within the application
  • Supports multiple messaging patterns (pub/sub, request/reply)
  • Provides reliable message delivery with minimal overhead
  • Proven architecture pattern (similar to ROS robotics framework)

ZeroMQ is chosen for its performance, flexibility, and suitability for all local and internal communication needs in AICO.

Message Format

All messages use Protocol Buffers for serialization, providing: - High-performance binary serialization - Strong typing and schema validation - Cross-language code generation - Efficient size (smaller than JSON) - Backward compatibility through versioning

Message Validation

Messages are validated through Protocol Buffers' built-in validation: - Compile-time type checking - Runtime schema validation - Required fields enforcement - Automatic versioning support

Topic Hierarchy

The message bus uses a hierarchical topic structure that organizes messages by functional domain and purpose:

Core Domains

  • emotion.* - Emotion simulation related messages
  • emotion.state.current - Current emotional state
  • emotion.state.update - Emotional state changes

  • emotion.appraisal.event - Emotional appraisal of events

  • personality.* - Personality simulation related messages

  • personality.state.current - Current personality state
  • personality.expression.communication - Communication style parameters
  • personality.expression.decision - Decision-making parameters

  • personality.expression.emotional - Emotional tendency parameters

  • agency.* - Autonomous agency related messages

  • agency.goals.current - Current agent goals
  • agency.initiative - Proactive engagement initiatives
  • agency.decision.request - Decision-making requests

  • agency.decision.response - Decision outcomes

  • conversation.* - Conversation and dialogue related messages

  • conversation.context - Current conversation context
  • conversation.history - Historical conversation data

  • conversation.intent - Detected user intents

  • memory.* - Memory and learning related messages

  • memory.store - Memory storage requests
  • memory.retrieve - Memory retrieval requests/responses

  • memory.consolidation - Consolidated memory data

  • user.* - User-related messages

  • user.interaction.history - User interaction patterns
  • user.feedback - Explicit and implicit user feedback

  • user.state - Inferred user state

  • llm.* - Large Language Model related messages

  • llm.conversation.events - Conversation events from LLM
  • llm.prompt.conditioning.request - Requests for prompt conditioning
  • llm.prompt.conditioning.response - Prompt conditioning parameters

Cross-Cutting Concerns

  • crisis.* - Crisis detection and handling
  • crisis.detection - Crisis signals and alerts

  • crisis.response - Crisis response coordination

  • expression.* - Cross-modal expression coordination

  • expression.coordination - Coordinated expression directives

  • expression.feedback - Expression effectiveness feedback

  • learning.* - Shared learning coordination

  • learning.coordination - Learning signals and coordination
  • learning.feedback - Learning effectiveness feedback

Module Integration Patterns

Publisher-Subscriber Pattern

Modules interact with the message bus through a consistent pattern:

  1. Initialization:
  2. Modules connect to the message bus on startup
  3. They declare topic subscriptions based on their functionality

  4. They register message handlers for each subscribed topic

  5. Message Publication:

  6. Modules publish messages when their internal state changes
  7. Messages include standardized metadata and domain-specific payloads

  8. Publication is non-blocking and asynchronous

  9. Message Consumption:

  10. Modules receive messages for their subscribed topics
  11. Message handlers process incoming messages
  12. Processing may trigger internal state changes or new message publications

Example: Emotion-Personality Integration

The Emotion Simulation and Personality Simulation modules integrate through the message bus:

  1. Personality Simulation publishes personality.expression.emotional messages
  2. Emotion Simulation subscribes to these messages to adjust emotional tendencies
  3. Emotion Simulation publishes emotion.state.current messages
  4. Personality Simulation subscribes to these messages to inform personality expression

This bidirectional communication happens without direct dependencies between the modules.

Plugin Integration

The Plugin Manager mediates plugin access to the message bus:

  1. Topic Access Control:
  2. Plugins request access to specific topics
  3. Plugin Manager enforces access policies based on plugin permissions

  4. Unauthorized topic access attempts are blocked and logged

  5. Message Validation:

  6. All plugin-originated messages are validated before publication
  7. Malformed messages are rejected to prevent system instability

  8. Message rate limiting prevents denial-of-service attacks

  9. Sandboxed Publication:

  10. Plugins publish through the Plugin Manager proxy
  11. Messages are tagged with plugin identity for traceability
  12. Plugin-specific topic prefixes isolate plugin messages

Security and Privacy Considerations

Message Security

  1. Authentication:
  2. All modules authenticate to the message bus
  3. Unauthorized connections are rejected

  4. Plugin authentication uses separate credentials

  5. Authorization:

  6. Topic-level access control limits which modules can publish/subscribe
  7. Sensitive topics have restricted access
  8. Plugin access is limited to approved topics

Privacy Protection

  1. Data Minimization:
  2. Messages contain only necessary information
  3. Sensitive data is filtered before publication

  4. User identifiers are anonymized where possible

  5. Encryption:

  6. Message payloads containing sensitive data are encrypted
  7. Transport-level encryption protects all message bus traffic
  8. Key rotation policies ensure long-term security

Performance Considerations

Message Throughput

The message bus is designed to handle: - High-frequency emotional state updates - Real-time conversation events - Periodic memory consolidation - Burst traffic during multi-modal coordination

Optimization Strategies

  1. Message Prioritization:
  2. Critical messages (e.g., crisis detection) receive higher priority

  3. Non-time-sensitive messages may be queued during high load

  4. Payload Optimization:

  5. Large payloads may use compression
  6. References instead of full content where appropriate

  7. Selective field inclusion for performance-critical paths

  8. Subscription Optimization:

  9. Fine-grained topic subscriptions to reduce unnecessary message processing
  10. Message filtering at the source when possible
  11. Local caching of frequently accessed message data

Message Persistence

Storage Strategy

Database: libSQL (already integrated and encrypted) - Selective persistence for audit logs, debugging, and cross-device sync - Append-only message log with SQL queryability - JSON metadata support for flexible message attributes

Storage Schema: 

CREATE TABLE events (
    id INTEGER PRIMARY KEY,
    timestamp DATETIME,
    topic TEXT,
    source TEXT,
    message_type TEXT,
    payload BLOB,      -- Protocol Buffer binary
    metadata JSON,     -- Flexible attributes
    INDEX(topic, timestamp)
);

Persistence Policy: - Always: Security events, audit logs, admin actions - Optional: Debug mode message replay, cross-device sync - Never: High-frequency emotion states (unless debugging)

Monitoring and Debugging

The message bus includes facilities for:

  1. Message Tracing:
  2. Correlation IDs link related messages
  3. End-to-end tracing of message flows

  4. Timing metrics for message processing

  5. Traffic Monitoring:

  6. Topic-level message volume metrics
  7. Latency measurements for critical paths

  8. Queue depth monitoring for backpressure detection

  9. Debugging Tools:

  10. Message bus inspector for real-time monitoring
  11. Message replay capabilities for testing
  12. Topic subscription viewer to understand module connectivity

Message Definition and Code Generation

Protocol Buffer Definitions

All message definitions are maintained as Protocol Buffer (.proto) files in the /proto/ directory:

  • Core message envelope: /proto/core/envelope.proto
  • Emotion messages: /proto/emotion/emotion.proto
  • Conversation messages: /proto/conversation/conversation.proto
  • Personality messages: /proto/personality/personality.proto
  • Integration messages: /proto/integration/integration.proto

Code Generation Pipeline

The build process automatically generates language-specific code from these definitions:

  1. Python classes for backend services
  2. Dart classes for Flutter frontend
  3. Additional language bindings as needed

Conclusion

The Core Message Bus architecture is fundamental to AICO's modular, event-driven design. It enables:

  • Modularity: Components can be developed, tested, and deployed independently
  • Extensibility: New modules and plugins can be integrated without modifying existing code
  • Resilience: Failures in one module don't cascade to others
  • Adaptability: The system can evolve through versioned message formats
  • Autonomy: Modules can operate independently based on events
  • Performance: Binary serialization optimizes for speed and size
  • Cross-Platform: Consistent message format across all platforms and devices

By providing a standardized communication backbone, the message bus facilitates the complex interactions required for AICO's proactive agency, emotional presence, personality consistency, and multi-modal embodiment across its federated device network.