Shared Experiences Framework

Overview
Core Assumptions
Architecture
- Component Diagram
- Component Breakdown
Experience Lifecycle
Event System
State Management
Threading Guidelines
Creating New Experiences
Firestore Layout

Overview

Pajama's Shared Experiences framework enables real-time collaborative activities during video calls. The framework supports synchronized state management, multi-channel event distribution, and extensible experience types. Currently implements co-reading, with future support planned for co-drawing, co-puzzles, and other collaborative activities.

The framework uses a Redux-like pattern with session actors for state management, multi-channel event synchronization, and a factory pattern for experience creation.

Core Assumptions

Call-dependent: Shared experiences only occur during active video calls
Multi-channel sync: Real-time events via PubSub, persistent state via Firestore, with local channels for testing
Single active experience: One experience active per call at a time
Sequential experiences: Multiple experiences can run in sequence during one call
Multi-participant support: Supports 1:1 and group calls
Owner/follower model: One participant owns the experience, others follow
Resumable sessions: Experiences can be resumed across calls using snapshots
Event ordering: Sequence numbers ensure consistent state across participants
Pluggable architecture: Video system and event channels are abstracted

Architecture

Component Diagram

┌─────────────────────────┐
│   ExperienceManager     │ ← Orchestrates lifecycle, creates components
│   (MainActor)           │
└────────────┬────────────┘
             │
┌────────────▼────────────┐
│ ExperienceInitializer   │ ← Factory for creating experience components
└────────────┬────────────┘
             │
    ┌────────▼────────┐         ┌─────────────────────┐
    │ ExperienceEvent │         │ ExperienceSnapshot  │
    │      Hub        │◄────────┤     Manager         │
    └────────┬────────┘         └─────────────────────┘
             │
    ┌────────▼────────┐
    │ Event Channels  │
    ├─────────────────┤
    │ • Firestore     │ ← Persistent state & event log
    │ • PubSub        │ ← Real-time synchronization  
    │ • Local         │ ← Testing & debugging
    └─────────────────┘

┌─────────────────────────┐         ┌─────────────────────────┐
│  Experience Definition  │         │   Session Actor         │
│  (Factory Pattern)      │────────▶│   (Redux Pattern)       │
│                         │         │                         │
│ • CoReadingDefinition   │         │ • CoReadingSessionActor │
│ • CoDrawingDefinition   │         │ • CoDrawingSessionActor │
│ • CoPuzzlesDefinition   │         │ • CoPuzzlesSessionActor │
└─────────────────────────┘         └────────────┬────────────┘
                                                  │
                                    ┌─────────────▼────────────┐
                                    │    Experience ViewModel  │
                                    │    (UI Coordination)     │
                                    │                          │
                                    │ • CoReadingViewModel     │
                                    │ • CoDrawingViewModel     │
                                    │ • CoPuzzlesViewModel     │
                                    └──────────────────────────┘

Component Breakdown

1. `ExperienceManager`

Central orchestrator for experience lifecycle

Manages experience activation/deactivation across the call
Creates experience components via ExperienceInitializer
Coordinates between RTC system and active experiences
Handles participant join/leave events
Publishes experience state (isFullScreenMode, isOwner, lifecycleState)
Discovers and joins existing experiences when entering calls

// Activate an experience
experienceManager.activateExperience(withId: "co_reading")

// Get UI for active experience
let experienceView = experienceManager.createContentView()

// End current experience (owner only)
experienceManager.endCurrentExperience()

2. `ExperienceDefinition`

Factory protocol for creating experience components

Each experience type implements this protocol to define how to create its session actor and view model:

protocol ExperienceDefinition {
    associatedtype Actor: ExperienceSessionActor
    associatedtype ViewModel: ExperienceViewModel
    static var id: String { get }
    
    func createSessionActor(context: ExperienceInitializationContext) -> Result<Actor, ExperienceError>
    func createViewModel(sessionActor: Actor, rtcViewModel: RTCViewModel, manager: ExperienceManager) -> Result<ViewModel, ExperienceError>
}

3. `ExperienceSessionActor`

Redux-like state management for each experience

Session actors manage experience-specific state using a reducer pattern:

// Example for co-reading
struct CoReadingState {
    var selectedStoryId: String?
    var currentLocation: StoryLocation?
    var isFullScreenMode: Bool = false
}

enum CoReadingAction {
    case storySelected(storyId: String)
    case navigationRequested(location: StoryLocation)
    case fullScreenToggled
}

4. `ExperienceEventHub`

Multi-channel event distribution system

Coordinates events across multiple channels with deduplication and ordering:

Firestore Channel: Persistent events and state snapshots
PubSub Channel: Real-time event distribution via HMS
Local Channel: Testing and development

// Send events through all channels
try await eventHub.send(event: storySelectedEvent).asyncSingle()

// Subscribe to validated, deduplicated events
eventHub.events.sink { event in
    // Process event
}.store(in: &cancellables)

5. `ExperienceViewModel`

UI coordination layer

View models handle UI presentation and user interactions:

protocol ExperienceViewModel: ObservableObject, Cleanupable {
    func createContentView() -> AnyView
}

6. `ExperienceSnapshotManager`

State persistence for session resumption

Periodically saves experience state to Firestore for recovery:

// Automatic periodic snapshots
snapshotManager.startPeriodicSaving(interval: 30) // seconds

// Manual snapshot on important state changes
await snapshotManager.saveSnapshotIfNeeded(force: true)

Experience Lifecycle

Inactive ──activate──▶ Initializing ──ready──▶ Active ──end/leave──▶ Deactivating ──cleanup──▶ Inactive
    ▲                      │                     │                         │
    │                      ▼                     ▼                         │
    └──── error/timeout ── Error                Degraded ─────────────────┘

States

Inactive: No experience running
Initializing: Creating components, connecting channels, loading state
Active: Experience running normally, all channels operational
Degraded: Experience running but some channels disconnected
Deactivating: Cleanup in progress
Error: Initialization or operation failed

Lifecycle Events

Activation: User initiates experience → creates session actor → connects event hub → transitions to active
Discovery: Entering call with existing experience → joins session → syncs state
Deactivation: Owner ends experience → sends end event → cleanup → transitions to inactive
Error Recovery: Failed channels → degraded state → potential recovery

Event System

Event Structure

All events conform to ExperienceEvent protocol:

protocol ExperienceEvent: Codable {
    var id: String { get }           // Unique identifier
    var sequence: Int64 { get }      // Ordering timestamp
    var type: String { get }         // Event type
    var senderId: String { get }     // User who sent event
    var data: [String: String] { get } // Event payload
}

Event Flow

Send: User action triggers event creation
Validate: Sequence number validation and deduplication
Distribute: Event sent through all active channels
Process: Session actor receives event and updates state
UI Update: View model observes state changes and updates UI

Built-in Event Types

participant_joined: User joins experience
participant_left: User leaves experience
experience_ended: Owner terminates experience

Experience-specific events are defined by each implementation.

State Management

Redux Pattern

Each experience uses a Redux-like pattern with:

State: Immutable data structure representing current experience state
Actions: Events that describe state changes
Reducer: Pure function that applies actions to create new state
Actor: Manages state transitions and side effects

// Example reducer
func coReadingReducer(state: CoReadingState, action: CoReadingAction) -> CoReadingState {
    var newState = state
    switch action {
    case let .storySelected(storyId):
        newState.selectedStoryId = storyId
    case let .navigationRequested(location):
        newState.currentLocation = location
    }
    return newState
}

State Synchronization

Optimistic Updates: Local state updated immediately
Event Broadcasting: Changes sent to other participants
Conflict Resolution: Sequence numbers ensure consistent ordering
Snapshot Recovery: Periodic state snapshots for crash recovery

Threading Guidelines

MainActor Usage

ExperienceManager: All public methods on main thread
ExperienceViewModel: UI updates must be main thread
State Updates: View model state changes on main thread

Background Processing

Event Processing: Hub processes events on background threads
Network Operations: Firestore/PubSub operations offloaded
State Computation: Reducers can run on background threads

// Correct threading pattern
Task { @MainActor in
    // UI updates on main thread
    self.currentPage = newPage
}

Task.detached {
    // Network operations on background
    try await eventHub.send(event: pageEvent).asyncSingle()
}

Creating New Experiences

Step 1: Define State and Actions

struct CoDrawingState: Codable {
    var canvasData: DrawingCanvas = DrawingCanvas()
    var currentTool: DrawingTool = .pen
    var isFullScreenMode: Bool = false
}

enum CoDrawingAction {
    case strokeAdded(stroke: DrawingStroke)
    case toolChanged(tool: DrawingTool)
    case canvasCleared
}

Step 2: Create Session Actor

actor CoDrawingSessionActor: ExperienceSessionActor {
    typealias State = CoDrawingState
    typealias Action = CoDrawingAction
    
    // Implement required methods
    func reducer(state: State, action: Action) -> State {
        // Apply action to state
    }
    
    func handleEvent(_ event: ExperienceEvent) async -> Action? {
        // Convert events to actions
    }
}

Step 3: Create View Model

@MainActor
class CoDrawingViewModel: ExperienceViewModel {
    @Published var canvasData: DrawingCanvas = DrawingCanvas()
    @Published var currentTool: DrawingTool = .pen
    
    func createContentView() -> AnyView {
        AnyView(CoDrawingView(viewModel: self))
    }
}

Step 4: Create Experience Definition

struct CoDrawingExperienceDefinition: ExperienceDefinition {
    typealias Actor = CoDrawingSessionActor
    typealias ViewModel = CoDrawingViewModel
    static let id = "co_drawing"
    
    func createSessionActor(context: ExperienceInitializationContext) -> Result<Actor, ExperienceError> {
        // Create and configure actor
    }
    
    func createViewModel(sessionActor: Actor, rtcViewModel: RTCViewModel, manager: ExperienceManager) -> Result<ViewModel, ExperienceError> {
        // Create and configure view model
    }
}

Step 5: Register Experience

// In app initialization
ExperienceRegistry.shared.register(CoDrawingExperienceDefinition())

Firestore Layout

Session Documents

/calls/{callId}/experience_sessions/{sessionId}

{
  "experienceId": "co_reading",
  "sessionId": "session_123",
  "createdAt": "2025-01-01T10:00:00Z",
  "createdBy": "user_abc",
  "ownerId": "user_abc",
  "participantIds": ["user_abc", "user_xyz"],
  "status": "active",
  "lastUpdated": "2025-01-01T10:15:00Z"
}

Event Log

/calls/{callId}/experience_sessions/{sessionId}/events/{eventId}

{
  "id": "event_456",
  "sequence": 1704110400000,
  "type": "story_selected",
  "senderId": "user_abc",
  "data": {
    "storyId": "story_789"
  },
  "timestamp": "2025-01-01T10:00:00Z"
}

State Snapshots

/calls/{callId}/experience_sessions/{sessionId}/snapshots/{snapshotId}

{
  "snapshotId": "snapshot_101",
  "sequence": 1704110400000,
  "experienceState": {
    "selectedStoryId": "story_789",
    "currentLocation": {
      "beatId": "beat_1",
      "pageId": "page_1", 
      "lineIndex": 0
    },
    "isFullScreenMode": true
  },
  "createdAt": "2025-01-01T10:00:00Z"
}

Event Ordering Rules

Events include monotonic sequence field (Unix milliseconds)
Conflicts resolved by sender ID lexical comparison
Clients track lastAppliedSequence to prevent duplicates
Firestore timestamps used for persistence only, not ordering

Table of Contents​

Overview​

Core Assumptions​

Architecture​

Component Diagram​

Component Breakdown​

1. ExperienceManager​

2. ExperienceDefinition​

3. ExperienceSessionActor​

4. ExperienceEventHub​

5. ExperienceViewModel​

6. ExperienceSnapshotManager​

Experience Lifecycle​

States​

Lifecycle Events​

Event System​

Event Structure​

Event Flow​

Built-in Event Types​

State Management​

Redux Pattern​

State Synchronization​

Threading Guidelines​

MainActor Usage​

Background Processing​

Creating New Experiences​

Step 1: Define State and Actions​

Step 2: Create Session Actor​

Step 3: Create View Model​

Step 4: Create Experience Definition​

Step 5: Register Experience​

Firestore Layout​

Session Documents​

Event Log​

State Snapshots​

Event Ordering Rules​

Table of Contents