Unified Asset Loading Architecture Plan

Executive Summary

This document outlines a plan to unify the asset loading patterns across beats, images, and audio in the Pajama story reader. The goal is to implement a consistent "ask for it immediately, if not available then wait for it" pattern that allows for critical asset blocking, predictable loading states, and future-proof extensibility.

Current Architecture Analysis

Existing Patterns

Beat Loading (Current - Works Well)

// StoryContentManager pattern
func getBeat(_ beatId: BeatId) -> PajamaScriptStoryBeat?           // Sync cache check
func loadBeat(_ beatId: BeatId) async throws -> PajamaScriptStoryBeat  // Async load with hierarchy
func isBeatLoaded(_ beatId: BeatId) -> Bool                       // Sync availability check
func doesBeatExist(_ beatId: BeatId) -> Bool                      // Sync existence check

// Usage pattern
if let beat = contentManager.getBeat(beatId) {
    // Use immediately
} else {
    // Show loading, async load with retry
    beat = try await contentManager.loadBeat(beatId) 
}

Image Loading (Current - Inconsistent)

// ImageLoadingService pattern
func loadImage(for url: URL, processing: ProcessingConfiguration) async -> CGImage?
func preloadImages(for urlStrings: [String], processing: ProcessingConfiguration)
func getMemoryCachedImageSync(for urlString: String, processing: ProcessingConfiguration) -> CGImage?

// Usage pattern - UI components only
CachedAsyncImage(url: url, imageLoader: service) { image in
    // Display when loaded
} placeholder: {
    // Show loading spinner
}

Audio Loading (Current - Fire-and-Forget)

// AudioManager pattern
func preloadSounds(names: [String], soundType: SoundType)  // Fire-and-forget
func playSoundEffect(named filename: String)               // Fails silently if not loaded
private func fetchBuffer(name: String, _ type: SoundType) async -> AVAudioPCMBuffer?  // Private only

// Usage pattern
AudioManager.shared.preloadSounds(names: audioUrls, soundType: .effect)  // Hope it's ready later
AudioManager.shared.playSoundEffect(named: url)  // No way to check if ready

Identified Inconsistencies

1. Missing Sync Checks: Audio has no public sync availability check
2. Different Blocking Behaviors: Only beats can block navigation
3. Inconsistent Error Handling: Different retry mechanisms across asset types
4. No Criticality Support: Can't mark images/audio as navigation-blocking
5. Limited Future Extensibility: Hard to add new asset-dependent features

Vision: Unified Asset Loading Pattern

Core Principle

All assets should support the same "sync check, async load" pattern that beats use successfully.

Unified Interface Design

protocol AssetManaging {
    associatedtype Asset
    associatedtype AssetId
    
    // Sync operations (immediate cache check)
    func getAsset(_ id: AssetId) -> Asset?
    func isAssetLoaded(_ id: AssetId) -> Bool
    func doesAssetExist(_ id: AssetId) -> Bool
    
    // Async operations (full hierarchy loading)
    func loadAsset(_ id: AssetId) async throws -> Asset
    func preloadAssets(_ ids: [AssetId]) -> Void
    
    // Management
    func cancelAssetLoading(_ id: AssetId) async
    func clearCache() async
}

Enhanced StoryContentManager

class StoryContentManager {
    // MARK: - Unified Asset Access Pattern
    
    // Beats (existing - works well)
    func getBeat(_ beatId: BeatId) -> PajamaScriptStoryBeat?
    func loadBeat(_ beatId: BeatId) async throws -> PajamaScriptStoryBeat
    
    // Images (new unified pattern)
    func getImage(_ url: String, processing: ProcessingConfiguration = .storyImage) -> CGImage?
    func loadImage(_ url: String, processing: ProcessingConfiguration = .storyImage) async throws -> CGImage
    func isImageLoaded(_ url: String, processing: ProcessingConfiguration = .storyImage) -> Bool
    
    // Audio (new unified pattern)  
    func getAudio(_ url: String) -> AVAudioPCMBuffer?
    func loadAudio(_ url: String) async throws -> AVAudioPCMBuffer
    func isAudioLoaded(_ url: String) -> Bool
    
    // MARK: - Asset Criticality System
    
    enum AssetCriticality {
        case optional     // Progressive loading OK (current image behavior)
        case important    // Preload aggressively but don't block navigation
        case critical     // Block navigation until loaded (current beat behavior)
    }
    
    func setImageCriticality(_ criticality: AssetCriticality, for url: String)
    func setAudioCriticality(_ criticality: AssetCriticality, for url: String)
    
    // MARK: - Navigation-Aware Loading
    
    func canNavigateToLocation(_ location: StoryLocation) async -> Bool {
        guard let beat = getBeat(location.beatId) else { return false }
        
        // Check critical assets for this location
        let criticalAssets = getCriticalAssets(for: location)
        return await areAssetsReady(criticalAssets)
    }
    
    func loadAssetsForLocation(_ location: StoryLocation, 
                              priority: AssetPriority = .normal) async throws {
        // Load all critical assets before allowing navigation
        let criticalAssets = getCriticalAssets(for: location)
        try await loadAssets(criticalAssets, priority: .critical)
        
        // Start preloading important/optional assets
        let optionalAssets = getOptionalAssets(for: location)
        preloadAssets(optionalAssets, priority: priority)
    }
}

Implementation Strategy

Phase 1: Foundation - Audio Manager Enhancement

Goal: Bring AudioManager up to beat loading pattern standards

Changes:

1. Add public sync methods to AudioManager:

   func getAudioBuffer(_ name: String) -> AVAudioPCMBuffer?
   func isAudioLoaded(_ name: String) -> Bool
   func loadAudio(_ name: String) async throws -> AVAudioPCMBuffer

2. Expose error handling and cancellation:

   func cancelAudioLoading(_ name: String) async

3. Add asset existence checking:

   func doesAudioExist(_ name: String) -> Bool  // Check if URL is valid

Testing: Ensure audio loading works with sync checks before playback

Phase 2: Foundation - Image Service Enhancement

Goal: Add missing sync methods to ImageLoadingService

Changes:

1. Enhance ImageLoadingServiceProtocol:

   func getImage(_ url: String, processing: ProcessingConfiguration) -> CGImage?
   func isImageLoaded(_ url: String, processing: ProcessingConfiguration) -> Bool
   func loadImage(_ url: String, processing: ProcessingConfiguration) async throws -> CGImage

2. Unify error handling with beats pattern

3. Add cancellation support per image URL

Testing: Verify image loading works with explicit async calls

Phase 3: StoryContentManager Unification

Goal: Add unified asset methods to StoryContentManager

Changes:

1. Add image methods that delegate to ImageLoadingService:

   func getImage(_ url: String, processing: ProcessingConfiguration = .storyImage) -> CGImage? {
       imageLoadingService.getImage(url, processing: processing)
   }
   
   func loadImage(_ url: String, processing: ProcessingConfiguration = .storyImage) async throws -> CGImage {
       try await imageLoadingService.loadImage(url, processing: processing)
   }

2. Add audio methods that delegate to AudioManager:

   func getAudio(_ url: String) -> AVAudioPCMBuffer? {
       AudioManager.shared.getAudioBuffer(url)
   }
   
   func loadAudio(_ url: String) async throws -> AVAudioPCMBuffer {
       try await AudioManager.shared.loadAudio(url)
   }

3. Implement asset extraction and management:

   private func extractCriticalAssets(for location: StoryLocation) -> [AssetReference] {
       // Extract images, audio from beat pages based on criticality configuration
   }

Testing: Verify unified interface works end-to-end

Phase 4: Asset Criticality System

Goal: Allow configuration of which assets block navigation

Changes:

1. Asset criticality configuration:

   enum AssetCriticality { case optional, important, critical }
   private var imageCriticality: [String: AssetCriticality] = [:]
   private var audioCriticality: [String: AssetCriticality] = [:]

2. Navigation blocking logic:

   func canNavigateToLocation(_ location: StoryLocation) async -> Bool {
       // Check that all critical assets are loaded
   }

3. Integration with LoadingStateManager for unified loading UI

Testing: Test navigation blocking for critical assets

Phase 5: Navigation Integration

Goal: Integrate asset loading with navigation system

Changes:

1. Update StoryReaderViewModel navigation methods:

   func changePage(to location: StoryLocation, direction: NavigationDirection, reason: NavigationReason) async {
       // Check if critical assets are ready
       if await !contentManager.canNavigateToLocation(location) {
           // Show loading state and load critical assets
           try await contentManager.loadAssetsForLocation(location, priority: .critical)
       }
       
       // Proceed with navigation
       await pageController?.executeNavigation(to: location, direction: direction, ...)
   }

2. Update choice handling to check audio availability:

   func makeChoice(choiceIndex: Int) async {
       // For tap-triggered audio, ensure it's loaded
       if let tapAudio = getTapAudioForChoice(choiceIndex) {
           if !contentManager.isAudioLoaded(tapAudio) {
               try await contentManager.loadAudio(tapAudio)
           }
       }
       // Proceed with choice...
   }

Testing: End-to-end navigation with asset loading

Phase 6: UI Component Migration

Goal: Migrate UI components to use unified pattern where beneficial

Changes:

1. Update page views to support critical image loading:

   // For critical images, block until loaded
   if isImageCritical {
       if let image = viewModel.contentManager.getImage(imageUrl) {
           // Display immediately
       } else {
           // Show loading and wait
           let image = try await viewModel.contentManager.loadImage(imageUrl)
       }
   } else {
       // Use existing CachedAsyncImage for progressive loading
       CachedAsyncImage(...)
   }

2. Add immediate audio checks for tap triggers:

   .onTapGesture {
       if let tapAudio = page.tapAudio {
           if viewModel.contentManager.isAudioLoaded(tapAudio) {
               AudioManager.shared.playSoundEffect(named: tapAudio)
           } else {
               // Either skip audio or show loading
           }
       }
   }

Testing: Verify UI behavior for both critical and optional assets

API Design Details

Error Handling

All asset loading should use consistent error types:

enum AssetLoadingError: Error {
    case notFound(assetId: String)
    case networkError(underlying: Error)
    case processingError(underlying: Error)
    case cancelled
    case timeout
    case invalidFormat
}

Loading States

Unified loading state management:

extension LoadingStateManager {
    func executeAssetLoading<T>(
        description: String,
        assetId: String,
        operation: () async throws -> T
    ) async rethrows -> T {
        // Same retry/timeout logic as beat loading
    }
}

Configuration

Asset behavior configuration:

struct AssetConfiguration {
    var imageCriticality: [String: AssetCriticality] = [:]
    var audioCriticality: [String: AssetCriticality] = [:]
    var enableImageBlocking: Bool = false
    var enableAudioBlocking: Bool = false
    var defaultImageCriticality: AssetCriticality = .optional
    var defaultAudioCriticality: AssetCriticality = .optional
}

Migration Strategy

Backward Compatibility

1. Phase 1-3: All existing code continues to work unchanged
2. Phase 4-5: New features opt-in to criticality system
3. Phase 6: UI components gradually migrate to unified pattern

Feature Flags

struct FeatureFlags {
    static var enableUnifiedAssetLoading = false
    static var enableAssetCriticality = false  
    static var enableImageBlocking = false
    static var enableAudioBlocking = false
}

Rollout Plan

1. Development: Implement behind feature flags
2. Internal Testing: Enable for specific stories/features
3. Beta: Gradual rollout with monitoring
4. Production: Full rollout after validation

Testing Strategy

Unit Tests

1. AudioManager: Test sync methods, async loading, error handling
2. ImageLoadingService: Test unified interface consistency
3. StoryContentManager: Test asset coordination and criticality
4. LoadingStateManager: Test unified loading states

Integration Tests

1. Navigation Blocking: Test critical asset blocking behavior
2. Asset Coordination: Test mixed critical/optional loading
3. Error Recovery: Test retry mechanisms across asset types
4. Performance: Test loading times and memory usage

UI Tests

1. Critical Image Loading: Test navigation blocking for images
2. Critical Audio Loading: Test playback blocking for audio
3. Progressive Loading: Test optional asset behavior unchanged
4. Loading States: Test unified loading indicators

Performance Considerations

Memory Management

1. Unified Caching: Coordinate cache limits across all asset types
2. Priority Loading: Critical assets get priority in concurrent loading
3. Smart Preloading: Avoid redundant loading across asset types

Network Optimization

1. Batched Requests: Load multiple assets in parallel when possible
2. Request Deduplication: Avoid duplicate requests across systems
3. Intelligent Prioritization: Critical > Important > Optional

User Experience

1. Loading Indicators: Unified loading states across asset types
2. Progressive Enhancement: Optional assets enhance without blocking
3. Error Recovery: Consistent retry behavior for all assets

Success Metrics

Functionality

• Audio loading supports sync availability checks
• Image loading supports async explicit loading
• Navigation can block for critical assets
• Unified error handling across all asset types
• Feature flags allow gradual rollout

Performance

• Loading times comparable to current architecture
• Memory usage within acceptable bounds
• Network requests remain efficient
• Cache hit rates maintain or improve

User Experience

• Smooth navigation for critical assets
• Progressive loading preserved for optional assets
• Consistent loading indicators
• Reliable error recovery

Future Extensions

New Asset Types

The unified pattern makes it easy to add new asset types:

// Video assets
func getVideo(_ url: String) -> AVAsset?
func loadVideo(_ url: String) async throws -> AVAsset

// 3D models  
func getModel(_ url: String) -> SCNNode?
func loadModel(_ url: String) async throws -> SCNNode

Advanced Features

1. Smart Preloading: ML-based prediction of needed assets
2. Adaptive Quality: Load different asset qualities based on network
3. Offline Mode: Coordinated offline asset management
4. Analytics: Unified asset loading metrics

Conclusion

This unified asset loading architecture brings consistency, predictability, and extensibility to the Pajama asset system. By adopting the successful beat loading pattern for images and audio, we enable:

1. Critical Asset Support: Navigation blocking for essential content
2. Immediate Availability Checks: Support for features like tap-triggered audio
3. Consistent Error Handling: Unified retry and recovery mechanisms
4. Future Extensibility: Easy addition of new asset types and features
5. Backward Compatibility: Gradual migration without breaking changes

The phased implementation approach minimizes risk while delivering incremental value, and the feature flag system allows for careful rollout and validation.