Asset-Frameworker/ProjectNotes/MEMORY_OPTIMIZATION_PLAN.md

Memory Optimization Plan: Strategy 2 - Load Grayscale Directly

This plan outlines the steps to implement memory optimization strategy #2, which involves loading known grayscale map types directly as grayscale images using OpenCV's IMREAD_GRAYSCALE flag. This reduces the memory footprint compared to loading them with IMREAD_UNCHANGED and then potentially converting later.

1. Identify Target Grayscale Map Types

Define a list of map type names (case-insensitive check recommended during implementation) that should always be treated as single-channel grayscale data.

Initial List:

GRAYSCALE_MAP_TYPES = ['HEIGHT', 'ROUGH', 'METAL', 'AO', 'OPC', 'MASK']

(Note: This list might need adjustment based on specific preset configurations or workflow requirements.)

2. Modify _process_maps Loading Logic

Locate the primary image loading section within the _process_maps method in asset_processor.py (around line 608).

Change: Before calling cv2.imread, determine the correct flag based on the map_type:

# (Define GRAYSCALE_MAP_TYPES list earlier in the scope or class)

# ... inside the loop ...
full_source_path = self.temp_dir / source_path_rel

# Determine the read flag
read_flag = cv2.IMREAD_GRAYSCALE if map_type.upper() in GRAYSCALE_MAP_TYPES else cv2.IMREAD_UNCHANGED
log.debug(f"Loading source {source_path_rel.name} with flag: {'GRAYSCALE' if read_flag == cv2.IMREAD_GRAYSCALE else 'UNCHANGED'}")

# Load the image using the determined flag
img_loaded = cv2.imread(str(full_source_path), read_flag)

if img_loaded is None:
    raise AssetProcessingError(f"Failed to load image file: {full_source_path.name} with flag {read_flag}")

# ... rest of the processing logic ...

3. Modify _merge_maps Loading Logic

Locate the image loading section within the resolution loop in the _merge_maps method (around line 881).

Change: Apply the same conditional logic to determine the imread flag when loading input maps for merging:

# ... inside the loop ...
input_file_path = self.temp_dir / res_details['path']

# Determine the read flag (reuse GRAYSCALE_MAP_TYPES list)
read_flag = cv2.IMREAD_GRAYSCALE if map_type.upper() in GRAYSCALE_MAP_TYPES else cv2.IMREAD_UNCHANGED
log.debug(f"Loading merge input {input_file_path.name} ({map_type}) with flag: {'GRAYSCALE' if read_flag == cv2.IMREAD_GRAYSCALE else 'UNCHANGED'}")

# Load the image using the determined flag
img = cv2.imread(str(input_file_path), read_flag)

if img is None:
    raise AssetProcessingError(f"Failed to load merge input {input_file_path.name} with flag {read_flag}")

# ... rest of the merging logic ...

4. Verification

During implementation in code mode:

• Ensure the GRAYSCALE_MAP_TYPES list is defined appropriately (e.g., as a class constant or module-level constant).
• Confirm that downstream code (e.g., stats calculation, channel extraction, data type conversions) correctly handles numpy arrays that might be 2D (grayscale) instead of 3D (BGR/BGRA). The existing code appears to handle this, but it's important to verify.

Mermaid Diagram of Change

graph TD
    subgraph _process_maps
        A[Loop through map_info] --> B{Is map_type Grayscale?};
        B -- Yes --> C[imread(path, GRAYSCALE)];
        B -- No --> D[imread(path, UNCHANGED)];
        C --> E[Process Image];
        D --> E;
    end
    subgraph _merge_maps
        F[Loop through resolutions] --> G[Loop through required_input_types];
        G --> H{Is map_type Grayscale?};
        H -- Yes --> I[imread(path, GRAYSCALE)];
        H -- No --> J[imread(path, UNCHANGED)];
        I --> K[Use Image in Merge];
        J --> K;
    end

    style B fill:#f9f,stroke:#333,stroke-width:2px
    style H fill:#f9f,stroke:#333,stroke-width:2px