Asset-Frameworker/Project Notes/FORCE_LOSSLESS_PLAN.md

Plan: Implement "Force Lossless" Format for Specific Map Types

Goal: Modify the asset processor to ensure specific map types ("NRM", "DISP") are always saved in a lossless format (PNG or EXR based on bit depth), overriding the JPG threshold and input format rules. This rule should apply to both individually processed maps and merged maps.

Steps:

1. Add Configuration Setting (config.py):

   • Introduce a new list named FORCE_LOSSLESS_MAP_TYPES in config.py.
   • Populate this list: FORCE_LOSSLESS_MAP_TYPES = ["NRM", "DISP"].

2. Expose Setting in Configuration Class (configuration.py):

   • Add a default value for FORCE_LOSSLESS_MAP_TYPES in the _load_core_config method's default_core_settings dictionary.
   • Add a new property to the Configuration class to access this list:

     @property
     def force_lossless_map_types(self) -> list:
         """Gets the list of map types that must always be saved losslessly."""
         return self._core_settings.get('FORCE_LOSSLESS_MAP_TYPES', [])
     

3. Modify _process_maps Method (asset_processor.py):

   • Locate the section determining the output format (around line 805).
   • Before the if output_bit_depth == 8 and target_dim >= threshold: check (line 811), insert the new logic:
     • Check if the current map_type is in self.config.force_lossless_map_types.
     • If yes, determine the appropriate lossless format (png or configured 16-bit format like exr) based on output_bit_depth, set output_format, output_ext, save_params, and needs_float16 accordingly, and skip the subsequent elif / else blocks for format determination.
     • Use an elif for the existing JPG threshold check and the final else for the rule-based logic, ensuring they only run if force_lossless is false.

4. Modify _merge_maps Method (asset_processor.py):

   • Locate the section determining the output format for the merged map (around line 1151).
   • Before the if output_bit_depth == 8 and target_dim >= threshold: check (line 1158), insert similar logic as in step 3:
     • Check if the output_map_type (the type of the merged map) is in self.config.force_lossless_map_types.
     • If yes, determine the appropriate lossless format based on the merged map's output_bit_depth, set output_format, output_ext, save_params, and needs_float16, and skip the subsequent elif / else blocks.
     • Use elif and else for the existing threshold and hierarchy logic.

Process Flow Diagram:

graph TD
    subgraph Format Determination (per resolution, _process_maps & _merge_maps)
        A[Start] --> B(Get map_type / output_map_type);
        B --> C{Determine output_bit_depth};
        C --> D{Is map_type in FORCE_LOSSLESS_MAP_TYPES?};
        D -- Yes --> E[Set format = Lossless (PNG/EXR based on bit_depth)];
        D -- No --> F{Is output_bit_depth == 8 AND target_dim >= threshold?};
        F -- Yes --> G[Set format = JPG];
        F -- No --> H[Set format based on input/hierarchy/rules];
        G --> I[Set Save Params];
        H --> I;
        E --> I;
        I --> J[Save Image];
    end