# Plan: Enforcing "MAP_" Prefix for Internal Processing and Standard Type for Output Naming

**Date:** 2025-05-13

**I. Goal:**
The primary goal is to ensure that for all internal processing, the system *exclusively* uses `FileRule.item_type` values that start with the "MAP_" prefix (e.g., "MAP_COL", "MAP_NRM"). The "standard type" (e.g., "COL", "NRM") associated with these "MAP_" types (as defined in `config/app_settings.json`) should *only* be used during the file saving stages for output naming. Any `FileRule` whose `item_type` does not start with "MAP_" (and isn't a special type like "EXTRA" or "MODEL") should be skipped by the relevant map processing stages.

**II. Current State Analysis Summary:**

*   **Output Naming:** The use of "standard type" for output filenames via the `get_filename_friendly_map_type` utility in `SaveVariantsStage` and `OutputOrganizationStage` is **correct** and already meets the requirement.
*   **Internal "MAP_" Prefix Usage:**
    *   Some stages like `GlossToRoughConversionStage` correctly check for "MAP_" prefixes (e.g., `processing_map_type.startswith("MAP_GLOSS")`).
    *   Other stages like `RegularMapProcessorStage` and `MergedTaskProcessorStage` (and its helpers) implicitly expect "MAP_" prefixed types for their internal regex-based logic but lack explicit checks to skip items if the prefix is missing.
    *   Stages like `AlphaExtractionToMaskStage` and `NormalMapGreenChannelStage` currently use non-"MAP_" prefixed "standard types" (e.g., "NORMAL", "ALBEDO") when reading from `context.processed_maps_details` for their decision-making logic.
    *   The `PrepareProcessingItemsStage` adds `FileRule`s to the processing queue without filtering based on the "MAP_" prefix in `item_type`.
*   **Data Consistency in `AssetProcessingContext`:**
    *   `FileRule.item_type` is the field that should hold the "MAP_" prefixed type from the initial rule generation.
    *   `context.processed_maps_details` entries can contain various map type representations:
        *   `map_type`: Often stores the "standard type" (e.g., "Roughness", "MASK", "NORMAL").
        *   `processing_map_type` / `internal_map_type`: Generally seem to store the "MAP_" prefixed type. This needs to be consistent.
*   **Configuration (`config/app_settings.json`):**
    *   `FILE_TYPE_DEFINITIONS` correctly use "MAP_" prefixed keys.
    *   `MAP_MERGE_RULES` need to be reviewed to ensure their `output_map_type` and input map types are "MAP_" prefixed.

**III. Proposed Changes (Code Identification & Recommendations):**

**A. Enforce "MAP_" Prefix for Processing Items (Skipping Logic):**
The core requirement is that processing stages should skip `FileRule` items if their `item_type` doesn't start with "MAP_".

1.  **`RegularMapProcessorStage` (`processing/pipeline/stages/regular_map_processor.py`):**
    *   **Identify:** In the `execute` method, `initial_internal_map_type` is derived from `file_rule.item_type_override` or `file_rule.item_type`.
    *   **Recommend:** Add an explicit check after determining `initial_internal_map_type`. If `initial_internal_map_type` does not start with `"MAP_"`, the stage should log a warning, set the `result.status` to "Skipped (Invalid Type)" or similar, and return `result` early, effectively skipping processing for this item.

2.  **`MergedTaskProcessorStage` (`processing/pipeline/stages/merged_task_processor.py`):**
    *   **Identify:** This stage processes `MergeTaskDefinition`s. The definitions for these tasks (input types, output type) come from `MAP_MERGE_RULES` in `config/app_settings.json`. The stage uses `required_map_type_from_rule` for its inputs.
    *   **Recommend:**
        *   **Configuration First:** Review all entries in `MAP_MERGE_RULES` in `config/app_settings.json`.
            *   Ensure the `output_map_type` for each rule (e.g., "MAP_NRMRGH") starts with "MAP_".
            *   Ensure all map type values within the `inputs` dictionary (e.g., `"R": "MAP_NRM"`) start with "MAP_".
        *   **Stage Logic:** In the `execute` method, when iterating through `merge_inputs_config.items()`, check if `required_map_type_from_rule` starts with `"MAP_"`. If not, log a warning and either:
            *   Skip loading/processing this specific input channel (potentially using its fallback if the overall merge can still proceed).
            *   Or, if a non-"MAP_" input is critical, fail the entire merge task for this asset.
        *   The helper `_apply_in_memory_transformations` already uses regex expecting "MAP_" prefixes; this will naturally fail or misbehave if inputs are not "MAP_" prefixed, reinforcing the need for the check above.

**B. Standardize Map Type Fields and Usage in `context.processed_maps_details`:**
Ensure consistency in how "MAP_" prefixed types are stored and accessed within `context.processed_maps_details` for internal logic (not naming).

1.  **Recommendation:** Establish a single, consistent field name within `context.processed_maps_details` to store the definitive "MAP_" prefixed internal map type (e.g., `internal_map_type` or `processing_map_type`). All stages that perform logic based on the specific *kind* of map (e.g., transformations, source selection) should read from this standardized field. The `map_type` field can continue to store the "standard type" (e.g., "Roughness") primarily for informational/metadata purposes if needed, but not for core processing logic.

2.  **`AlphaExtractionToMaskStage` (`processing/pipeline/stages/alpha_extraction_to_mask.py`):**
    *   **Identify:**
        *   Checks for existing MASK map using `file_rule.map_type == "MASK"`. (Discrepancy: `FileRule` uses `item_type`).
        *   Searches for suitable source maps using `details.get('map_type') in self.SUITABLE_SOURCE_MAP_TYPES` where `SUITABLE_SOURCE_MAP_TYPES` are standard types like "ALBEDO".
        *   When adding new details, it sets `map_type: "MASK"` and the new `FileRule` gets `item_type="MAP_MASK"`.
    *   **Recommend:**
        *   Change the check for an existing MASK map to `file_rule.item_type == "MAP_MASK"`.
        *   Modify the source map search to use the standardized "MAP_" prefixed field from `details` (e.g., `details.get('internal_map_type')`) and update `SUITABLE_SOURCE_MAP_TYPES` to be "MAP_" prefixed (e.g., "MAP_COL", "MAP_ALBEDO").
        *   When adding new details for the created MASK map to `context.processed_maps_details`, ensure the standardized "MAP_" prefixed field is set to "MAP_MASK", and `map_type` (if kept) is "MASK".

3.  **`NormalMapGreenChannelStage` (`processing/pipeline/stages/normal_map_green_channel.py`):**
    *   **Identify:** Checks `map_details.get('map_type') == "NORMAL"`.
    *   **Recommend:** Change this check to use the standardized "MAP_" prefixed field from `map_details` (e.g., `map_details.get('internal_map_type')`) and verify if it `startswith("MAP_NRM")`.

4.  **`GlossToRoughConversionStage` (`processing/pipeline/stages/gloss_to_rough_conversion.py`):**
    *   **Identify:** This stage already uses `processing_map_type.startswith("MAP_GLOSS")` and updates `processing_map_type` to "MAP_ROUGH" in `map_details`. It also updates the `FileRule.item_type` correctly.
    *   **Recommend:** This stage is largely consistent. Ensure the field it reads/writes (`processing_map_type`) aligns with the chosen standardized "MAP_" prefixed field for `processed_maps_details`.

**C. Review Orchestration Logic (Conceptual):**
*   When the orchestrator populates `context.processed_maps_details` after stages like `SaveVariantsStage`, ensure it stores the "MAP_" prefixed `internal_map_type` (from `SaveVariantsInput`) into the chosen standardized field in `processed_maps_details`.

**IV. Testing Recommendations:**

*   Create test cases with `AssetRule`s containing `FileRule`s where `item_type` is intentionally set to a non-"MAP_" prefixed value (e.g., "COLOR_MAP", "TEXTURE_ROUGH"). Verify that `RegularMapProcessorStage` skips these.
*   Modify `MAP_MERGE_RULES` in a test configuration:
    *   Set an `output_map_type` to a non-"MAP_" value.
    *   Set an input map type (e.g., for channel "R") to a non-"MAP_" value.
    *   Verify that `MergedTaskProcessorStage` correctly handles these (e.g., fails the task, skips the input, logs warnings).
*   Test `AlphaExtractionToMaskStage`:
    *   With an existing `FileRule` having `item_type="MAP_MASK"` to ensure extraction is skipped.
    *   With source maps having "MAP_COL" (with alpha) as their `internal_map_type` in `processed_maps_details` to ensure they are correctly identified as sources.
*   Test `NormalMapGreenChannelStage` with a normal map having "MAP_NRM" as its `internal_map_type` in `processed_maps_details` to ensure it's processed.
*   Verify that output filenames continue to use the "standard type" (e.g., "COL", "ROUGH", "NRM") correctly.

**V. Mermaid Diagram (Illustrative Flow for `FileRule` Processing):**

```mermaid
graph TD
    A[AssetRule with FileRules] --> B{FileRuleFilterStage};
    B -- files_to_process --> C{PrepareProcessingItemsStage};
    C -- processing_items (FileRule) --> D{PipelineOrchestrator};
    D -- FileRule --> E(RegularMapProcessorStage);
    E --> F{Check FileRule.item_type};
    F -- Starts with "MAP_"? --> G[Process Map];
    F -- No --> H[Skip Map / Log Warning];
    G --> I[...subsequent stages...];
    H --> I;