Asset-Frameworker/Deprecated/detailed_documentation_plan.md

# Final Plan for Updating documentation.txt with Debugging Details

This document outlines the final plan for enhancing `documentation.txt` with detailed internal information crucial for debugging the Asset Processor Tool. This plan incorporates analysis of `asset_processor.py`, `configuration.py`, `main.py`, and `gui/processing_handler.py`.

## Task Objective

Analyze relevant source code (`asset_processor.py`, `configuration.py`, `main.py`, `gui/processing_handler.py`) to gather specific details about internal logic, state management, error handling, data structures, concurrency, resource management, and limitations. Integrate this information into the existing `documentation.txt` to aid developers in debugging.

## Analysis Steps Completed

1.  Read and analyzed `readme.md`.
2.  Listed code definitions in the root directory (`.`).
3.  Listed code definitions in the `gui/` directory.
4.  Read and analyzed `asset_processor.py`.
5.  Read and analyzed `configuration.py`.
6.  Read and analyzed `main.py`.
7.  Read and analyzed `gui/processing_handler.py`.

## Final Integration Plan (Merged Structure)

1.  **Add New Top-Level Section:**
    *   Append the following section header to the end of `documentation.txt`:
        ```
        ================================
        Internal Details for Debugging
        ================================
        ```

2.  **Create Subsections:**
    *   Under the new "Internal Details" section, create the following subsections (renumbering from 7 onwards):
        *   `7. Internal Logic & Algorithms`
        *   `8. State Management`
        *   `9. Error Handling & Propagation`
        *   `10. Key Data Structures`
        *   `11. Concurrency Models (CLI & GUI)`
        *   `12. Resource Management`
        *   `13. Known Limitations & Edge Cases`

3.  **Populate Subsections with Specific Details:**

    *   **7. Internal Logic & Algorithms:**
        *   **Configuration Preparation (`Configuration` class):** Detail the `__init__` process: loading `config.py` and preset JSON, validating structure (`_validate_configs`), compiling regex (`_compile_regex_patterns` using `_fnmatch_to_regex`). Mention compiled patterns storage.
        *   **CLI Argument Parsing (`main.py:setup_arg_parser`):** Briefly describe `argparse` usage and key flags influencing execution.
        *   **Output Directory Resolution (`main.py:main`):** Explain how the final output path is determined and resolved.
        *   **Asset Processing (`AssetProcessor` class):**
            *   *Classification (`_inventory_and_classify_files`):* Describe multi-pass approach using compiled regex from `Configuration`. Detail variant sorting criteria.
            *   *Map Processing (`_process_maps`):* Detail image loading, Gloss->Rough inversion, resizing, format determination (using `Configuration` rules), bit depth conversion, stats calculation, aspect ratio logic, save fallback.
            *   *Merging (`_merge_maps`):* Detail common resolution finding, input loading, channel merging (using `Configuration` rules), output determination, save fallback.
            *   *Metadata (`_determine_base_metadata`, etc.):* Summarize base name extraction, category/archetype determination (using `Configuration` rules), `metadata.json` population.
        *   **Blender Integration (`main.py:run_blender_script`, `gui/processing_handler.py:_run_blender_script_subprocess`):** Explain subprocess execution, command construction (`-b`, `--python`, `--`), argument passing (`asset_root_dir`).

    *   **8. State Management:**
        *   `Configuration` object: Holds loaded config state and compiled regex. Instantiated per worker.
        *   `AssetProcessor`: Primarily stateless between `process()` calls. Internal state within `process()` managed by local variables (e.g., `current_asset_metadata`). `self.classified_files` populated once and filtered per asset.
        *   `main.py`: Tracks overall CLI run counts (processed, skipped, failed).
        *   `gui/processing_handler.py`: Manages GUI processing run state via flags (`_is_running`, `_cancel_requested`) and stores `Future` objects (`_futures`).

    *   **9. Error Handling & Propagation:**
        *   Custom Exceptions: `AssetProcessingError`, `ConfigurationError`.
        *   `Configuration`: Raises `ConfigurationError` on load/validation failure. Logs regex compilation warnings.
        *   `AssetProcessor`: Catches exceptions within per-asset loop, logs error, marks asset "failed", continues loop. Handles specific save fallbacks (EXR->PNG). Raises `AssetProcessingError` for critical setup failures.
        *   Worker Wrapper (`main.py:process_single_asset_wrapper`): Catches exceptions from `Configuration`/`AssetProcessor`, logs, returns "failed" status tuple.
        *   Process Pool (`main.py`, `gui/processing_handler.py`): `try...except` around executor block catches pool-level errors.
        *   GUI Communication (`ProcessingHandler`): Catches errors during future result retrieval, emits failure status via signals.
        *   Blender Scripts: Checks subprocess return code, logs stderr. Catches `FileNotFoundError`.

    *   **10. Key Data Structures:**
        *   `Configuration` attributes: `compiled_map_keyword_regex`, `compiled_extra_regex`, etc. (compiled `re.Pattern` objects).
        *   `AssetProcessor` structures: `self.classified_files` (dict[str, list[dict]]), `processed_maps_details_asset` (dict[str, dict[str, dict]]), `file_to_base_name_map` (dict[Path, Optional[str]]).
        *   Return values: Status dictionary from `AssetProcessor.process()`, status tuple from `process_single_asset_wrapper`.
        *   `ProcessingHandler._futures`: dict[Future, str].

    *   **11. Concurrency Models (CLI & GUI):**
        *   **Common Core:** Both use `concurrent.futures.ProcessPoolExecutor` running `main.process_single_asset_wrapper`. `Configuration` and `AssetProcessor` instantiated within each worker process for isolation.
        *   **CLI Orchestration (`main.py:run_processing`):** Direct executor usage, `as_completed` gathers results synchronously.
        *   **GUI Orchestration (`gui/processing_handler.py`):** `ProcessingHandler` (QObject) runs executor logic in a `QThread`. Results processed in handler thread, communicated asynchronously to UI thread via Qt signals (`progress_updated`, `file_status_updated`, `processing_finished`).
        *   **Cancellation (`gui/processing_handler.py:request_cancel`):** Sets flag, attempts `executor.shutdown(wait=False)`, tries cancelling pending futures. Limitation: Does not stop already running workers.

    *   **12. Resource Management:**
        *   `Configuration`: Uses `with open` for preset files.
        *   `AssetProcessor`: Manages temporary workspace (`tempfile.mkdtemp`, `shutil.rmtree` in `finally`). Uses `with open` for metadata JSON.
        *   `ProcessPoolExecutor`: Lifecycle managed via `with` statement in `main.py` and `gui/processing_handler.py`.

    *   **13. Known Limitations & Edge Cases:**
        *   Configuration: Basic structural validation only; regex compilation errors are warnings; `_fnmatch_to_regex` helper is basic.
        *   AssetProcessor: Relies heavily on filename patterns; high memory potential for large images; limited intra-asset error recovery; simplified prediction logic.
        *   CLI: Basic preset file existence check only before starting workers; Blender executable finding logic order (config > PATH).
        *   GUI Concurrency: Cancellation doesn't stop currently executing worker processes.

4.  **Switch Mode:** Request switching to Code mode to apply these changes by appending to `documentation.txt`.