Asset-Frameworker/Documentation/02_Developer_Guide/03_Key_Components.md

Developer Guide: Key Components

This document describes the major classes and modules that form the core of the Asset Processor Tool.

ProcessingEngine (processing_engine.py)

The ProcessingEngine class is the new core component responsible for executing the asset processing pipeline for a single input asset. Unlike the older AssetProcessor, this engine operates solely based on a complete SourceRule object provided to its process() method and the static Configuration object passed during initialization. It contains no internal prediction, classification, or fallback logic. Its key responsibilities include:

• Setting up and cleaning up a temporary workspace for processing (potentially using utils.workspace_utils).
• Extracting or copying input files to the workspace.
• Processing files based on the explicit rules and predicted values contained within the input SourceRule.
• Processing texture maps (resizing, format/bit depth conversion, inversion, stats calculation) using parameters from the SourceRule or static Configuration.
• Merging channels based on rules defined in the static Configuration and parameters from the SourceRule.
• Generating the metadata.json file containing details about the processed asset, incorporating information from the SourceRule.
• Organizing the final output files into the structured library directory.

Rule Structure (rule_structure.py)

This module defines the data structures used to represent the hierarchical processing rules:

• SourceRule: A dataclass representing rules applied at the source level. It contains nested AssetRule objects.
• AssetRule: A dataclass representing rules applied at the asset level. It contains nested FileRule objects.
• FileRule: A dataclass representing rules applied at the file level.

These classes hold specific rule parameters (e.g., supplier_identifier, asset_type, asset_type_override, item_type, item_type_override, target_asset_name_override). Attributes like asset_type and item_type_override now use string types, which are validated against centralized lists in config/app_settings.json. These structures support serialization (Pickle, JSON) to allow them to be passed between different parts of the application, including across process boundaries.

Configuration (configuration.py)

The Configuration class manages the tool's settings. It is responsible for:

• Loading the core default settings defined in config/app_settings.json.
• Loading the supplier-specific rules from a selected preset JSON file (Presets/*.json).
• Merging the core settings and preset rules into a single, unified configuration object.
• Validating the loaded configuration to ensure required settings are present.
• Pre-compiling regular expression patterns defined in the preset for efficient file classification by the PredictionHandler.

An instance of the Configuration class is typically created once per application run (or per processing batch) and passed to the ProcessingEngine.

GUI Components (gui/)

The GUI has been refactored into several key components:

MainWindow (gui/main_window.py)

The MainWindow class acts as the main application window and coordinator for the GUI. Its primary responsibilities now include:

• Setting up the main window structure (using a QSplitter) and menu bar.
• Instantiating and arranging the major GUI widgets:
  • PresetEditorWidget (providing selector and JSON editor parts)
  • LLMEditorWidget (for LLM settings)
  • MainPanelWidget (containing the rule view and processing controls)
  • LogConsoleWidget
• Layout Management: Placing the preset selector statically and using a QStackedWidget to switch between the PresetEditorWidget's JSON editor and the LLMEditorWidget.
• Editor Switching: Handling the preset_selection_changed_signal from PresetEditorWidget to switch the stacked editor view (_on_preset_selection_changed slot).
• Connecting signals and slots between widgets, models (UnifiedViewModel), and handlers (LLMInteractionHandler, AssetRestructureHandler).
• Managing the overall application state related to GUI interactions (e.g., enabling/disabling controls).
• Handling top-level actions like loading sources (drag-and-drop), initiating predictions (update_preview), and starting the processing task (_on_process_requested).
• Managing background prediction threads (Rule-Based via QThread, LLM via LLMInteractionHandler).
• Implementing slots (_on_rule_hierarchy_ready, _on_llm_prediction_ready_from_handler, _on_prediction_error, _handle_prediction_completion) to update the model/view when prediction results/errors arrive.

MainPanelWidget (gui/main_panel_widget.py)

This widget contains the central part of the GUI, including:

• Controls for loading source files/directories.
• The preset selection dropdown.
• Buttons for initiating prediction and processing.
• The RuleEditorWidget which houses the hierarchical rule view.

PresetEditorWidget (gui/preset_editor_widget.py)

This widget provides the interface for managing presets:

• Loading, saving, and editing preset files (Presets/*.json).
• Displaying preset rules and settings in a tabbed JSON editor.
• Providing the preset selection list (QListWidget) including the "LLM Interpretation" option.
• Refactored: Exposes its selector (selector_container) and JSON editor (json_editor_container) as separate widgets for use by MainWindow.
• Emits preset_selection_changed_signal when the selection changes.

LogConsoleWidget (gui/log_console_widget.py)

This widget displays application logs within the GUI:

• Provides a text area for log messages.
• Integrates with Python's logging system via a custom QtLogHandler.
• Can be shown/hidden via the main window's "View" menu.

LLMEditorWidget (gui/llm_editor_widget.py)

A new widget dedicated to editing LLM settings:

• Provides a tabbed interface ("Prompt Settings", "API Settings") to edit config/llm_settings.json.
• Allows editing the main prompt, managing examples (add/delete/edit JSON), and configuring API details (URL, key, model, temperature, timeout).
• Loads settings via load_settings() and saves them using _save_settings() (which calls configuration.save_llm_config()).
• Placed within MainWindow's QStackedWidget.

UnifiedViewModel (gui/unified_view_model.py)

The UnifiedViewModel implements a QAbstractItemModel for use with Qt's model-view architecture. It is specifically designed to:

• Wrap a list of SourceRule objects and expose their hierarchical structure (Source -> Asset -> File) to a QTreeView (the Unified Hierarchical View).
• Provide methods (data, index, parent, rowCount, columnCount, flags, setData) required by QAbstractItemModel to allow the QTreeView to display the rule hierarchy and support inline editing of specific attributes (e.g., supplier_override, asset_type_override, item_type_override, target_asset_name_override).
• Handle requests for data editing (setData) by validating input and updating the underlying RuleHierarchyModel. Note: Complex restructuring logic (e.g., moving files between assets when target_asset_name_override changes) is now delegated to the AssetRestructureHandler.
• Determine row background colors based on the asset_type and item_type/item_type_override using color metadata from the Configuration.
• Hold the SourceRule data (via RuleHierarchyModel) that is the single source of truth for the GUI's processing rules.
• Cache configuration data (ASSET_TYPE_DEFINITIONS, FILE_TYPE_DEFINITIONS, color maps) during initialization for improved performance in the data() method.
• Includes the update_rules_for_sources method, which intelligently merges new prediction results into the existing model data, preserving user overrides where possible.

RuleHierarchyModel (gui/rule_hierarchy_model.py)

A simpler, non-Qt model used internally by UnifiedViewModel to manage the list of SourceRule objects and provide methods for accessing and modifying the hierarchy.

AssetRestructureHandler (gui/asset_restructure_handler.py)

This handler contains the complex logic required to modify the SourceRule hierarchy when a file's target asset is changed via the GUI's UnifiedViewModel. It:

• Is triggered by a signal (targetAssetOverrideChanged) from the UnifiedViewModel.
• Uses dedicated methods on the RuleHierarchyModel (moveFileRule, createAssetRule, removeAssetRule) to safely move FileRule objects between AssetRules, creating or removing AssetRules as needed.
• Ensures data consistency during these potentially complex restructuring operations.

Delegates (gui/delegates.py)

This module contains custom QStyledItemDelegate implementations used by the Unified Hierarchical View (QTreeView) to provide inline editors for specific data types or rule attributes. Examples include delegates for:

• ComboBoxDelegate: For selecting from predefined lists of allowed asset and file types, sourced from the Configuration (originally from config/app_settings.json).
• LineEditDelegate: For free-form text editing, such as the target_asset_name_override.
• SupplierSearchDelegate: For the "Supplier" column. Provides a QLineEdit with auto-completion suggestions loaded from config/suppliers.json and handles adding/saving new suppliers.

These delegates handle the presentation and editing of data within the tree view cells, interacting with the UnifiedViewModel to get and set data.

Prediction Handlers (gui/)

Prediction logic is handled by classes inheriting from a common base class, running in background threads.

BasePredictionHandler (gui/base_prediction_handler.py)

An abstract base class (QRunnable) for prediction handlers. It defines the common structure and signals (prediction_signal) used by specific predictor implementations. It's designed to be run in a QThreadPool.

RuleBasedPredictionHandler (gui/prediction_handler.py)

This class (inheriting from BasePredictionHandler) is responsible for generating the initial SourceRule hierarchy using predefined rules from presets. It:

• Takes an input source identifier, file list, and Configuration object.
• Analyzes files based on regex patterns and rules defined in the loaded preset.
• Constructs a SourceRule hierarchy with predicted values.
• Emits the prediction_signal with the generated SourceRule object.

LLMPredictionHandler (gui/llm_prediction_handler.py)

An experimental predictor (inheriting from BasePredictionHandler) that uses a Large Language Model (LLM). It:

• Takes an input source identifier, file list, and Configuration object.
• Interacts with the LLMInteractionHandler to send data to the LLM and receive predictions.
• Parses the LLM's JSON response: It expects a specific two-part JSON structure (see 12_LLM_Predictor_Integration.md). It first sanitizes the response (removing comments/markdown) and then parses the JSON.
• Constructs SourceRule: It groups files based on the proposed_asset_group_name from the JSON, assigns the final asset_type using the asset_group_classifications map, and builds the complete SourceRule hierarchy.
• Emits the prediction_signal with the generated SourceRule object or error_signal on failure.

LLMInteractionHandler (gui/llm_interaction_handler.py)

This class now acts as the central manager for LLM prediction tasks:

• Manages the LLM prediction queue and processes items sequentially.
• Loads LLM configuration directly from config/llm_settings.json and config/app_settings.json.
• Instantiates and manages the LLMPredictionHandler and its QThread.
• Handles LLM task state (running/idle) and signals changes to the GUI.
• Receives results/errors from LLMPredictionHandler and emits signals (llm_prediction_ready, llm_prediction_error, llm_status_update, llm_processing_state_changed) to MainWindow.

Utility Modules (utils/)

Common utility functions have been extracted into separate modules:

workspace_utils.py

Contains functions related to managing the processing workspace:

• prepare_processing_workspace: Creates temporary directories, extracts archive files (ZIP, RAR, 7z), and returns the path to the prepared workspace. Used by main.ProcessingTask and monitor.py.

prediction_utils.py

Contains utility functions supporting prediction tasks:

• generate_source_rule_from_archive: A helper function used by monitor.py to perform rule-based prediction directly on an archive file without needing the full GUI setup. It extracts files temporarily, runs prediction logic similar to RuleBasedPredictionHandler, and returns a SourceRule.

Monitor (monitor.py)

The monitor.py script implements the directory monitoring feature. It has been refactored to:

• Use watchdog to detect new archive files in the input directory.
• Use a ThreadPoolExecutor to process detected archives asynchronously in a _process_archive_task function.
• Within the task, it:
  • Loads the necessary Configuration.
  • Calls utils.prediction_utils.generate_source_rule_from_archive to get the SourceRule.
  • Calls utils.workspace_utils.prepare_processing_workspace to set up the workspace.
  • Instantiates and runs the ProcessingEngine.
  • Handles moving the source archive to 'processed' or 'error' directories.
  • Cleans up the workspace.

Summary

These key components, along with the refactored GUI structure and new utility modules, work together to provide the tool's functionality. The architecture emphasizes separation of concerns (configuration, rule generation, processing, UI), utilizes background processing for responsiveness (GUI prediction, Monitor tasks), and relies on the SourceRule object as the central data structure passed between different stages of the workflow.