Rusfort/Asset-Frameworker
1
0
Fork 0
Code Issues 24 Pull Requests Actions Packages Projects 3 Releases Wiki Activity

Various Issue Completions

Browse Source 
#10
#9
#8
#7
#6
#5
This commit is contained in:
Rusfort
2025-05-01 15:44:40 +02:00
parent 26e1a769ce
commit a5be50b587
31 changed files with 589 additions and 123 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
Documentation/02_Developer_Guide/01_Architecture.md
View File

@@ -7,12 +7,12 @@ This document provides a high-level overview of the Asset Processor Tool's archi
The Asset Processor Tool is designed to process 3D asset source files into a standardized library format. Its high-level architecture consists of:
1. **Core Processing Engine (`processing_engine.py`):** The primary component responsible for executing the asset processing pipeline for a single input asset based on a provided `SourceRule` object and static configuration. The older `asset_processor.py` remains in the codebase for reference but is no longer used in the main processing flow.
2. **Configuration System (`Configuration`):** Handles loading core settings and merging them with supplier-specific rules defined in JSON presets.
2. **Configuration System (`Configuration`):** Handles loading core settings (including centralized type definitions) and merging them with supplier-specific rules defined in JSON presets and the persistent `config/suppliers.json` file.
3. **Multiple Interfaces:** Provides different ways to interact with the tool:
* Graphical User Interface (GUI)
* Command-Line Interface (CLI)
* Directory Monitor for automated processing.
The GUI now acts as the primary source of truth for processing rules, generating and managing the `SourceRule` hierarchy before sending it to the processing engine. The CLI and Monitor interfaces can also generate `SourceRule` objects to bypass the GUI for automated workflows.
The GUI now acts as the primary source of truth for processing rules, generating and managing the `SourceRule` hierarchy before sending it to the processing engine. It also accumulates prediction results from multiple input sources before updating the view. The CLI and Monitor interfaces can also generate `SourceRule` objects to bypass the GUI for automated workflows.
4. **Optional Integration:** Includes scripts and logic for integrating with external software, specifically Blender, to automate material and node group creation.
## Hierarchical Rule System
@@ -27,14 +27,15 @@ This hierarchy allows for fine-grained control over processing parameters. The G
## Core Components
* `config.py`: Defines core, global settings, constants, and centralized lists of allowed asset and file types.
* `config.py`: Defines core, global settings, constants, and centralized definitions for allowed asset and file types (`ASSET_TYPE_DEFINITIONS`, `FILE_TYPE_DEFINITIONS`), including metadata like colors and descriptions.
* `config/suppliers.json`: A persistent JSON file storing known supplier names for GUI auto-completion.
* `Presets/*.json`: Supplier-specific JSON files defining rules for file interpretation and initial prediction.
* `configuration.py` (`Configuration` class): Loads `config.py` settings and merges them with a selected preset, pre-compiling regex patterns for efficiency. This static configuration is used by the processing engine.
* `rule_structure.py`: Defines the `SourceRule`, `AssetRule`, and `FileRule` dataclasses used to represent the hierarchical processing rules.
* `gui/`: Directory containing modules for the Graphical User Interface (GUI), built with PySide6. The GUI is responsible for generating and managing the `SourceRule` hierarchy via the Unified View and interacting with background handlers (`ProcessingHandler`, `PredictionHandler`).
* `unified_view_model.py`: Implements the `QAbstractItemModel` for the Unified Hierarchical View, holding the `SourceRule` data and handling inline editing.
* `delegates.py`: Contains custom `QStyledItemDelegate` implementations for inline editing in the Unified View.
* `prediction_handler.py`: Generates the initial `SourceRule` hierarchy with predicted values based on input files and the selected preset.
* `gui/`: Directory containing modules for the Graphical User Interface (GUI), built with PySide6. The GUI is responsible for generating and managing the `SourceRule` hierarchy via the Unified View, accumulating prediction results, and interacting with background handlers (`ProcessingHandler`, `PredictionHandler`).
* `unified_view_model.py`: Implements the `QAbstractItemModel` for the Unified Hierarchical View, holding the `SourceRule` data, handling inline editing (including direct model restructuring for `target_asset_name_override`), and managing row coloring based on config definitions.
* `delegates.py`: Contains custom `QStyledItemDelegate` implementations for inline editing in the Unified View, including the new `SupplierSearchDelegate` for supplier name auto-completion and management.
* `prediction_handler.py`: Generates the initial `SourceRule` hierarchy with predicted values for a single input source based on its files and the selected preset.
* `processing_engine.py` (`ProcessingEngine` class): The new core component that executes the processing pipeline for a single `SourceRule` object using the static `Configuration`. It contains no internal prediction or fallback logic.
* `asset_processor.py` (`AssetProcessor` class): The older processing engine, kept for reference but not used in the main processing flow.
* `main.py`: The entry point for the Command-Line Interface (CLI). It handles argument parsing, logging, parallel processing orchestration, and triggering Blender scripts. It now orchestrates processing by passing `SourceRule` objects to the `ProcessingEngine`.

8
Documentation/02_Developer_Guide/02_Codebase_Structure.md
View File

@@ -5,7 +5,9 @@ This document outlines the key files and directories within the Asset Processor
```
Asset_processor_tool/
├── asset_processor.py # Older core class, kept for reference (not used in main flow)
├── config.py # Core settings, constants, and allowed types
├── config.py # Core settings, constants, and definitions for allowed asset/file types
├── config/ # Directory for configuration files
│ └── suppliers.json # Persistent list of known supplier names for GUI auto-completion
├── configuration.py # Class for loading and accessing configuration (merges config.py and presets)
├── detailed_documentation_plan.md # (Existing file, potentially outdated)
├── Dockerfile # Instructions for building the Docker container image
@@ -50,7 +52,9 @@ Asset_processor_tool/
**Key Files and Directories:**
* `asset_processor.py`: Contains the older `AssetProcessor` class. It is kept for reference but is no longer used in the main processing flow orchestrated by `main.py` or the GUI.
* `config.py`: Stores global default settings, constants, core rules, and centralized lists of `ALLOWED_ASSET_TYPES` and `ALLOWED_FILE_TYPES` used for validation and GUI dropdowns.
* `config.py`: Stores global default settings, constants, core rules, and centralized definitions for allowed asset and file types (`ASSET_TYPE_DEFINITIONS`, `FILE_TYPE_DEFINITIONS`) used for validation, GUI dropdowns, and coloring.
* `config/`: Directory containing configuration files, such as `suppliers.json`.
* `config/suppliers.json`: A JSON file storing a persistent list of known supplier names, used by the GUI's `SupplierSearchDelegate` for auto-completion.
* `configuration.py`: Defines the `Configuration` class. Responsible for loading core settings from `config.py` and merging them with a specified preset JSON file (`Presets/*.json`). Pre-compiles regex patterns from presets for efficiency. An instance of this class is passed to the `ProcessingEngine`.
* `rule_structure.py`: Defines the `SourceRule`, `AssetRule`, and `FileRule` dataclasses. These structures represent the hierarchical processing rules and are the primary data contract passed from the GUI/prediction layer to the processing engine.
* `processing_engine.py`: Defines the new `ProcessingEngine` class. This is the core component that executes the processing pipeline for a single asset based *solely* on a provided `SourceRule` object and the static `Configuration`. It contains no internal prediction or fallback logic.

19
Documentation/02_Developer_Guide/03_Key_Components.md
View File

@@ -26,7 +26,7 @@ This module defines the data structures used to represent the hierarchical proce
* `AssetRule`: A dataclass representing rules applied at the asset level. It contains nested `FileRule` objects.
* `FileRule`: A dataclass representing rules applied at the file level.
These classes hold specific rule parameters (e.g., `supplier_identifier`, `asset_type`, `map_type_override`). Attributes like `asset_type` and `item_type_override` now use string types, which are validated against centralized lists in `config.py`. These structures support serialization (Pickle, JSON) to allow them to be passed between different parts of the application, including across process boundaries.
These classes hold specific rule parameters (e.g., `supplier_identifier`, `asset_type`, `asset_type_override`, `item_type`, `item_type_override`, `target_asset_name_override`). Attributes like `asset_type` and `item_type_override` now use string types, which are validated against centralized lists in `config.py`. These structures support serialization (Pickle, JSON) to allow them to be passed between different parts of the application, including across process boundaries.
## `Configuration` (`configuration.py`)
@@ -58,15 +58,18 @@ The `MainWindow` class is the main application window for the Graphical User Int
The `UnifiedViewModel` implements a `QAbstractItemModel` for use with Qt's model-view architecture. It is specifically designed to:
* Wrap a list of `SourceRule` objects and expose their hierarchical structure (Source -> Asset -> File) to a `QTreeView` (the Unified Hierarchical View).
* Provide methods (`data`, `index`, `parent`, `rowCount`, `columnCount`, `flags`, `setData`) required by `QAbstractItemModel` to allow the `QTreeView` to display the rule hierarchy and support inline editing of specific attributes (e.g., asset type, item type override, target asset name override).
* Provide methods (`data`, `index`, `parent`, `rowCount`, `columnCount`, `flags`, `setData`) required by `QAbstractItemModel` to allow the `QTreeView` to display the rule hierarchy and support inline editing of specific attributes (e.g., `supplier_override`, `asset_type_override`, `item_type_override`, `target_asset_name_override`).
* Handle the direct restructuring of the underlying `SourceRule` hierarchy when `target_asset_name_override` is edited, including moving `FileRule`s and managing `AssetRule` creation/deletion.
* Determine row background colors based on the `asset_type` and `item_type`/`item_type_override` using color metadata from `config.py`.
* Hold the `SourceRule` data that is the single source of truth for the GUI's processing rules.
## `Delegates` (`gui/delegates.py`)
This module contains custom `QStyledItemDelegate` implementations used by the Unified Hierarchical View (`QTreeView`) to provide inline editors for specific data types or rule attributes. Examples include delegates for:
* `QComboBox`: For selecting from a predefined list of options (e.g., allowed asset types, allowed file types sourced from `config.py`).
* `QLineEdit`: For free-form text editing (e.g., target asset name override, supplier identifier override).
* `ComboBoxDelegate`: For selecting from predefined lists of allowed asset and file types, sourced from `config.py`.
* `LineEditDelegate`: For free-form text editing, such as the `target_asset_name_override`.
* `SupplierSearchDelegate`: A new delegate for the "Supplier" column. It provides a `QLineEdit` with auto-completion suggestions loaded from `config/suppliers.json` and handles adding/saving new suppliers.
These delegates handle the presentation and editing of data within the tree view cells, interacting with the `UnifiedViewModel` to get and set data.
@@ -84,10 +87,10 @@ The `ProcessingHandler` class is designed to run in a separate `QThread` within
The `PredictionHandler` class also runs in a separate `QThread` in the GUI. It is responsible for generating the initial `SourceRule` hierarchy with predicted values based on the input files and the selected preset. It:
* Takes a list of input files and the selected preset name as input.
* Uses logic (including accessing preset rules and `config.py`'s allowed types) to analyze files and predict initial values for overridable fields in the `SourceRule`, `AssetRule`, and `FileRule` objects (e.g., asset type, item type, target asset name).
* Constructs the complete `SourceRule` hierarchy based on these predictions.
* Emits a signal (`rule_hierarchy_ready`) with the generated `List[SourceRule]` to the `MainWindow` to populate the Unified Hierarchical View.
* Takes an input source identifier (path), a list of files within that source, and the selected preset name as input.
* Uses logic (including accessing preset rules and `config.py`'s allowed types) to analyze files and predict initial values for overridable fields in the `SourceRule`, `AssetRule`, and `FileRule` objects (e.g., `supplier_identifier`, `asset_type`, `item_type`, `target_asset_name_override`).
* Constructs a `SourceRule` hierarchy for the single input source.
* Emits a signal (`rule_hierarchy_ready`) with the input source identifier and the generated `SourceRule` object (within a list) to the `MainWindow` for accumulation and eventual population of the `UnifiedViewModel`.
## `ZipHandler` (`monitor.py`)

18
Documentation/02_Developer_Guide/04_Configuration_System_and_Presets.md
View File

@@ -6,9 +6,25 @@ This document provides technical details about the configuration system and the
The tool utilizes a two-tiered configuration system:
1. **Core Settings (`config.py`):** This Python module defines global default settings, constants, and core rules that apply generally across different asset sources. Examples include default output paths, standard image resolutions, map merge rules, output format rules, Blender executable paths, and default map types.
1. **Core Settings (`config.py`):** This Python module defines global default settings, constants, and core rules that apply generally across different asset sources. Examples include default output paths, standard image resolutions, map merge rules, output format rules, Blender executable paths, and default map types. It also now centrally defines metadata for allowed asset and file types.
2. **Preset Files (`Presets/*.json`):** These JSON files define supplier-specific rules and overrides. They contain patterns (often regular expressions) to interpret filenames, classify map types, handle variants, define naming conventions, and specify other source-specific behaviors.
## Core Definitions in `config.py`
The `config.py` file now uses dictionary structures to define allowed asset and file types and their associated metadata:
* **`ASSET_TYPE_DEFINITIONS`:** A dictionary where keys are the standard asset type names (e.g., `"Surface"`, `"Model"`, `"Decal"`) and values are dictionaries containing metadata such as `description` and `color` (used for GUI coloring).
* **`FILE_TYPE_DEFINITIONS`:** A dictionary where keys are the standard file/item type names (e.g., `"MAP_COL"`, `"MAP_NRM"`, `"MODEL"`, `"EXTRA"`) and values are dictionaries containing metadata such as `description`, `color` (used for GUI coloring), and `examples` (example filename patterns).
These dictionaries serve as the central source of truth for valid types and their associated display information throughout the application, particularly in the GUI for dropdowns and coloring.
## Supplier Management (`config/suppliers.json`)
A new file, `config/suppliers.json`, is used to store a persistent list of known supplier names. This file is a simple JSON array of strings.
* **Purpose:** Provides a list of suggestions for the "Supplier" field in the GUI's Unified View, enabling auto-completion.
* **Management:** The GUI's `SupplierSearchDelegate` is responsible for loading this list on startup, adding new, unique supplier names entered by the user, and saving the updated list back to the file.
## `Configuration` Class (`configuration.py`)
The `Configuration` class is responsible for loading, merging, and preparing the configuration settings for use by the `AssetProcessor`.

10
Documentation/02_Developer_Guide/06_GUI_Internals.md
View File

@@ -16,6 +16,7 @@ The `MainWindow` class is the central component of the GUI application. It is re
* Connecting user interactions (button clicks, drag-and-drop events, edits in the Unified View) to corresponding methods (slots) within the `MainWindow` or other handler classes.
* Managing the display of application logs in the UI console using a custom `QtLogHandler`.
* Interacting with background handlers (`ProcessingHandler`, `PredictionHandler`) via Qt signals and slots to ensure thread-safe updates to the UI during long-running operations.
* Accumulating prediction results from the `PredictionHandler` for multiple input sources before updating the `UnifiedViewModel`.
* Receiving the initial `SourceRule` hierarchy from the `PredictionHandler` and populating the `UnifiedViewModel`.
* Sending the final, potentially user-modified, `SourceRule` list to `main.py` to initiate processing via the `ProcessingEngine`.
@@ -42,9 +43,14 @@ The GUI includes an integrated preset editor panel. This allows users to interac
The core of the GUI's rule editing interface is the Unified Hierarchical View, implemented using a `QTreeView` with a custom model and delegates.
* **`Unified View Model` (`gui/unified_view_model.py`):** This class implements a `QAbstractItemModel` to expose the structure of a list of `SourceRule` objects (Source -> Asset -> File) to the `QTreeView`. It holds the `SourceRule` data that is the single source of truth for the GUI's processing rules. It provides data and flags for display in multiple columns and supports inline editing of specific rule attributes (e.g., asset type, item type override, target asset name override) by interacting with delegates.
* **`Delegates` (`gui/delegates.py`):** This module contains custom `QStyledItemDelegate` implementations used by the `QTreeView` to provide inline editors for specific data types or rule attributes. Examples include delegates for `QComboBox` (for selecting from allowed types sourced from `config.py`) and `QLineEdit` (for free-form text editing). These delegates handle the presentation and editing of data within the tree view cells, interacting with the `UnifiedViewModel` to get and set data.
* **Direct Model Restructuring:** The `setData` method now includes logic to directly restructure the underlying `SourceRule` hierarchy when the `target_asset_name_override` field of a `FileRule` is edited. This involves moving the `FileRule` to a different `AssetRule` (creating a new one if necessary) and removing the old `AssetRule` if it becomes empty. This replaces the previous mechanism of re-running prediction after an edit.
* **Row Coloring:** Row background colors are dynamically determined based on the `asset_type` (for `AssetRule`s) and `item_type` or `item_type_override` (for `FileRule`s), using the color metadata defined in the `ASSET_TYPE_DEFINITIONS` and `FILE_TYPE_DEFINITIONS` dictionaries in `config.py`. `SourceRule` rows have a fixed color.
* **`Delegates` (`gui/delegates.py`):** This module contains custom `QStyledItemDelegate` implementations used by the `QTreeView` to provide inline editors for specific data types or rule attributes.
* **`ComboBoxDelegate`:** Used for selecting from predefined lists (e.g., allowed asset types, allowed file types sourced from `config.py`).
* **`LineEditDelegate`:** Used for free-form text editing (e.g., target asset name override).
* **`SupplierSearchDelegate`:** A new delegate used for the "Supplier" column. It provides a `QLineEdit` with auto-completion suggestions loaded from `config/suppliers.json`. It also handles adding new, unique supplier names entered by the user to the list and saving the updated list back to the JSON file.
The `PredictionHandler` generates the initial `SourceRule` hierarchy, which is then set on the `UnifiedViewModel`. The `QTreeView` displays this model, allowing users to navigate the hierarchy and make inline edits to the rule attributes. Edits made in the view directly modify the attributes of the underlying rule objects in the `SourceRule` hierarchy held by the model.
The `PredictionHandler` generates the initial `SourceRule` hierarchy, which is then set on the `UnifiedViewModel`. The `QTreeView` displays this model, allowing users to navigate the hierarchy and make inline edits to the rule attributes. Edits made in the view directly modify the attributes of the underlying rule objects in the `SourceRule` hierarchy held by the model, with the `UnifiedViewModel` handling the necessary model restructuring and signal emission for view updates.
**Data Flow Diagram (GUI Rule Management):**