Preferences, Config.py Migration, Bug Fixes, and Documention updates
This commit is contained in:
@@ -27,16 +27,16 @@ This hierarchy allows for fine-grained control over processing parameters. The G
|
||||
|
||||
## Core Components
|
||||
|
||||
* `config.py`: Defines core, global settings, constants, and centralized definitions for allowed asset and file types (`ASSET_TYPE_DEFINITIONS`, `FILE_TYPE_DEFINITIONS`), including metadata like colors and descriptions.
|
||||
* `config/app_settings.json`: Defines core, global settings, constants, and centralized definitions for allowed asset and file types (`ASSET_TYPE_DEFINITIONS`, `FILE_TYPE_DEFINITIONS`), including metadata like colors and descriptions. This replaces the old `config.py` file.
|
||||
* `config/suppliers.json`: A persistent JSON file storing known supplier names for GUI auto-completion.
|
||||
* `Presets/*.json`: Supplier-specific JSON files defining rules for file interpretation and initial prediction.
|
||||
* `configuration.py` (`Configuration` class): Loads `config.py` settings and merges them with a selected preset, pre-compiling regex patterns for efficiency. This static configuration is used by the processing engine.
|
||||
* `configuration.py` (`Configuration` class): Loads `config/app_settings.json` settings and merges them with a selected preset, pre-compiling regex patterns for efficiency. This static configuration is used by the processing engine.
|
||||
* `rule_structure.py`: Defines the `SourceRule`, `AssetRule`, and `FileRule` dataclasses used to represent the hierarchical processing rules.
|
||||
* `gui/`: Directory containing modules for the Graphical User Interface (GUI), built with PySide6. The GUI is responsible for generating and managing the `SourceRule` hierarchy via the Unified View, accumulating prediction results, and interacting with background handlers (`ProcessingHandler`, `PredictionHandler`).
|
||||
* `unified_view_model.py`: Implements the `QAbstractItemModel` for the Unified Hierarchical View, holding the `SourceRule` data, handling inline editing (including direct model restructuring for `target_asset_name_override`), and managing row coloring based on config definitions.
|
||||
* `delegates.py`: Contains custom `QStyledItemDelegate` implementations for inline editing in the Unified View, including the new `SupplierSearchDelegate` for supplier name auto-completion and management.
|
||||
* `prediction_handler.py`: Generates the initial `SourceRule` hierarchy with predicted values for a single input source based on its files and the selected preset.
|
||||
* `processing_engine.py` (`ProcessingEngine` class): The new core component that executes the processing pipeline for a single `SourceRule` object using the static `Configuration`. It contains no internal prediction or fallback logic.
|
||||
* `prediction_handler.py`: Generates the initial `SourceRule` hierarchy with predicted values for a single input source based on its files and the selected preset. It uses the `"standard_type"` from the configuration's `FILE_TYPE_DEFINITIONS` to populate `FileRule.standard_map_type` and implements a two-pass classification logic to handle and prioritize bit-depth variants (e.g., `_DISP16_` vs `_DISP_`).
|
||||
* `processing_engine.py` (`ProcessingEngine` class): The new core component that executes the processing pipeline for a single `SourceRule` object using the static `Configuration`. A new instance is created per task for state isolation. It contains no internal prediction or fallback logic. Supplier overrides from the GUI are correctly preserved and used by the engine for output path generation and metadata.
|
||||
* `asset_processor.py` (`AssetProcessor` class): The older processing engine, kept for reference but not used in the main processing flow.
|
||||
* `main.py`: The entry point for the Command-Line Interface (CLI). It handles argument parsing, logging, parallel processing orchestration, and triggering Blender scripts. It now orchestrates processing by passing `SourceRule` objects to the `ProcessingEngine`.
|
||||
* `monitor.py`: Implements the directory monitoring feature using `watchdog`.
|
||||
|
||||
@@ -6,10 +6,10 @@ This document provides technical details about the configuration system and the
|
||||
|
||||
The tool utilizes a two-tiered configuration system managed by the `configuration.py` module:
|
||||
|
||||
1. **Application Settings (`config/app_settings.json`):** This JSON file defines the core global default settings, constants, and rules that apply generally across different asset sources. Examples include default output paths, standard image resolutions, map merge rules, output format rules, Blender executable paths, and default map types. It also centrally defines metadata for allowed asset and file types.
|
||||
1. **Application Settings (`config/app_settings.json`):** This JSON file defines the core global default settings, constants, and rules that apply generally across different asset sources. Examples include default output paths, standard image resolutions, map merge rules, output format rules, Blender executable paths, and default map types. It also centrally defines metadata for allowed asset and file types. Key sections include `FILE_TYPE_DEFINITIONS`, `ASSET_TYPE_DEFINITIONS`, and `MAP_MERGE_RULES`.
|
||||
2. **Preset Files (`Presets/*.json`):** These JSON files define supplier-specific rules and overrides. They contain patterns (often regular expressions) to interpret filenames, classify map types, handle variants, define naming conventions, and specify other source-specific behaviors.
|
||||
|
||||
The `configuration.py` module is responsible for loading the base settings from `config/app_settings.json` and then merging them with the rules from the selected preset file. Preset values generally override core settings where applicable.
|
||||
The `configuration.py` module is responsible for loading the base settings from `config/app_settings.json` (including loading and saving the JSON content), merging them with the rules from the selected preset file, and providing the base configuration via the `load_base_config()` function. Preset values generally override core settings where applicable. Note that the old `config.py` file has been deleted.
|
||||
|
||||
## Supplier Management (`config/suppliers.json`)
|
||||
|
||||
@@ -37,7 +37,8 @@ An instance of `Configuration` is created within each worker process (`main.proc
|
||||
The GUI includes a dedicated editor for modifying the `config/app_settings.json` file. This is implemented in `gui/config_editor_dialog.py`.
|
||||
|
||||
* **Purpose:** Provides a user-friendly interface for viewing and editing the core application settings defined in `app_settings.json`.
|
||||
* **Implementation:** The dialog loads the JSON content of `app_settings.json`, presents it in an editable format (likely using standard GUI widgets mapped to the JSON structure), and saves the changes back to the file.
|
||||
* **Implementation:** The dialog loads the JSON content of `app_settings.json`, presents it in a tabbed layout ("General", "Output & Naming", etc.) using standard GUI widgets mapped to the JSON structure, and saves the changes back to the file. It supports editing basic fields, tables for definitions (`FILE_TYPE_DEFINITIONS`, `ASSET_TYPE_DEFINITIONS`), and a list/detail view for merge rules (`MAP_MERGE_RULES`). The definitions tables include dynamic color editing features.
|
||||
* **Limitations:** Currently, editing complex fields like `IMAGE_RESOLUTIONS` or the full details of `MAP_MERGE_RULES` via the UI is not fully supported.
|
||||
* **Note:** Changes made through the GUI editor are written directly to `config/app_settings.json` but require an application restart to be loaded and applied by the `Configuration` class.
|
||||
|
||||
## Preset File Structure (`Presets/*.json`)
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Developer Guide: Processing Pipeline
|
||||
|
||||
This document details the step-by-step technical process executed by the `ProcessingEngine` class (`processing_engine.py`) when processing a single asset.
|
||||
This document details the step-by-step technical process executed by the `ProcessingEngine` class (`processing_engine.py`) when processing a single asset. A new instance of `ProcessingEngine` is created for each processing task to ensure state isolation.
|
||||
|
||||
The `ProcessingEngine.process()` method orchestrates the following pipeline based *solely* on the provided `SourceRule` object and the static `Configuration` object passed during engine initialization. It contains no internal prediction, classification, or fallback logic. All necessary overrides and static configuration values are accessed directly from these inputs.
|
||||
|
||||
@@ -21,7 +21,7 @@ The pipeline steps are:
|
||||
* Sorts potential map variants based on the order provided in the `SourceRule` or static configuration.
|
||||
|
||||
4. **Base Metadata Determination (`_determine_base_metadata`, `_determine_single_asset_metadata`)**:
|
||||
* Determines the base asset name, category, and archetype using the explicit values provided in the input `SourceRule` object and the static configuration from the `Configuration` object. Overrides (like `supplier_identifier`, `asset_type`, and `asset_name_override`) are taken directly from the `SourceRule`.
|
||||
* Determines the base asset name, category, and archetype using the explicit values provided in the input `SourceRule` object and the static configuration from the `Configuration` object. Overrides (like `supplier_identifier`, `asset_type`, and `asset_name_override`), including supplier overrides from the GUI, are taken directly from the `SourceRule`.
|
||||
|
||||
5. **Skip Check**:
|
||||
* If the `overwrite` flag (passed during initialization) is `False`, the tool checks if the final output directory for the determined asset name already exists and contains a `metadata.json` file.
|
||||
@@ -57,7 +57,7 @@ The pipeline steps are:
|
||||
* Writes this collected data into the `metadata.json` file within the temporary workspace using `json.dump`.
|
||||
|
||||
9. **Output Organization (`_organize_output_files`)**:
|
||||
* Creates the final structured output directory: `<output_base_dir>/<supplier_name>/<asset_name>/`.
|
||||
* Creates the final structured output directory: `<output_base_dir>/<supplier_name>/<asset_name>/`. The `supplier_name` used here is derived from the `SourceRule`, ensuring that supplier overrides from the GUI are respected in the output path.
|
||||
* Creates subdirectories `Extra/`, `Unrecognised/`, and `Ignored/` within the asset directory.
|
||||
* Moves the processed maps, merged maps, model files, `metadata.json`, and files classified as Extra, Unrecognised, or Ignored from the temporary workspace into their respective locations in the final output directory structure.
|
||||
|
||||
|
||||
@@ -40,12 +40,15 @@ The GUI includes an integrated preset editor panel. This allows users to interac
|
||||
|
||||
## Unified Hierarchical View (`gui/unified_view_model.py`, `gui/delegates.py`)
|
||||
|
||||
The core of the GUI's rule editing interface is the Unified Hierarchical View, implemented using a `QTreeView` with a custom model and delegates.
|
||||
## Unified Hierarchical View (`gui/unified_view_model.py`, `gui/delegates.py`, `gui/main_window.py`)
|
||||
|
||||
The core of the GUI's rule editing interface is the Unified Hierarchical View, implemented using a `QTreeView` with a custom model and delegates. This view is managed within the `MainWindow`.
|
||||
|
||||
* **`Unified View Model` (`gui/unified_view_model.py`):** This class implements a `QAbstractItemModel` to expose the structure of a list of `SourceRule` objects (Source -> Asset -> File) to the `QTreeView`. It holds the `SourceRule` data that is the single source of truth for the GUI's processing rules. It provides data and flags for display in multiple columns and supports inline editing of specific rule attributes (e.g., asset type, item type override, target asset name override) by interacting with delegates.
|
||||
* **Column Order and Resizing:** The view currently displays the following columns in order: Name, Target Asset, Supplier, Asset Type, Item Type. The "Target Asset" column is set to stretch to fill available space, while other columns resize to their contents. The previous "Status" and "Output Path" columns have been removed.
|
||||
* **Direct Model Restructuring:** The `setData` method now includes logic to directly restructure the underlying `SourceRule` hierarchy when the `target_asset_name_override` field of a `FileRule` is edited. This involves moving the `FileRule` to a different `AssetRule` (creating a new one if necessary) and removing the old `AssetRule` if it becomes empty. This replaces the previous mechanism of re-running prediction after an edit.
|
||||
* **Row Coloring:** Row background colors are dynamically determined based on the `asset_type` (for `AssetRule`s) and `item_type` or `item_type_override` (for `FileRule`s), using the color metadata defined in the `ASSET_TYPE_DEFINITIONS` and `FILE_TYPE_DEFINITIONS` dictionaries sourced from the configuration loaded by `configuration.py` (which includes data from `config/app_settings.json`). `SourceRule` rows have a fixed color.
|
||||
* **`Delegates` (`gui/delegates.py`):** This module contains custom `QStyledItemDelegate` implementations used by the `QTreeView` to provide inline editors for specific data types or rule attributes.
|
||||
* **`Delegates` (`gui/delegates.py`):** This module contains custom `QStyledItemDelegate` implementations used by the `QTreeView` to provide inline editors for specific data types or rule attributes.
|
||||
* **`ComboBoxDelegate`:** Used for selecting from predefined lists (e.g., allowed asset types, allowed file types sourced from the configuration loaded by `configuration.py`).
|
||||
* **`LineEditDelegate`:** Used for free-form text editing (e.g., target asset name override).
|
||||
* **`SupplierSearchDelegate`:** A new delegate used for the "Supplier" column. It provides a `QLineEdit` with auto-completion suggestions loaded from `config/suppliers.json`. It also handles adding new, unique supplier names entered by the user to the list and saving the updated list back to the JSON file.
|
||||
@@ -84,7 +87,8 @@ The GUI provides a "Cancel" button to stop ongoing processing. The `ProcessingHa
|
||||
|
||||
A dedicated dialog, implemented in `gui/config_editor_dialog.py`, provides a graphical interface for editing the core application settings stored in `config/app_settings.json`.
|
||||
|
||||
* **Functionality:** This dialog loads the current content of `config/app_settings.json`, presents it in an editable format (likely using standard Qt widgets), and allows the user to save modifications back to the file.
|
||||
* **Functionality:** This dialog loads the current content of `config/app_settings.json` and presents it in a tabbed layout (e.g., "General", "Output & Naming") using standard GUI widgets mapped to the JSON structure. It supports editing basic fields, tables for definitions (`FILE_TYPE_DEFINITIONS`, `ASSET_TYPE_DEFINITIONS`), and a list/detail view for merge rules (`MAP_MERGE_RULES`). The definitions tables include dynamic color editing features.
|
||||
* **Limitations:** Currently, editing complex fields like `IMAGE_RESOLUTIONS` or the full details of `MAP_MERGE_RULES` via the UI is not fully supported.
|
||||
* **Integration:** The `MainWindow` is responsible for creating and displaying an instance of this dialog when the user selects the "Edit" -> "Preferences..." menu option.
|
||||
* **Persistence:** Changes saved via this editor are written directly to the `config/app_settings.json` file, ensuring they persist across application sessions. However, the `Configuration` class loads settings at application startup, so a restart is required for changes made in the editor to take effect in the application's processing logic.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user