Config Updates - User settings - Saving Methods
This commit is contained in:
@@ -2,43 +2,125 @@
|
||||
|
||||
This document provides technical details about the configuration system and the structure of preset files for developers working on the Asset Processor Tool.
|
||||
|
||||
## Configuration Flow
|
||||
## Configuration System Overview
|
||||
|
||||
The tool utilizes a two-tiered configuration system managed by the `configuration.py` module:
|
||||
The tool's configuration is managed by the `configuration.py` module and loaded from several JSON files, providing a layered approach for defaults, user overrides, definitions, and source-specific presets.
|
||||
|
||||
1. **Application Settings (`config/app_settings.json`):** This JSON file defines the core global default settings, constants, and rules that apply generally across different asset sources (e.g., the global `OUTPUT_DIRECTORY_PATTERN` and `OUTPUT_FILENAME_PATTERN`, standard image resolutions, map merge rules, output format rules, Blender paths, `FILE_TYPE_DEFINITIONS`, `ASSET_TYPE_DEFINITIONS`). See the [User Guide: Output Structure](../01_User_Guide/09_Output_Structure.md#available-tokens) for a list of available tokens for these patterns.
|
||||
* **`FILE_TYPE_DEFINITIONS` Enhancements:**
|
||||
* **`keybind` Property:** Each file type object within `FILE_TYPE_DEFINITIONS` can now optionally include a `keybind` property. This property accepts a single character string (e.g., `"C"`, `"R"`) representing the keyboard key. In the GUI, this key (typically combined with `Ctrl`, or standalone like `F2` for asset naming) is used as a shortcut to set or toggle the corresponding file type for selected items in the Preview Table.
|
||||
*Example:*
|
||||
```json
|
||||
"MAP_COL": {
|
||||
"description": "Color/Albedo Map",
|
||||
"color": [200, 200, 200],
|
||||
"examples": ["albedo", "col", "basecolor"],
|
||||
"standard_type": "COL",
|
||||
"bit_depth_rule": "respect",
|
||||
"is_grayscale": false,
|
||||
"keybind": "C"
|
||||
},
|
||||
```
|
||||
* **New File Type `MAP_GLOSS`:** A new standard file type, `MAP_GLOSS`, has been added. It is typically configured as follows:
|
||||
*Example:*
|
||||
```json
|
||||
"MAP_GLOSS": {
|
||||
"description": "Glossiness Map",
|
||||
"color": [180, 180, 220],
|
||||
"examples": ["gloss", "gls"],
|
||||
"standard_type": "GLOSS",
|
||||
"bit_depth_rule": "respect",
|
||||
"is_grayscale": true,
|
||||
"keybind": "R"
|
||||
}
|
||||
```
|
||||
Note: The `keybind` "R" for `MAP_GLOSS` is often shared with `MAP_ROUGH` to allow toggling between them.
|
||||
2. **LLM Settings (`config/llm_settings.json`):** This JSON file contains settings specifically related to the LLM predictor, such as the API endpoint, model name, prompt template, and examples. These settings can be edited through the GUI using the `LLMEditorWidget`.
|
||||
3. **Preset Files (`Presets/*.json`):** These JSON files define supplier-specific rules and overrides. They contain patterns to interpret filenames, classify map types, handle variants, define naming conventions, and specify other source-specific behaviors.
|
||||
### Configuration Files
|
||||
|
||||
The `configuration.py` module contains the `Configuration` class (for loading/merging settings for processing) and standalone functions like `load_base_config()` (for accessing `app_settings.json` directly) and `save_llm_config()` / `save_base_config()` (for writing settings back to files). Note that the old `config.py` file has been deleted.
|
||||
1. **Application Settings (`config/app_settings.json`):** This JSON file defines the core global default settings, constants, and rules that apply generally across different asset sources (e.g., the global `OUTPUT_DIRECTORY_PATTERN` and `OUTPUT_FILENAME_PATTERN`, standard image resolutions, map merge rules, output format rules, Blender paths). See the [User Guide: Output Structure](../01_User_Guide/09_Output_Structure.md#available-tokens) for a list of available tokens for these patterns.
|
||||
* *Note:* `ASSET_TYPE_DEFINITIONS` and `FILE_TYPE_DEFINITIONS` are no longer stored here; they have been moved to dedicated files.
|
||||
|
||||
2. **User Settings (`config/user_settings.json`):** This optional JSON file allows users to override specific settings defined in `config/app_settings.json`. If this file exists, its values for corresponding keys will take precedence over the base application settings. This file is primarily managed through the GUI's Configuration Editor.
|
||||
|
||||
3. **Asset Type Definitions (`config/asset_type_definitions.json`):** This dedicated JSON file contains the definitions for different asset types (e.g., Surface, Model, Decal), including their descriptions, colors, and examples.
|
||||
|
||||
4. **File Type Definitions (`config/file_type_definitions.json`):** This dedicated JSON file contains the definitions for different file types (specifically texture maps and models), including descriptions, colors, examples, standard aliases, bit depth rules, grayscale flags, and GUI keybinds.
|
||||
* **`keybind` Property:** Each file type object within `FILE_TYPE_DEFINITIONS` can optionally include a `keybind` property. This property accepts a single character string (e.g., `"C"`, `"R"`) representing the keyboard key. In the GUI, this key (typically combined with `Ctrl`, or standalone like `F2` for asset naming) is used as a shortcut to set or toggle the corresponding file type for selected items in the Preview Table.
|
||||
*Example:*
|
||||
```json
|
||||
"MAP_COL": {
|
||||
"description": "Color/Albedo Map",
|
||||
"color": [200, 200, 200],
|
||||
"examples": ["albedo", "col", "basecolor"],
|
||||
"standard_type": "COL",
|
||||
"bit_depth_rule": "respect",
|
||||
"is_grayscale": false,
|
||||
"keybind": "C"
|
||||
},
|
||||
```
|
||||
* **New File Type `MAP_GLOSS`:** A standard file type, `MAP_GLOSS`, is defined here.
|
||||
*Example:*
|
||||
```json
|
||||
"MAP_GLOSS": {
|
||||
"description": "Glossiness Map",
|
||||
"color": [180, 180, 220],
|
||||
"examples": ["gloss", "gls"],
|
||||
"standard_type": "GLOSS",
|
||||
"bit_depth_rule": "respect",
|
||||
"is_grayscale": true,
|
||||
"keybind": "R"
|
||||
}
|
||||
```
|
||||
Note: The `keybind` "R" for `MAP_GLOSS` is often shared with `MAP_ROUGH` to allow toggling between them.
|
||||
|
||||
5. **LLM Settings (`config/llm_settings.json`):** This JSON file contains settings specifically related to the LLM predictor, such as the API endpoint, model name, prompt template, and examples. These settings can be edited through the GUI using the `LLMEditorWidget`.
|
||||
|
||||
6. **Preset Files (`Presets/*.json`):** These JSON files define supplier-specific rules and overrides. They contain patterns to interpret filenames, classify map types, handle variants, define naming conventions, and specify other source-specific behaviors. Preset settings override values from `app_settings.json` and `user_settings.json` where applicable.
|
||||
|
||||
### Configuration Loading and Access
|
||||
|
||||
The `configuration.py` module contains the `Configuration` class and standalone functions for loading and saving settings.
|
||||
|
||||
* **`Configuration` Class:** This is the primary class used by the processing engine and other core components. When initialized with a `preset_name`, it loads settings in the following order, with later files overriding earlier ones for shared keys:
|
||||
1. `config/app_settings.json` (Base Defaults)
|
||||
2. `config/user_settings.json` (User Overrides - if exists)
|
||||
3. `config/asset_type_definitions.json` (Asset Type Definitions)
|
||||
4. `config/file_type_definitions.json` (File Type Definitions)
|
||||
5. `config/llm_settings.json` (LLM Settings)
|
||||
6. `Presets/{preset_name}.json` (Preset Overrides)
|
||||
|
||||
The loaded settings are merged into internal dictionaries, and most are accessible via instance properties (e.g., `config.output_base_dir`, `config.llm_endpoint_url`, `config.get_asset_type_definitions()`). Regex patterns defined in the merged configuration are pre-compiled for performance.
|
||||
|
||||
* **`load_base_config()` function:** This standalone function is primarily used by the GUI for initial setup and displaying default/user-overridden settings before a specific preset is selected. It loads and merges the following files:
|
||||
1. `config/app_settings.json`
|
||||
2. `config/user_settings.json` (if exists)
|
||||
3. `config/asset_type_definitions.json`
|
||||
4. `config/file_type_definitions.json`
|
||||
|
||||
It returns a single dictionary containing the combined settings and definitions.
|
||||
|
||||
* **Saving Functions:**
|
||||
* `save_base_config(settings_dict)`: Saves the provided dictionary to `config/app_settings.json`. (Used less frequently now for user-driven saves).
|
||||
* `save_user_config(settings_dict)`: Saves the provided dictionary to `config/user_settings.json`. Used by `ConfigEditorDialog`.
|
||||
* `save_llm_config(settings_dict)`: Saves the provided dictionary to `config/llm_settings.json`. Used by `LLMEditorWidget`.
|
||||
|
||||
## Supplier Management (`config/suppliers.json`)
|
||||
|
||||
A file, `config/suppliers.json`, is used to store a persistent list of known supplier names. This file is a simple JSON array of strings.
|
||||
|
||||
* **Purpose:** Provides a list of suggestions for the "Supplier" field in the GUI's Unified View, enabling auto-completion.
|
||||
* **Management:** The GUI's `SupplierSearchDelegate` is responsible for loading this list on startup, adding new, unique supplier names entered by the user, and saving the updated list back to the file.
|
||||
|
||||
## GUI Configuration Editors
|
||||
|
||||
The GUI provides dedicated editors for modifying configuration files:
|
||||
|
||||
* **`ConfigEditorDialog` (`gui/config_editor_dialog.py`):** Edits user-configurable application settings.
|
||||
* **`LLMEditorWidget` (`gui/llm_editor_widget.py`):** Edits the LLM-specific settings.
|
||||
|
||||
### `ConfigEditorDialog` (`gui/config_editor_dialog.py`)
|
||||
|
||||
The GUI includes a dedicated editor for modifying user-configurable settings. This is implemented in `gui/config_editor_dialog.py`.
|
||||
|
||||
* **Purpose:** Provides a user-friendly interface for viewing the effective application settings (defaults + user overrides + definitions) and editing the user-specific overrides.
|
||||
* **Implementation:** The dialog loads the effective settings using `load_base_config()`. It presents relevant settings in a tabbed layout ("General", "Output & Naming", etc.). When saving, it now performs a **granular save**: it loads the current content of `config/user_settings.json`, identifies only the settings that were changed by the user during the current dialog session (by comparing against the initial state), updates only those specific values in the loaded `user_settings.json` content, and saves the modified content back to `config/user_settings.json` using `save_user_config()`. This preserves any other settings in `user_settings.json` that were not touched. The dialog displays definitions from `asset_type_definitions.json` and `file_type_definitions.json` but does not save changes to these files.
|
||||
* **Limitations:** Currently, editing complex fields like `IMAGE_RESOLUTIONS` or the full details of `MAP_MERGE_RULES` via the UI is not fully supported for saving to `user_settings.json`.
|
||||
|
||||
### `LLMEditorWidget` (`gui/llm_editor_widget.py`)
|
||||
|
||||
* **Purpose:** Provides a user-friendly interface for viewing and editing the LLM settings defined in `config/llm_settings.json`.
|
||||
* **Implementation:** Uses tabs for "Prompt Settings" and "API Settings". Allows editing the prompt, managing examples, and configuring API details. When saving, it also performs a **granular save**: it loads the current content of `config/llm_settings.json`, identifies only the settings changed by the user in the current session, updates only those values, and saves the modified content back to `config/llm_settings.json` using `configuration.save_llm_config()`.
|
||||
|
||||
## Preset File Structure (`Presets/*.json`)
|
||||
|
||||
Preset files are the primary way to adapt the tool to new asset sources. Developers should use `Presets/_template.json` as a starting point. Key fields include:
|
||||
|
||||
* `supplier_name`: The name of the asset source (e.g., `"Poliigon"`). Used for output directory naming.
|
||||
* `map_type_mapping`: A list of dictionaries, each mapping source filename patterns/keywords to a specific file type. The `target_type` for this mapping **must** be a key from the `FILE_TYPE_DEFINITIONS` now located in `config/file_type_definitions.json`.
|
||||
* `target_type`: The specific file type key from `FILE_TYPE_DEFINITIONS` (e.g., `"MAP_COL"`, `"MAP_NORM_GL"`, `"MAP_RGH"`). This replaces previous alias-based systems. The common aliases like "COL" or "NRM" are now derived from the `standard_type` property within `FILE_TYPE_DEFINITIONS` but are not used directly for `target_type`.
|
||||
* `keywords`: A list of filename patterns (regex or fnmatch-style wildcards) used to identify this map type. The order of keywords within this list, and the order of dictionaries in the `map_type_mapping` list, determines the priority for assigning variant suffixes (`-1`, `-2`, etc.) when multiple files match the same `target_type`.
|
||||
* `bit_depth_variants`: A dictionary mapping standard map types (e.g., `"NRM"`) to a pattern identifying its high bit-depth variant (e.g., `"*_NRM16*.tif"`). Files matching these patterns are prioritized over their standard counterparts.
|
||||
* `map_bit_depth_rules`: Defines how to handle the bit depth of source maps. Can specify a default behavior (`"respect"` or `"force_8bit"`) and overrides for specific map types.
|
||||
* `model_patterns`: A list of regex patterns to identify model files (e.g., `".*\\.fbx"`, `".*\\.obj"`).
|
||||
* `move_to_extra_patterns`: A list of regex patterns for files that should be moved directly to the `Extra/` output subdirectory without further processing.
|
||||
* `source_naming_convention`: Rules for extracting the base asset name and potentially the archetype from source filenames or directory structures (e.g., using separators and indices).
|
||||
* `asset_category_rules`: Keywords or patterns used to determine the asset category (e.g., identifying `"Decal"` based on keywords).
|
||||
* `archetype_rules`: Keywords or patterns used to determine the asset archetype (e.g., identifying `"Wood"` or `"Metal"`).
|
||||
|
||||
Careful definition of these patterns and rules, especially the regex in `map_type_mapping`, `bit_depth_variants`, `model_patterns`, and `move_to_extra_patterns`, is essential for correct asset processing.
|
||||
|
||||
**Note on Data Passing:** As mentioned in the Architecture documentation, major changes to the data passing mechanisms between the GUI, Main (CLI orchestration), and `AssetProcessor` modules are currently being planned. The descriptions of how configuration data is handled and passed within this document reflect the current state and will require review and updates once the plan for these changes is finalized.
|
||||
|
||||
## Supplier Management (`config/suppliers.json`)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user