Rusfort/Asset-Frameworker
1
0
Fork 0
Code Issues 24 Pull Requests Actions Packages Projects 3 Releases Wiki Activity
Asset-Frameworker/Documentation/02_Developer_Guide/01_Architecture.md

5.6 KiB
Raw Blame History

Developer Guide: Architecture

This document provides a high-level overview of the Asset Processor Tool's architecture and core components for developers.

High-Level Architecture

The Asset Processor Tool is designed to process 3D asset source files into a standardized library format. Its high-level architecture consists of:

  1. Core Processing Engine (AssetProcessor): The central component responsible for orchestrating the asset processing pipeline for a single input asset.
  2. Configuration System (Configuration): Handles loading core settings and merging them with supplier-specific rules defined in JSON presets.
  3. Multiple Interfaces: Provides different ways to interact with the tool:
    • Graphical User Interface (GUI)
    • Command-Line Interface (CLI)
    • Directory Monitor for automated processing. These interfaces exchange data structures containing information about each file and asset set being processed.
  4. Optional Integration: Includes scripts and logic for integrating with external software, specifically Blender, to automate material and node group creation.

Hierarchical Rule System

A key addition to the architecture is the Hierarchical Rule System, which provides a dynamic layer of configuration that can override the static settings loaded from presets. This system is represented by a hierarchy of rule objects: SourceRule, AssetRule, and FileRule.

  • SourceRule: Represents rules applied at the top level, typically corresponding to an entire input source (e.g., a ZIP file or folder).
  • AssetRule: Represents rules applied to a specific asset within a source (a source can contain multiple assets).
  • FileRule: Represents rules applied to individual files within an asset.

This hierarchy allows for fine-grained control over processing parameters. When the processing engine needs a configuration value (e.g., a naming convention or image processing setting), it first checks the FileRule for the specific file, then the AssetRule for the asset containing the file, then the SourceRule for the overall source. If a value is not found at any of these dynamic levels, it falls back to the static configuration loaded from the selected preset. This prioritization logic (File > Asset > Source > Static Preset) ensures that the most specific rule available is always used.

Core Components

  • config.py: Defines core, global settings and constants.
  • Presets/*.json: Supplier-specific JSON files defining rules for file interpretation and processing.
  • configuration.py (Configuration class): Loads config.py settings and merges them with a selected preset, pre-compiling regex patterns for efficiency.
  • asset_processor.py (AssetProcessor class): Contains the core logic for processing a single asset through the defined pipeline steps. This component works with data structures containing detailed information about the asset and its files.
  • main.py: The entry point for the Command-Line Interface (CLI). It handles argument parsing, logging, parallel processing orchestration, and triggering Blender scripts. It orchestrates the processing of multiple assets by interacting with the AssetProcessor for individual assets and manages the overall CLI execution flow.
  • gui/: Directory containing modules for the Graphical User Interface (GUI), built with PySide6. The GUI interacts with the core processing logic indirectly via dedicated handler classes (ProcessingHandler, PredictionHandler) running in separate threads. Data structures, such as input paths, configuration details, processing progress updates, and file prediction results, are passed between the GUI, these handlers, and the AssetProcessor using thread-safe mechanisms like Qt signals and slots.
  • monitor.py: Implements the directory monitoring feature using watchdog.
  • blenderscripts/: Contains Python scripts designed to be executed within Blender for post-processing tasks.

Processing Pipeline (Simplified)

The core processing engine (AssetProcessor) executes a series of steps for each asset:

  1. Extraction of input to a temporary workspace.
  2. Classification of files (map, model, extra, ignored, unrecognised) using preset rules.
  3. Determination of base metadata (asset name, category, archetype).
  4. Skip check if output exists and overwrite is not forced.
  5. Processing of maps (resize, format/bit depth conversion, inversion, stats calculation).
  6. Merging of channels based on rules.
  7. Generation of metadata.json file.
  8. Organization of processed files into the final output structure.
  9. Cleanup of the temporary workspace.
  10. (Optional) Execution of Blender scripts for post-processing.

This architecture allows for a modular design, separating configuration, core processing logic, and different interfaces. The data structures flowing through this pipeline carry detailed information about individual files and asset sets. Examples include lists of input file paths, configuration objects, dictionaries summarizing processing outcomes, and detailed lists of file predictions. Parallel processing is utilized for efficiency, and background threads keep the GUI responsive.

Note on Data Passing: Major changes to the data passing mechanisms between the GUI, Main (CLI orchestration), and AssetProcessor modules are currently being planned. These changes are expected to involve new data structures and updated interaction patterns to convey detailed specifications for datasets/asset-sets and processing instructions for individual files. The documentation in this section, particularly regarding data flow, will require significant review and updates once the plan for these changes is finalized.