Asset-Frameworker/Project Notes/BLENDER_INTEGRATION_PLAN.md

Blender Integration Plan: Node Groups from Processed Assets

Objective: Develop a Python script (blenderscripts/create_nodegroups.py) to run manually inside Blender. This script will scan the output directory generated by the Asset Processor Tool, read metadata.json files, and create/update corresponding PBR node groups in the active Blender file, leveraging the pre-calculated metadata.

Key Principles:

• Leverage Existing Tool Output: Rely entirely on the structured output and metadata.json from the Asset Processor Tool. Avoid reprocessing or recalculating data already available.
• Blender Environment: The script is designed solely for Blender's Python environment (bpy).
• Manual Execution: Users will manually run this script from Blender's Text Editor.
• Target Active File: All operations modify the currently open .blend file.
• Assume Templates: The script will assume node group templates (Template_PBRSET, Template_PBRTYPE) exist in the active file. Error handling will be added if they are missing.
• Focus on Node Groups: The script's scope is limited to creating and updating the node groups, not materials.

Detailed Plan:

1. Script Setup & Configuration:

   • Create a new Python file named create_nodegroups.py (intended location: blenderscripts/).
   • Import necessary modules (bpy, os, json, pathlib).
   • Define user-configurable variables at the top:
     • PROCESSED_ASSET_LIBRARY_ROOT: Path to the root output directory of the Asset Processor Tool.
     • PARENT_TEMPLATE_NAME: Name of the parent node group template (e.g., "Template_PBRSET").
     • CHILD_TEMPLATE_NAME: Name of the child node group template (e.g., "Template_PBRTYPE").
     • ASPECT_RATIO_NODE_LABEL: Label of the Value node in the parent template for aspect ratio correction (e.g., "AspectRatioCorrection").
     • STATS_NODE_PREFIX: Prefix for Combine XYZ nodes storing stats in the parent template (e.g., "Histogram-").
     • ENABLE_MANIFEST: Boolean flag to enable/disable the manifest system (default: True).

2. Manifest Handling:

   • Location: Use a separate JSON file named [ActiveBlendFileName]_manifest.json, located in the same directory as the active .blend file.
   • Loading: Implement load_manifest(context) that finds and reads the manifest JSON file. If not found or invalid, return an empty dictionary.
   • Saving: Implement save_manifest(context, manifest_data) that writes the manifest_data dictionary to the manifest JSON file.
   • Checking: Implement helper functions is_asset_processed(manifest_data, asset_name) and is_map_processed(manifest_data, asset_name, map_type, resolution) to check against the loaded manifest.
   • Updating: Update the manifest dictionary in memory as assets/maps are processed. Save the manifest once at the end of the script.

3. Core Logic - process_library() function:

   • Get Blender context.
   • Load manifest data (if enabled).
   • Validate that PROCESSED_ASSET_LIBRARY_ROOT exists.
   • Validate that template node groups exist in bpy.data.node_groups. Exit gracefully with an error message if not found.
   • Initialize counters (new groups, updated groups, etc.).
   • Scan Directory: Use os.walk or pathlib.rglob to find all metadata.json files within the PROCESSED_ASSET_LIBRARY_ROOT.
   • Iterate Metadata: For each metadata.json found:
     • Parse the JSON data. Extract key information: asset_name, supplier_name, archetype, maps (dictionary of maps with resolutions, paths, stats), aspect_ratio_change_string.
     • Check manifest if asset is already processed (if enabled). Skip if true.
     • Parent Group Handling:
       • Determine target parent group name (e.g., f"PBRSET_{asset_name}").
       • Find existing group or create a copy from PARENT_TEMPLATE_NAME.
       • Mark group as asset (asset_mark()) if not already.
       • Apply Metadata:
         • Find the aspect ratio node using ASPECT_RATIO_NODE_LABEL. Calculate the correction factor based on the aspect_ratio_change_string from metadata (using helper function calculate_factor_from_string) and set the node's default value.
         • For relevant map types (e.g., ROUGH, DISP), find the stats node (STATS_NODE_PREFIX + map type). Set the X, Y, Z inputs using the min, max, mean values stored in the map's metadata entry for the reference resolution.
       • Apply Asset Tags: Use asset_data.tags.new() to add the supplier_name and archetype tags (checking for existence first).
     • Child Group Handling (Iterate through maps in metadata):
       • For each map_type (e.g., "COL", "NRM") and its data in the metadata:
         • Determine target child group name (e.g., f"PBRTYPE_{asset_name}_{map_type}").
         • Find existing child group or create a copy from CHILD_TEMPLATE_NAME.
         • Find the corresponding placeholder node in the parent group (by label matching map_type). Assign the child node group to this placeholder (placeholder_node.node_tree = child_group).
         • Link the child group's output to the corresponding parent group's output socket. Ensure the parent output socket type is NodeSocketColor.
         • Image Node Handling (Iterate through resolutions for the map type):
           • For each resolution (e.g., "4K", "2K") and its image_path in the metadata:
             • Check manifest if this specific map/resolution is processed (if enabled). Skip if true.
             • Find the corresponding Image Texture node in the child group (by label matching resolution, e.g., "4K").
             • Load the image using bpy.data.images.load(image_path, check_existing=True). Handle potential file-not-found errors.
             • Assign the loaded image to the image_node.image.
             • Set the image_node.image.colorspace_settings.name based on the map_type (using a helper function get_color_space).
             • Update manifest dictionary for this map/resolution (if enabled).
     • Update manifest dictionary for the processed asset (if enabled).
   • Save manifest data (if enabled and changes were made).
   • Print summary (duration, groups created/updated, etc.).

4. Helper Functions:

   • find_nodes_by_label(node_tree, label, node_type): Reusable function to find nodes.
   • calculate_factor_from_string(aspect_string): Parses the aspect_ratio_change_string from metadata and returns the appropriate UV X-scaling factor.
   • get_color_space(map_type): Returns the appropriate Blender color space name for a given map type string.
   • add_tag_if_new(asset_data, tag_name): Adds a tag if it doesn't exist.
   • Manifest loading/saving/checking functions.

5. Execution Block (if __name__ == "__main__":)

   • Add pre-run checks (templates exist, library path valid, blend file saved if manifest enabled).
   • Call the main process_library() function.
   • Include basic timing and print statements for start/end.

Mermaid Diagram:

graph TD
    A[Start Script in Blender] --> B(Load Config: Lib Path, Template Names, Node Labels);
    B --> C{Check Templates Exist};
    C -- Templates OK --> D(Load Manifest from adjacent .json file);
    C -- Templates Missing --> X(Error & Exit);
    D --> E{Scan Processed Library for metadata.json};
    E --> F{For each metadata.json};
    F --> G{Parse Metadata (Asset Name, Supplier, Archetype, Maps, Aspect Str, Stats)};
    G --> H{Is Asset in Manifest?};
    H -- Yes --> F;
    H -- No --> I{Find/Create Parent Group (PBRSET_)};
    I --> J(Mark as Asset & Apply Supplier + Archetype Tags);
    J --> K(Find Aspect Node & Set Value from Aspect String);
    K --> M{For each Map Type in Metadata};
    M --> N(Find Stats Node & Set Values from Stats in Metadata);
    N --> O{Find/Create Child Group (PBRTYPE_)};
    O --> P(Assign Child to Parent Placeholder);
    P --> Q(Link Child Output to Parent Output);
    Q --> R{For each Resolution of Map};
    R --> S{Is Map/Res in Manifest?};
    S -- Yes --> R;
    S -- No --> T(Find Image Node in Child);
    T --> U(Load Processed Image);
    U --> V(Assign Image to Node);
    V --> W(Set Image Color Space);
    W --> W1(Update Manifest Dict for Map/Res);
    W1 --> R;
    R -- All Resolutions Done --> M;
    M -- All Map Types Done --> X1(Update Manifest Dict for Asset);
    X1 --> F;
    F -- All metadata.json Processed --> Y(Save Manifest Dict to adjacent .json file);
    Y --> Z(Print Summary & Finish);

    subgraph "Manifest Operations (External File)"
        D; H; S; W1; X1; Y;
    end

    subgraph "Node/Asset Operations"
        I; J; K; N; O; P; Q; T; U; V; W;
    end