Rusfort/Asset-Frameworker
1
0
Fork 0
Code Issues 24 Pull Requests Actions Packages Projects 3 Releases Wiki Activity

LLM Restructure - UNTESTED!

Browse Source
This commit is contained in:
Rusfort
2025-05-04 12:56:16 +02:00
parent 74b3d008ea
commit 01c8f68ea0
6 changed files with 467 additions and 397 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
Documentation/02_Developer_Guide/03_Key_Components.md
View File

@@ -136,8 +136,9 @@ An experimental predictor (inheriting from `BasePredictionHandler`) that uses a
* Takes an input source identifier, file list, and `Configuration` object.
* Interacts with the `LLMInteractionHandler` to send data to the LLM and receive predictions.
* Parses the LLM response to construct a `SourceRule` hierarchy.
* Emits the `prediction_signal` with the generated `SourceRule` object.
* **Parses the LLM's JSON response**: It expects a specific two-part JSON structure (see `12_LLM_Predictor_Integration.md`). It first sanitizes the response (removing comments/markdown) and then parses the JSON.
* **Constructs `SourceRule`**: It groups files based on the `proposed_asset_group_name` from the JSON, assigns the final `asset_type` using the `asset_group_classifications` map, and builds the complete `SourceRule` hierarchy.
* Emits the `prediction_signal` with the generated `SourceRule` object or `error_signal` on failure.
### `LLMInteractionHandler` (`gui/llm_interaction_handler.py`)

100
Documentation/02_Developer_Guide/12_LLM_Predictor_Integration.md
View File

@@ -18,46 +18,94 @@ The LLM Predictor is configured via new settings in the `config/app_settings.jso
The prompt structure is crucial for effective classification. It should clearly instruct the LLM on the task and the expected output format. Placeholders within the prompt template (e.g., `{asset_name}`) are dynamically replaced with relevant data before the request is sent.
## `LLMPredictionHandler`
## Expected LLM Output Format (Refactored)
The `gui/llm_prediction_handler.py` module contains the `LLMPredictionHandler` class, which is responsible for interacting with the LLM API. It operates in a separate thread to avoid blocking the GUI during potentially long API calls.
The LLM is now expected to return a JSON object containing two distinct parts. This structure helps the LLM maintain context across multiple files belonging to the same conceptual asset and allows for a more robust grouping mechanism.
Key methods:
**Rationale:** The previous implicit format made it difficult for the LLM to consistently group related files (e.g., different texture maps for the same material) under a single asset, especially in complex archives. The new two-part structure explicitly separates file-level analysis from asset-level classification, improving accuracy and consistency.
- `run()`: The main method executed when the thread starts. It processes prediction requests from a queue.
- `_prepare_prompt(asset_name)`: Constructs the final prompt string by loading the template from settings, including examples, and replacing placeholders like `{asset_name}`.
- `_call_llm(prompt)`: Sends the prepared prompt to the configured LLM API endpoint using the `requests` library and handles the HTTP communication.
- `_parse_llm_response(response)`: Parses the response received from the LLM API to extract the predicted classification.
**Structure:**
```json
{
"individual_file_analysis": [
{
"relative_file_path": "Textures/Wood_Floor_01/Wood_Floor_01_BaseColor.png",
"classified_file_type": "BaseColor",
"proposed_asset_group_name": "Wood_Floor_01"
},
{
"relative_file_path": "Textures/Wood_Floor_01/Wood_Floor_01_Roughness.png",
"classified_file_type": "Roughness",
"proposed_asset_group_name": "Wood_Floor_01"
},
{
"relative_file_path": "Textures/Metal_Plate_03/Metal_Plate_03_Metallic.jpg",
"classified_file_type": "Metallic",
"proposed_asset_group_name": "Metal_Plate_03"
}
],
"asset_group_classifications": {
"Wood_Floor_01": "PBR Material",
"Metal_Plate_03": "PBR Material"
}
}
```
- **`individual_file_analysis`**: A list where each object represents a single file within the source.
- `relative_file_path`: The path of the file relative to the source root.
- `classified_file_type`: The LLM's prediction for the *type* of this specific file (e.g., "BaseColor", "Normal", "Model"). This corresponds to the `item_type` in the `FileRule`.
- `proposed_asset_group_name`: A name suggested by the LLM to group this file with others belonging to the same conceptual asset. This is used internally by the parser.
- **`asset_group_classifications`**: A dictionary mapping the `proposed_asset_group_name` values from the list above to a final `asset_type` (e.g., "PBR Material", "HDR Environment").
## `LLMPredictionHandler` (Refactored Parsing)
The `gui/llm_prediction_handler.py` module contains the `LLMPredictionHandler` class (inheriting from `BasePredictionHandler`), which orchestrates the LLM prediction process. It runs in a background thread managed by the `MainWindow`'s `QThreadPool`.
Key Responsibilities & Methods:
- **Initialization**: Takes the source identifier, file list, and `Configuration` object.
- **`run()`**: The main method executed by the thread pool. It prepares the prompt, calls the LLM (via `LLMInteractionHandler`), parses the response, and emits the result or error.
- **Interaction**: Uses `LLMInteractionHandler` to handle the actual prompt construction and API communication (details in `03_Key_Components.md` and `llm_interaction_handler.py`).
- **`_parse_llm_response(response_text)`**: This method contains the **new parsing logic**:
1. **Sanitization**: Removes common non-JSON elements like comments (`//`, `/* */`) and markdown code fences (```json ... ```) from the raw `response_text` to increase the likelihood of successful JSON parsing.
2. **JSON Parsing**: Parses the sanitized string into a Python dictionary.
3. **Structure Validation**: Checks if the parsed dictionary contains the required top-level keys: `individual_file_analysis` (list) and `asset_group_classifications` (dict).
4. **Grouping**: Iterates through the `individual_file_analysis` list. For each file:
* Retrieves the `proposed_asset_group_name`.
* Uses the `asset_group_classifications` dictionary to find the corresponding final `asset_type` for that group.
* Creates or updates an `AssetRule` for the group name, assigning the determined `asset_type`.
* Creates a `FileRule` for the specific file, assigning its `classified_file_type` as the `item_type`.
5. **Hierarchy Construction**: Organizes the created `AssetRule` and `FileRule` objects into a single `SourceRule` object representing the entire source.
6. **Validation**: Ensures all files from the input list were accounted for in the LLM response.
Signals:
- `prediction_ready(asset_name, prediction_result)`: Emitted when a prediction is successfully received and parsed for a given asset.
- `prediction_error(asset_name, error_message)`: Emitted if an error occurs during the prediction process (e.g., API call failure, parsing error).
The handler uses the `requests` library to make HTTP POST requests to the LLM endpoint, including the API key in the headers for authentication.
- `prediction_signal(source_id, source_rule)`: Emitted when a prediction is successfully parsed and the `SourceRule` hierarchy is constructed. The `source_rule` argument contains the complete hierarchy.
- `error_signal(source_id, error_message)`: Emitted if an error occurs during any stage (API call, sanitization, parsing, validation).
## GUI Integration
The `gui/main_window.py` module integrates the LLM Predictor feature into the main application window.
Integration remains largely the same at the `MainWindow` level:
Integration points:
- The LLM predictor is selected via the preset dropdown or triggered explicitly.
- `MainWindow` manages the `QThreadPool` and starts the `LLMPredictionHandler` task.
- Slots in `MainWindow` connect to the `prediction_signal` and `error_signal` of the handler.
- **Preset Dropdown Option:** A new option is added to the preset dropdown to enable LLM prediction as the classification method.
- **Re-interpret Button:** The "Re-interpret" button's functionality is extended to trigger LLM prediction when the LLM method is selected.
- `llm_processing_queue`: A queue (`Queue` object) is used to hold asset names that require LLM prediction. The `LLMPredictionHandler` thread consumes items from this queue.
- `_start_llm_prediction(asset_name)`: A method to add an asset name to the `llm_processing_queue` and ensure the `LLMPredictionHandler` thread is running.
- `_process_next_llm_item()`: A slot connected to the `prediction_ready` and `prediction_error` signals. It processes the results received from the `LLMPredictionHandler` and updates the GUI accordingly.
- **Signal Handling:** Connections are established between the `LLMPredictionHandler`'s signals (`prediction_ready`, `prediction_error`) and slots in `main_window.py` to handle prediction results and errors asynchronously.
## Model Integration (Refactored)
## Model Integration
The `gui/unified_view_model.py` module's `update_rules_for_sources` method still incorporates the results.
The `gui/unified_view_model.py` module, specifically the `update_rules_for_sources` method, is responsible for incorporating the prediction results into the application's data model. When a prediction is received via the `prediction_ready` signal, the `update_rules_for_sources` method is called to update the classification rules for the corresponding asset source based on the LLM's output.
- When the `prediction_signal` is received from `LLMPredictionHandler`, the accompanying `SourceRule` object (which has already been constructed based on the new two-part JSON parsing logic) is passed to `update_rules_for_sources`.
- This method then merges the new `SourceRule` hierarchy into the existing model data, preserving user overrides where applicable. The internal structure of the received `SourceRule` now directly reflects the groupings and classifications determined by the LLM and the new parser.
## Error Handling
## Error Handling (Updated)
Error handling for the LLM Predictor includes:
Error handling now covers additional scenarios:
- **LLM API Errors:** The `_call_llm` method in `LLMPredictionHandler` catches exceptions during the HTTP request and emits the `prediction_error` signal with a relevant error message.
- **Parsing Errors:** The `_parse_llm_response` method handles potential errors during the parsing of the LLM's response, emitting `prediction_error` if the response format is unexpected or invalid.
- **LLM API Errors:** Handled by `LLMInteractionHandler` and propagated via the `error_signal`.
- **Sanitization/Parsing Errors:** The `_parse_llm_response` method catches errors during comment/markdown removal and `json.loads()`.
- **Structure Errors:** Explicit checks for the presence and types of `individual_file_analysis` and `asset_group_classifications` keys in the parsed JSON.
- **Data Consistency Errors:** Validation errors if the parsed data doesn't align (e.g., a `proposed_asset_group_name` missing from `asset_group_classifications`, or files missing from the analysis).
These errors are then handled in `main_window.py` by the slot connected to the `prediction_error` signal, typically by displaying an error message to the user.
These errors trigger the `error_signal`, allowing `MainWindow` to inform the user.