leafspark
/
AutoGGUF
mirror of https://github.com/leafspark/AutoGGUF
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

docs: update docstrings and small code fixes 

- update docstrings for AutoGGUF.py and add for lora_conversion.py and Logger.py
- fix IDE detected code typos and errors
This commit is contained in:
BuildTools 2024-08-16 19:43:48 -07:00
parent 7ac297a3bc
commit c02df3005e
No known key found for this signature in database
GPG Key ID: 3270C066C15D530B
12 changed files with 363 additions and 171 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/workflows/codeql.yml vendored
View File

@ -77,7 +77,7 @@ jobs:
# For more details on CodeQL's query packs, refer to: https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
# queries: security-extended,security-and-quality
# If the analyze step fails for one of the languages you are analyzing with
# If the analysis step fails for one of the languages you are analyzing with
# "We were unable to automatically build your code", modify the matrix above
# to set the build mode to "manual" for that language. Then modify this step
# to build your code.

2
CONTRIBUTING.md
View File

@ -33,7 +33,7 @@ ### Git Commit Messages
- Use the present tense ("Add feature" not "Added feature")
- Use the imperative mood ("Move cursor to..." not "Moves cursor to...")
- Limit the first line to 72 characters or less
- Limit the first line to 72 characters or fewer
### Commit Types:

2
README.md
View File

@ -107,7 +107,7 @@ ## Localizations
View the list of supported languages at [AutoGGUF/wiki/Installation#configuration](https://github.com/leafspark/AutoGGUF/wiki/Installation#configuration) (LLM translated, except for English).
To use a specific language, set the `AUTOGGUF_LANGUAGE` environment variable to one of the listed language codes (note: some languages may not be fully supported yet, those will fallback to English).
To use a specific language, set the `AUTOGGUF_LANGUAGE` environment variable to one of the listed language codes (note: some languages may not be fully supported yet, those will fall back to English).
## Known Issues

2
SECURITY.md
View File

@ -3,7 +3,7 @@ # Security Policy
## Supported Versions
| Version | Supported |
| ----------------- | ------------------ |
|-----------------|--------------------|
| stable (v1.6.2) | :white_check_mark: |
## Reporting a Vulnerability

389
docs/AutoGGUF.py
View File

@ -1,145 +1,314 @@
import json
import re
import shutil
import os
from functools import partial
from PySide6.QtCore import *
from PySide6.QtGui import *
from PySide6.QtWidgets import *
from GPUMonitor import GPUMonitor
from KVOverrideEntry import KVOverrideEntry
from Logger import Logger
from ModelInfoDialog import ModelInfoDialog
from error_handling import show_error, handle_error
from imports_and_globals import (
open_file_safe,
resource_path,
show_about,
ensure_directory,
)
from Localizations import *
import presets
import ui_update
import lora_conversion
import utils
class CustomTitleBar(QWidget):
"""
Custom title bar for the main window, providing drag-and-drop functionality
and minimize/close buttons.
"""
def __init__(self, parent=None):
"""
Initializes the custom title bar.
Args:
parent (QWidget, optional): The parent widget. Defaults to None.
"""
class AutoGGUF(QMainWindow):
"""
AutoGGUF is a PySide6-based graphical user interface for managing and quantizing large language models.
This class provides functionality for:
- Loading and displaying models (including sharded models)
- Quantizing models with various options
- Downloading llama.cpp releases
- Generating importance matrices
- Converting and exporting LoRA models
- Managing quantization tasks
- Converting Hugging Face models to GGUF format
The GUI allows users to interact with these features in an intuitive way, providing
options for model selection, quantization parameters, and task management.
Attributes:
logger (Logger): Instance of the Logger class for logging operations.
ram_bar (QProgressBar): Progress bar for displaying RAM usage.
cpu_label (QLabel): Label for displaying CPU usage.
gpu_monitor (GPUMonitor): Widget for monitoring GPU usage.
backend_combo (QComboBox): Dropdown for selecting the backend.
model_tree (QTreeWidget): Tree widget for displaying available models.
task_list (QListWidget): List widget for displaying ongoing tasks.
quant_threads (list): List to store active quantization threads.
The class also contains numerous UI elements for user input and interaction,
including text inputs, checkboxes, and buttons for various operations.
Main application window for AutoGGUF, providing a user interface for
quantizing and converting large language models.
"""
def __init__(self):
"""
Initialize the AutoGGUF application window.
Initializes the main window, setting up the UI, logger, and other
necessary components.
"""
This method sets up the main window, initializes the UI components,
sets up layouts, and connects various signals to their respective slots.
It also initializes the logger, sets up the system info update timer,
and prepares the application for model management and quantization tasks.
def keyPressEvent(self, event):
"""
Handles key press events for window resizing.
The initialization process includes:
- Setting up the main window properties (title, icon, size)
- Creating and arranging UI components for different functionalities
- Initializing backend and release information
- Setting up file browsers for various inputs
- Preparing quantization options and task management interface
- Initializing iMatrix generation interface
- Setting up LoRA conversion and export interfaces
- Preparing Hugging Face to GGUF conversion interface
Args:
event (QKeyEvent): The key press event.
"""
def resize_window(self, larger):
"""
Resizes the window by a specified factor.
Args:
larger (bool): Whether to make the window larger or smaller.
"""
def reset_size(self):
"""Resets the window to its default size."""
def parse_resolution(self):
"""
Parses the resolution from the AUTOGGUF_RESOLUTION environment variable.
Returns:
tuple: The width and height of the window.
"""
def resizeEvent(self, event):
"""
Handles resize events to maintain rounded corners.
Args:
event (QResizeEvent): The resize event.
"""
def refresh_backends(self):
"""Refreshes the list of available backends."""
def save_task_preset(self, task_item):
"""
Refresh the list of available backends.
Saves the preset for a specific task.
This method scans the 'llama_bin' directory for valid backends,
updates the backend selection combo box, and enables/disables
it based on the availability of backends.
The method logs the refresh operation and the number of valid
backends found.
Args:
task_item (TaskListItem): The task item to save the preset for.
"""
def update_assets(self):
"""
Update the list of assets for the selected llama.cpp release.
def browse_export_lora_model(self):
"""Opens a file dialog to browse for the export LORA model file."""
This method clears the current asset list and populates it with
the assets of the selected release. It also updates the CUDA
option visibility based on the selected asset.
def browse_export_lora_output(self):
"""Opens a file dialog to browse for the export LORA output file."""
def add_lora_adapter(self):
"""Adds a LORA adapter to the export LORA list."""
def browse_base_model(self):
"""Opens a file dialog to browse for the base model folder."""
def delete_lora_adapter_item(self, adapter_widget):
"""
Deletes a LORA adapter item from the export LORA list.
Args:
adapter_widget (QWidget): The widget containing the adapter information.
"""
def download_llama_cpp(self):
"""
Initiate the download of the selected llama.cpp release asset.
def browse_hf_model_input(self):
"""Opens a file dialog to browse for the HuggingFace model directory."""
This method starts a download thread for the selected asset,
updates the UI to show download progress, and sets up signal
connections for download completion and error handling.
def browse_hf_outfile(self):
"""Opens a file dialog to browse for the HuggingFace to GGUF output file."""
def convert_hf_to_gguf(self):
"""Converts a HuggingFace model to GGUF format."""
def export_lora(self):
"""Exports a LORA from a GGML model."""
def restart_task(self, task_item):
"""
Restarts a specific task.
Args:
task_item (TaskListItem): The task item to restart.
"""
def lora_conversion_finished(self, thread, input_path, output_path):
"""
Handles the completion of a LORA conversion task.
Args:
thread (QuantizationThread): The thread that handled the conversion.
input_path (str): The path to the input LORA file.
output_path (str): The path to the output GGML file.
"""
def download_finished(self, extract_dir):
"""
Handles the completion of a download, extracting files and updating the UI.
Args:
extract_dir (str): The directory where the downloaded files were extracted.
"""
def extract_cuda_files(self, extract_dir, destination):
"""
Extracts CUDA files from a downloaded archive.
Args:
extract_dir (str): The directory where the downloaded files were extracted.
destination (str): The destination directory for the CUDA files.
"""
def download_error(self, error_message):
"""
Handles download errors, displaying an error message and cleaning up.
Args:
error_message (str): The error message.
"""
def show_task_context_menu(self, position):
"""
Shows the context menu for a task item in the task list.
Args:
position (QPoint): The position of the context menu.
"""
def show_task_properties(self, item):
"""
Shows the properties dialog for a specific task.
Args:
item (QListWidgetItem): The task item.
"""
def toggle_gpu_offload_auto(self, state):
"""
Toggles the automatic GPU offload option.
Args:
state (Qt.CheckState): The state of the checkbox.
"""
def cancel_task_by_item(self, item):
"""
Cancels a task by its item in the task list.
Args:
item (QListWidgetItem): The task item.
"""
def cancel_task(self, item):
"""
Cancels a specific task.
Args:
item (QListWidgetItem): The task item.
"""
def delete_task(self, item):
"""
Deletes a specific task.
Args:
item (QListWidgetItem): The task item.
"""
def create_label(self, text, tooltip):
"""
Creates a QLabel with a tooltip.
Args:
text (str): The text for the label.
tooltip (str): The tooltip for the label.
Returns:
QLabel: The created label.
"""
def load_models(self):
"""
Load and display the list of available models.
"""Loads the available models and displays them in the model tree."""
This method scans the specified models directory for .gguf files,
organizes them into sharded and single models, and populates
the model tree widget with this information.
"""
def browse_models(self):
"""Opens a file dialog to browse for the models directory."""
def browse_output(self):
"""Opens a file dialog to browse for the output directory."""
def browse_logs(self):
"""Opens a file dialog to browse for the logs directory."""
def browse_imatrix(self):
"""Opens a file dialog to browse for the imatrix file."""
def validate_quantization_inputs(self):
"""Validates the inputs for quantization."""
def add_kv_override(self, override_string=None):
"""Adds a KV override entry to the list."""
def remove_kv_override(self, entry):
"""Removes a KV override entry from the list."""
def quantize_model(self):
"""
Start the quantization process for the selected model.
"""Quantizes the selected model."""
This method prepares the quantization command based on user-selected
options, creates a new quantization thread, and sets up a task item
in the task list to track the quantization progress.
def parse_progress(self, line, task_item):
"""
Parses the progress from the output line and updates the task item.
Args:
line (str): The output line.
task_item (TaskListItem): The task item.
"""
def task_finished(self, thread, task_item):
"""
Handles the completion of a task.
Args:
thread (QuantizationThread): The thread that handled the task.
task_item (TaskListItem): The task item.
"""
def show_task_details(self, item):
"""
Shows the details of a specific task.
Args:
item (QListWidgetItem): The task item.
"""
def browse_imatrix_datafile(self):
"""Opens a file dialog to browse for the imatrix data file."""
def browse_imatrix_model(self):
"""Opens a file dialog to browse for the imatrix model file."""
def browse_imatrix_output(self):
"""Opens a file dialog to browse for the imatrix output file."""
def get_models_data(self):
"""Retrieves data for all loaded models."""
def get_tasks_data(self):
"""Retrieves data for all tasks in the task list."""
def generate_imatrix(self):
"""
Start the importance matrix generation process.
This method prepares the iMatrix generation command based on user inputs,
creates a new thread for the operation, and sets up a task item
in the task list to track the generation progress.
"""
def convert_lora(self):
"""
Start the LoRA conversion process.
This method prepares the LoRA conversion command based on user inputs,
creates a new thread for the conversion, and sets up a task item
in the task list to track the conversion progress.
"""
def export_lora(self):
"""
Start the LoRA export process.
This method prepares the LoRA export command based on user inputs,
creates a new thread for the export operation, and sets up a task item
in the task list to track the export progress.
"""
def convert_hf_to_gguf(self):
"""
Start the process of converting a Hugging Face model to GGUF format.
This method prepares the conversion command based on user inputs,
creates a new thread for the conversion, and sets up a task item
in the task list to track the conversion progress.
"""
"""Generates an imatrix file."""
def closeEvent(self, event: QCloseEvent):
"""
Handle the window close event.
This method is called when the user attempts to close the application.
It checks for any running tasks and prompts the user for confirmation
before closing if tasks are still in progress.
Handles close events, prompting the user if there are running tasks.
Args:
event (QCloseEvent): The close event object.
event (QCloseEvent): The close event.
"""

56
docs/Logger.py Normal file
View File

@ -0,0 +1,56 @@
class Logger:
"""
This module provides a custom logger class for logging messages to both the console and a rotating log file.
The log file will be created in the specified `log_dir` with a timestamp in the filename.
The file will rotate when it reaches 10MB, keeping a maximum of 5 backup files.
"""
def __init__(self, name, log_dir):
"""
Initializes the logger with a specified name and log directory.
Args:
name (str): The name of the logger.
log_dir (str): The directory where log files will be stored.
"""
def debug(self, message):
"""
Logs a message with the DEBUG level.
Args:
message (str): The message to log.
"""
def info(self, message):
"""
Logs a message with the INFO level.
Args:
message (str): The message to log.
"""
def warning(self, message):
"""
Logs a message with the WARNING level.
Args:
message (str): The message to log.
"""
def error(self, message):
"""
Logs a message with the ERROR level.
Args:
message (str): The message to log.
"""
def critical(self, message):
"""
Logs a message with the CRITICAL level.
Args:
message (str): The message to log.
"""

17
docs/lora_conversion.py Normal file
View File

@ -0,0 +1,17 @@
def convert_lora(self):
"""Converts a LORA file to either GGML or GGUF format.
This function initiates the conversion process based on user input,
utilizing a separate thread for the actual conversion and providing
progress updates in the UI.
It validates input paths, constructs the conversion command, creates
a log file, manages the conversion thread, and handles errors.
Args:
self: The object instance.
Raises:
ValueError: If required input paths are missing.
"""

51
src/AutoGGUF.py
View File

@ -1,7 +1,6 @@
import json
import re
import shutil
import os
from functools import partial
from PySide6.QtCore import *
@ -841,27 +840,6 @@ def resizeEvent(self, event):
mask = QRegion(path.toFillPolygon().toPolygon())
self.setMask(mask)
def closeEvent(self, event: QCloseEvent):
self.logger.info(APPLICATION_CLOSING)
if self.quant_threads:
reply = QMessageBox.question(
self,
WARNING,
TASK_RUNNING_WARNING,
QMessageBox.StandardButton.Yes | QMessageBox.StandardButton.No,
QMessageBox.StandardButton.No,
)
if reply == QMessageBox.StandardButton.Yes:
for thread in self.quant_threads:
thread.terminate()
event.accept()
else:
event.ignore()
else:
event.accept()
self.logger.info(APPLICATION_CLOSED)
def refresh_backends(self):
self.logger.info(REFRESHING_BACKENDS)
llama_bin = os.path.abspath("llama_bin")
@ -1206,35 +1184,6 @@ def download_finished(self, extract_dir):
if index >= 0:
self.backend_combo.setCurrentIndex(index)
def download_finished(self, extract_dir):
self.download_button.setEnabled(True)
self.download_progress.setValue(100)
if (
self.cuda_extract_checkbox.isChecked()
and self.cuda_extract_checkbox.isVisible()
):
cuda_backend = self.backend_combo_cuda.currentData()
if cuda_backend:
self.extract_cuda_files(extract_dir, cuda_backend)
QMessageBox.information(
self,
DOWNLOAD_COMPLETE,
LLAMACPP_DOWNLOADED_AND_EXTRACTED.format(extract_dir, cuda_backend),
)
else:
QMessageBox.warning(
self, CUDA_EXTRACTION_FAILED, NO_CUDA_BACKEND_SELECTED
)
else:
QMessageBox.information(
self,
DOWNLOAD_COMPLETE,
LLAMACPP_BINARY_DOWNLOADED_AND_EXTRACTED.format(extract_dir),
)
self.refresh_backends()
def extract_cuda_files(self, extract_dir, destination):
self.logger.info(EXTRACTING_CUDA_FILES.format(extract_dir, destination))
for root, dirs, files in os.walk(extract_dir):

2
src/convert_hf_to_gguf.py
View File

@ -2952,7 +2952,7 @@ def modify_tensors(
) -> Iterable[tuple[str, Tensor]]:
del bid # unused
# we are only using BERT for embeddings so we don't need the pooling layer
# we are only using BERT for embeddings, so we don't need the pooling layer
if name in (
"embeddings.position_ids",
"pooler.dense.weight",

2
src/gguf-py/gguf/lazy.py
View File

@ -34,7 +34,7 @@ def __getattr__(self, name: str) -> Any:
# need to make a builder for the wrapped wrapper to copy the name,
# or else it fails with very cryptic error messages,
# because somehow the same string would end up in every closures
# because somehow the same string would end up in every closure
def mk_wrap(op_name: str, *, meta_noop: bool = False):
# need to wrap the wrapper to get self
def wrapped_special_op(self, *args, **kwargs):

4
src/gguf-py/gguf/metadata.py
View File

@ -50,7 +50,7 @@ def load(
model_name: Optional[str] = None,
total_params: int = 0,
) -> Metadata:
# This grabs as many contextual authorship metadata as possible from the model repository
# This grabs as much contextual authorship metadata as possible from the model repository
# making any conversion as required to match the gguf kv store metadata format
# as well as giving users the ability to override any authorship metadata that may be incorrect
@ -508,7 +508,7 @@ def apply_metadata_heuristic(
hf_name_or_path = hf_params.get("_name_or_path")
if hf_name_or_path is not None and hf_name_or_path.count("/") <= 1:
# Use _name_or_path only if its actually a model name and not some computer path
# Use _name_or_path only if it's actually a model name and not some computer path
# e.g. 'meta-llama/Llama-2-7b-hf'
model_id = hf_name_or_path
(

1
src/ui_update.py
View File

@ -1,5 +1,6 @@
from Localizations import *
import psutil
from error_handling import show_error
def update_model_info(logger, self, model_info):