leafspark
/
AutoGGUF
mirror of https://github.com/leafspark/AutoGGUF
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

add shields and reformat readme

This commit is contained in:
leafspark 2024-08-04 18:32:25 -07:00 committed by GitHub
parent 7af40477f4
commit 93be6139f7
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 64 additions and 46 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

110
README.md
View File

@ -2,30 +2,49 @@
# AutoGGUF - automated GGUF model quantizer # AutoGGUF - automated GGUF model quantizer
AutoGGUF provides a graphical user interface for quantizing GGUF models [![Powered by llama.cpp](https://img.shields.io/badge/Powered%20by-llama.cpp-green.svg)](https://github.com/ggerganov/llama.cpp)
using the llama.cpp library. It allows users to download different versions of ![GitHub release](https://img.shields.io/github/release/leafspark/AutoGGUF.svg)
llama.cpp, manage multiple backends, and perform quantization tasks with various ![GitHub last commit](https://img.shields.io/github/last-commit/leafspark/AutoGGUF.svg)
options. ![GitHub stars](https://img.shields.io/github/stars/leafspark/AutoGGUF.svg)
![GitHub forks](https://img.shields.io/github/forks/leafspark/AutoGGUF.svg)
![GitHub top language](https://img.shields.io/github/languages/top/leafspark/AutoGGUF.svg)
![GitHub repo size](https://img.shields.io/github/repo-size/leafspark/AutoGGUF.svg)
![GitHub license](https://img.shields.io/github/license/leafspark/AutoGGUF.svg)
## Features: AutoGGUF provides a graphical user interface for quantizing GGUF models using the llama.cpp library. It allows users to download different versions of llama.cpp, manage multiple backends, and perform quantization tasks with various options.
1. Download and manage llama.cpp backends
2. Select and quantize GGUF models
3. Configure quantization parameters
4. Monitor system resources during quantization
## Usage: ## Features
**Cross platform**: - Download and manage llama.cpp backends
1. Install dependencies, either using the `requirements.txt` file or `pip install PyQt6 requests psutil`. - Select and quantize GGUF models
2. Run the `run.bat` script to start the application, or run the command `python src/main.py`. - Configure quantization parameters
- Monitor system resources during quantization
**Windows**: ## Usage
1. Download latest release, extract all to folder and run `AutoGGUF.exe`
2. Enjoy!
## Building: ### Cross-platform
1. Install dependencies:
```
pip install -r requirements.txt
```
or
```
pip install PyQt6 requests psutil shutil
```
2. Run the application:
```
python src/main.py
```
or use the `run.bat` script.
**Cross platform**: ### Windows
1. Download the latest release
2. Extract all files to a folder
3. Run `AutoGGUF.exe`
## Building
### Cross-platform
```bash ```bash
cd src cd src
pip install -U pyinstaller pip install -U pyinstaller
@ -33,56 +52,55 @@ ## Building:
cd dist/main cd dist/main
./main ./main
``` ```
**Windows**:
### Windows
```bash ```bash
build RELEASE/DEV build RELEASE/DEV
``` ```
Find exe in `build/<type>/dist/AutoGGUF.exe`. Find the executable in `build/<type>/dist/AutoGGUF.exe`.
## Dependencies
## Dependencies:
- PyQt6 - PyQt6
- requests - requests
- psutil - psutil
- shutil - shutil
- OpenSSL - OpenSSL
## Localizations: ## Localizations
View the list of languages supported at [AutoGGUF/wiki/Installation#configuration](https://github.com/leafspark/AutoGGUF/wiki/Installation#configuration) (LLM translated, except for English) View the list of supported languages at [AutoGGUF/wiki/Installation#configuration](https://github.com/leafspark/AutoGGUF/wiki/Installation#configuration) (LLM translated, except for English).
In order to use them, please set the `AUTOGGUF_LANGUAGE` environment variable to one of the listed language codes. To use a specific language, set the `AUTOGGUF_LANGUAGE` environment variable to one of the listed language codes.
## Known Issues
## Issues:
- Saving preset while quantizing causes UI thread crash (planned fix: remove this feature) - Saving preset while quantizing causes UI thread crash (planned fix: remove this feature)
- Cannot delete task while processing, you must cancel it first or the program crashes (planned fix: don't allow deletion before cancelling, or cancel automatically) - Cannot delete task while processing (planned fix: disallow deletion before cancelling or cancel automatically)
- Base Model text still shows when GGML is selected as LoRA type (fix: include text in show/hide Qt layout) - Base Model text still shows when GGML is selected as LoRA type (fix: include text in show/hide Qt layout)
- ~~Cannot disable llama.cpp update check on startup~~ (fixed in v1.3.1)
- ~~`_internal` directory required, will see if I can package this into a single exe on the next release~~ (fixed in v1.3.1)
- ~~Custom command line parameters~~ (added in v1.3.0)
- ~~More iMatrix generation parameters~~ (added in v1.3.0)
- ~~Specify multiple KV overrides~~ (added in v1.1.0)
- ~~Better error handling~~ (added in v1.1.0)
- ~~Cannot select output/token embd type~~ (fixed in v1.1.0)
- ~~Importing presets with KV overrides causes UI thread crash~~ (fixed in v1.3.0)
## Planned features: ## Planned Features
- Actual progress bar tracking - Actual progress bar tracking
- Download safetensors from HF and convert to unquanted GGUF - Download safetensors from HF and convert to unquantized GGUF
- Perplexity testing - Perplexity testing
- Managing shards (coming in the next release) - Managing shards (coming in the next release)
- Time estimated for quantization - Time estimation for quantization
- Dynamic values for KV cache, e.g. autogguf.quantized.time=str:{system.time.milliseconds} (coming in the next release) - Dynamic values for KV cache (coming in the next release)
- Ability to select and start multiple quants at once (saved in presets) (coming in the next release) - Ability to select and start multiple quants at once (saved in presets, coming in the next release)
## Troubleshooting: ## Troubleshooting
- ~~llama.cpp quantizations errors out with an iostream error: create the `quantized_models` directory (or set a directory)~~ (fixed in v1.2.1, automatically created on launch)
- SSL module cannot be found error: Install OpenSSL or run from source `python src/main.py` using the `run.bat` script (`pip install requests`)
## Contributing: - SSL module cannot be found error: Install OpenSSL or run from source using `python src/main.py` with the `run.bat` script (`pip install requests`)
Simply fork the repo and make your changes; when merging make sure to have the latest commits. Description should contain a changelog of what's new.
## Contributing
Fork the repo, make your changes, and ensure you have the latest commits when merging. Include a changelog of new features in your pull request description.
## User Interface
## User interface:
![image](https://github.com/user-attachments/assets/2660c841-07ba-4c3f-ae3a-e63c7068bdc1) ![image](https://github.com/user-attachments/assets/2660c841-07ba-4c3f-ae3a-e63c7068bdc1)
## Stargazers: ## Stargazers
[![Star History Chart](https://api.star-history.com/svg?repos=leafspark/AutoGGUF&type=Date)](https://star-history.com/#leafspark/AutoGGUF&Date) [![Star History Chart](https://api.star-history.com/svg?repos=leafspark/AutoGGUF&type=Date)](https://star-history.com/#leafspark/AutoGGUF&Date)