leafspark
/
AutoGGUF
mirror of https://github.com/leafspark/AutoGGUF
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

add shields and reformat readme

This commit is contained in:
leafspark 2024-08-04 18:32:25 -07:00 committed by GitHub
parent 7af40477f4
commit 93be6139f7
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 64 additions and 46 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

110
README.md
View File

@ -2,30 +2,49 @@
# AutoGGUF - automated GGUF model quantizer
AutoGGUF provides a graphical user interface for quantizing GGUF models
using the llama.cpp library. It allows users to download different versions of
llama.cpp, manage multiple backends, and perform quantization tasks with various
options.
[![Powered by llama.cpp](https://img.shields.io/badge/Powered%20by-llama.cpp-green.svg)](https://github.com/ggerganov/llama.cpp)
![GitHub release](https://img.shields.io/github/release/leafspark/AutoGGUF.svg)
![GitHub last commit](https://img.shields.io/github/last-commit/leafspark/AutoGGUF.svg)
![GitHub stars](https://img.shields.io/github/stars/leafspark/AutoGGUF.svg)
![GitHub forks](https://img.shields.io/github/forks/leafspark/AutoGGUF.svg)
![GitHub top language](https://img.shields.io/github/languages/top/leafspark/AutoGGUF.svg)
![GitHub repo size](https://img.shields.io/github/repo-size/leafspark/AutoGGUF.svg)
![GitHub license](https://img.shields.io/github/license/leafspark/AutoGGUF.svg)
## Features:
1. Download and manage llama.cpp backends
2. Select and quantize GGUF models
3. Configure quantization parameters
4. Monitor system resources during quantization
AutoGGUF provides a graphical user interface for quantizing GGUF models using the llama.cpp library. It allows users to download different versions of llama.cpp, manage multiple backends, and perform quantization tasks with various options.
## Usage:
## Features
**Cross platform**:
1. Install dependencies, either using the `requirements.txt` file or `pip install PyQt6 requests psutil`.
2. Run the `run.bat` script to start the application, or run the command `python src/main.py`.
- Download and manage llama.cpp backends
- Select and quantize GGUF models
- Configure quantization parameters
- Monitor system resources during quantization
**Windows**:
1. Download latest release, extract all to folder and run `AutoGGUF.exe`
2. Enjoy!
## Usage
## Building:
### Cross-platform
1. Install dependencies:
```
pip install -r requirements.txt
```
or
```
pip install PyQt6 requests psutil shutil
```
2. Run the application:
```
python src/main.py
```
or use the `run.bat` script.
**Cross platform**:
### Windows
1. Download the latest release
2. Extract all files to a folder
3. Run `AutoGGUF.exe`
## Building
### Cross-platform
```bash
cd src
pip install -U pyinstaller
@ -33,56 +52,55 @@ ## Building:
cd dist/main
./main
```
**Windows**:
### Windows
```bash
build RELEASE/DEV
```
Find exe in `build/<type>/dist/AutoGGUF.exe`.
Find the executable in `build/<type>/dist/AutoGGUF.exe`.
## Dependencies
## Dependencies:
- PyQt6
- requests
- psutil
- shutil
- OpenSSL
## Localizations:
## Localizations
View the list of languages supported at [AutoGGUF/wiki/Installation#configuration](https://github.com/leafspark/AutoGGUF/wiki/Installation#configuration) (LLM translated, except for English)
View the list of supported languages at [AutoGGUF/wiki/Installation#configuration](https://github.com/leafspark/AutoGGUF/wiki/Installation#configuration) (LLM translated, except for English).
In order to use them, please set the `AUTOGGUF_LANGUAGE` environment variable to one of the listed language codes.
To use a specific language, set the `AUTOGGUF_LANGUAGE` environment variable to one of the listed language codes.
## Known Issues
## Issues:
- Saving preset while quantizing causes UI thread crash (planned fix: remove this feature)
- Cannot delete task while processing, you must cancel it first or the program crashes (planned fix: don't allow deletion before cancelling, or cancel automatically)
- Cannot delete task while processing (planned fix: disallow deletion before cancelling or cancel automatically)
- Base Model text still shows when GGML is selected as LoRA type (fix: include text in show/hide Qt layout)
- ~~Cannot disable llama.cpp update check on startup~~ (fixed in v1.3.1)
- ~~`_internal` directory required, will see if I can package this into a single exe on the next release~~ (fixed in v1.3.1)
- ~~Custom command line parameters~~ (added in v1.3.0)
- ~~More iMatrix generation parameters~~ (added in v1.3.0)
- ~~Specify multiple KV overrides~~ (added in v1.1.0)
- ~~Better error handling~~ (added in v1.1.0)
- ~~Cannot select output/token embd type~~ (fixed in v1.1.0)
- ~~Importing presets with KV overrides causes UI thread crash~~ (fixed in v1.3.0)
## Planned features:
## Planned Features
- Actual progress bar tracking
- Download safetensors from HF and convert to unquanted GGUF
- Download safetensors from HF and convert to unquantized GGUF
- Perplexity testing
- Managing shards (coming in the next release)
- Time estimated for quantization
- Dynamic values for KV cache, e.g. autogguf.quantized.time=str:{system.time.milliseconds} (coming in the next release)
- Ability to select and start multiple quants at once (saved in presets) (coming in the next release)
- Time estimation for quantization
- Dynamic values for KV cache (coming in the next release)
- Ability to select and start multiple quants at once (saved in presets, coming in the next release)
## Troubleshooting:
- ~~llama.cpp quantizations errors out with an iostream error: create the `quantized_models` directory (or set a directory)~~ (fixed in v1.2.1, automatically created on launch)
- SSL module cannot be found error: Install OpenSSL or run from source `python src/main.py` using the `run.bat` script (`pip install requests`)
## Troubleshooting
## Contributing:
Simply fork the repo and make your changes; when merging make sure to have the latest commits. Description should contain a changelog of what's new.
- SSL module cannot be found error: Install OpenSSL or run from source using `python src/main.py` with the `run.bat` script (`pip install requests`)
## Contributing
Fork the repo, make your changes, and ensure you have the latest commits when merging. Include a changelog of new features in your pull request description.
## User Interface
## User interface:
![image](https://github.com/user-attachments/assets/2660c841-07ba-4c3f-ae3a-e63c7068bdc1)
## Stargazers:
## Stargazers
[![Star History Chart](https://api.star-history.com/svg?repos=leafspark/AutoGGUF&type=Date)](https://star-history.com/#leafspark/AutoGGUF&Date)