mirror of https://github.com/leafspark/AutoGGUF
121 lines
4.3 KiB
Markdown
121 lines
4.3 KiB
Markdown
|
|
|
|
# AutoGGUF - automated GGUF model quantizer
|
|
|
|
<!-- Project Status -->
|
|
[](https://github.com/leafspark/AutoGGUF/releases)
|
|
[](https://github.com/leafspark/AutoGGUF/commits)
|
|
[]()
|
|
|
|
<!-- Project Info -->
|
|
[](https://github.com/ggerganov/llama.cpp)
|
|
|
|
[]()
|
|
[](https://github.com/leafspark/AutoGGUF/blob/main/LICENSE)
|
|
|
|
<!-- Repository Stats -->
|
|
|
|
|
|
|
|
|
|
<!-- Contribution -->
|
|
[](https://github.com/psf/black)
|
|
[](https://github.com/leafspark/AutoGGUF/issues)
|
|
[](https://github.com/leafspark/AutoGGUF/pulls)
|
|
|
|
AutoGGUF provides a graphical user interface for quantizing GGUF models using the llama.cpp library. It allows users to download different versions of llama.cpp, manage multiple backends, and perform quantization tasks with various options.
|
|
|
|
## Features
|
|
|
|
- Download and manage llama.cpp backends
|
|
- Select and quantize GGUF models
|
|
- Configure quantization parameters
|
|
- Monitor system resources during quantization
|
|
|
|
## Usage
|
|
|
|
### Cross-platform
|
|
1. Install dependencies:
|
|
```
|
|
pip install -r requirements.txt
|
|
```
|
|
or
|
|
```
|
|
pip install PyQt6 requests psutil shutil
|
|
```
|
|
2. Run the application:
|
|
```
|
|
python src/main.py
|
|
```
|
|
or use the `run.bat` script.
|
|
|
|
### Windows
|
|
1. Download the latest release
|
|
2. Extract all files to a folder
|
|
3. Run `AutoGGUF.exe`
|
|
|
|
## Building
|
|
|
|
### Cross-platform
|
|
```bash
|
|
cd src
|
|
pip install -U pyinstaller
|
|
pyinstaller main.py --onefile
|
|
cd dist/main
|
|
./main
|
|
```
|
|
|
|
### Windows
|
|
```bash
|
|
build RELEASE | DEV
|
|
```
|
|
Find the executable in `build/<type>/dist/AutoGGUF.exe`.
|
|
|
|
## Dependencies
|
|
|
|
- PyQt6
|
|
- psutil
|
|
- shutil
|
|
- numpy
|
|
- torch
|
|
- safetensors
|
|
- gguf (bundled)
|
|
|
|
## Localizations
|
|
|
|
View the list of supported languages at [AutoGGUF/wiki/Installation#configuration](https://github.com/leafspark/AutoGGUF/wiki/Installation#configuration) (LLM translated, except for English).
|
|
|
|
To use a specific language, set the `AUTOGGUF_LANGUAGE` environment variable to one of the listed language codes.
|
|
|
|
## Known Issues
|
|
|
|
- Saving preset while quantizing causes UI thread crash (planned fix: remove this feature)
|
|
- Cannot delete task while processing (planned fix: disallow deletion before cancelling or cancel automatically)
|
|
- ~~Base Model text still shows when GGML is selected as LoRA type (fix: include text in show/hide Qt layout)~~ (fixed in v1.4.2)
|
|
|
|
## Planned Features
|
|
|
|
- Actual progress bar tracking
|
|
- Download safetensors from HF and convert to unquantized GGUF
|
|
- Perplexity testing
|
|
- Managing shards (coming in the next release)
|
|
- Time estimation for quantization
|
|
- Dynamic values for KV cache (coming in the next release)
|
|
- Ability to select and start multiple quants at once (saved in presets, coming in the next release)
|
|
|
|
## Troubleshooting
|
|
|
|
- SSL module cannot be found error: Install OpenSSL or run from source using `python src/main.py` with the `run.bat` script (`pip install requests`)
|
|
|
|
## Contributing
|
|
|
|
Fork the repo, make your changes, and ensure you have the latest commits when merging. Include a changelog of new features in your pull request description. Read `CONTRIBUTING.md` for more information.
|
|
|
|
## User Interface
|
|
|
|
|
|
|
|
## Stargazers
|
|
|
|
[](https://star-history.com/#leafspark/AutoGGUF&Date)
|