mirror of https://github.com/leafspark/AutoGGUF
171 lines
6.7 KiB
Markdown
171 lines
6.7 KiB
Markdown
|
||
|
||
# AutoGGUF - automated GGUF model quantizer
|
||
|
||
<!-- Project Status -->
|
||
[](https://github.com/leafspark/AutoGGUF/releases)
|
||
[](https://github.com/leafspark/AutoGGUF/commits)
|
||
[]()
|
||
|
||
<!-- Project Info -->
|
||
[](https://github.com/ggerganov/llama.cpp)
|
||
[]()
|
||
[](https://github.com/leafspark/AutoGGUF/blob/main/LICENSE)
|
||
|
||
|
||
<!-- Repository Stats -->
|
||
|
||
|
||
|
||
|
||
<!--  -->
|
||
|
||
<!-- Contribution -->
|
||
[](https://github.com/leafspark/AutoGGUF/issues)
|
||
[](https://github.com/psf/black)
|
||
[](https://github.com/leafspark/AutoGGUF/pulls)
|
||
|
||
The most comprehensive GUI tool for GGUF model quantization. Stop wrestling with command lines - quantize, merge, and optimize your models with just a few clicks.
|
||
|
||
## Features
|
||
|
||
- 📩 Update and manage llama.cpp backends
|
||
- 🗃️ Download and quantize GGUF/safetensors models
|
||
- 📐 Configure quantization parameters
|
||
- 💻 Monitor system resources in real time during quantization
|
||
- ⏳ Parallel quantization + imatrix generation
|
||
- 🎉 LoRA conversion and merging
|
||
- 📁 Preset saving and loading
|
||
- 8️⃣ AutoFP8 quantization
|
||
- 🪓 GGUF splitting and merging
|
||
- 🌐 HTTP API for automation and monitoring
|
||
|
||
## Why AutoGGUF?
|
||
- Fast: Saves time on manual configuration
|
||
- Simple: Clean UI, no terminal needed
|
||
- Powerful: Handles models up to infinite size, only limited by your RAM
|
||
- Resource-aware: Optimized memory management and efficient UI library
|
||
|
||
|
||
|
||
## Quick Start
|
||
|
||
### Cross-platform (recommended)
|
||
1. `git clone https://github.com/leafspark/AutoGGUF`
|
||
2. `cd AutoGGUF`
|
||
3. Install dependencies:
|
||
```
|
||
pip install -r requirements.txt
|
||
```
|
||
4. Run the application:
|
||
```
|
||
python src/main.py
|
||
```
|
||
or use the `run.bat` script.
|
||
|
||
macOS and Ubuntu builds are provided with GitHub Actions, you may download the binaries in the releases section.
|
||
|
||
### Windows (for the impatient)
|
||
Standard builds:
|
||
1. Download the latest release
|
||
2. Extract all files to a folder
|
||
3. Run `AutoGGUF-x64.exe`
|
||
4. Any necessary folders will be automatically created
|
||
|
||
Setup builds:
|
||
1. Download the setup variant of latest release
|
||
2. Extract all files to a folder
|
||
3. Run the setup program
|
||
4. The .gguf extension will be registered with the program automatically
|
||
5. Run the program from the Start Menu or desktop shortcuts
|
||
|
||
After launching the program, you may access its local server at port 7001 (set `AUTOGGUF_SERVER` to "enabled" first).
|
||
|
||
### Verifying Releases
|
||
|
||
#### Linux/macOS:
|
||
```bash
|
||
gpg --import AutoGGUF-v1.5.0-prerel.asc
|
||
gpg --verify AutoGGUF-v1.9.1-Windows-avx2.zip.sig AutoGGUF-v1.9.1-Windows-avx2.zip
|
||
sha256sum -c AutoGGUF-v1.9.1.sha256
|
||
```
|
||
|
||
#### Windows (PowerShell):
|
||
```powershell
|
||
# Import the public key
|
||
gpg --import AutoGGUF-v1.5.0-prerel.asc
|
||
|
||
# Verify the signature
|
||
gpg --verify AutoGGUF-v1.9.1-Windows-avx2.zip.sig AutoGGUF-v1.9.1-Windows-avx2.zip
|
||
|
||
# Check SHA256
|
||
$fileHash = (Get-FileHash -Algorithm SHA256 AutoGGUF-v1.9.1-Windows-avx2.zip).Hash.ToLower()
|
||
$storedHash = (Get-Content AutoGGUF-v1.9.1.sha256 | Select-String AutoGGUF-v1.9.1-Windows-avx2.zip).Line.Split()[0]
|
||
if ($fileHash -eq $storedHash) { "SHA256 Match" } else { "SHA256 Mismatch" }
|
||
```
|
||
|
||
Release keys are identical to ones used for commiting.
|
||
|
||
## Building
|
||
|
||
### Cross-platform
|
||
```bash
|
||
pip install -U pyinstaller
|
||
./build.sh RELEASE | DEV
|
||
cd build/<type>/dist/
|
||
./AutoGGUF
|
||
```
|
||
|
||
### Windows
|
||
```bash
|
||
pip install -U pyinstaller
|
||
build RELEASE | DEV
|
||
```
|
||
Find the executable in `build/<type>/dist/AutoGGUF-x64.exe`.
|
||
|
||
You can also use Nuitka, which may result in a slower build but a faster output executable:
|
||
```bash
|
||
build_optimized RELEASE | DEV
|
||
```
|
||
|
||
## Localizations
|
||
|
||
View the list of supported languages at [AutoGGUF/wiki/Installation#configuration](https://github.com/leafspark/AutoGGUF/wiki/Installation#configuration) (LLM translated, except for English).
|
||
|
||
Languages will be updated as soon as possible after an update, or as a part of the update.
|
||
|
||
To use a specific language, set the `AUTOGGUF_LANGUAGE` environment variable to one of the listed language codes (note: some languages may not be fully supported yet, in which the UI elements will fall back to English).
|
||
|
||
## Issues
|
||
|
||
- Some inconsistent logging and signal handling
|
||
- Missing or duplicated translations (priority)
|
||
- Buggy/incomplete API interfaces
|
||
- Code review and formatting (priority)
|
||
|
||
## Planned Features
|
||
|
||
- [ ] Time estimation for quantization
|
||
- [ ] Quantization file size estimate
|
||
- [ ] Perplexity testing
|
||
- [ ] bitsandbytes support
|
||
|
||
#### Project Status
|
||
|
||
AutoGGUF has now entered maintenance mode. It's considered stable and feature-complete for most use cases, so I'm not actively developing new features, but I’ll continue to publish occasional builds, update dependencies regularly, and fix critical bugs as needed. If you encounter issues or have suggestions, feel free to open an issue.
|
||
|
||
## Support
|
||
|
||
- SSL module cannot be found error: Install OpenSSL or run from source using `python src/main.py` with the `run.bat` script (`pip install requests`)
|
||
- Check out the [Wiki](https://github.com/leafspark/AutoGGUF/wiki) for advanced usage and configuration
|
||
|
||
## Contributing
|
||
|
||
Fork the repo, make your changes, and ensure you have the latest commits when merging. Include a changelog of new features in your pull request description. Read `CONTRIBUTING.md` for more information.
|
||
|
||
## Stargazers
|
||
|
||
[](https://star-history.com/#leafspark/AutoGGUF&Date)
|
||
|
||
`Last Updated: May 24, 2025`
|