leafspark
/
AutoGGUF
mirror of https://github.com/leafspark/AutoGGUF
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Created Quick Start (markdown)

leafspark 2024-10-16 18:20:48 -07:00
parent 2e7cdabf07
commit ab6912dad2
1 changed files with 33 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

33
Quick-Start.md Normal file

@ -0,0 +1,33 @@
# Install
1. Download the latest release for your system on GitHub
2. Extract to a suitable folder, or use the installer (if using the -setup variant).
3. Download the .env.example file from here: [file](https://raw.githubusercontent.com/leafspark/AutoGGUF/refs/heads/main/.env.example)
4. Rename it to .env and move it to the AutoGGUF root folder (where the AutoGGUF-x64 binary is). Open in your choice of text editor, configure as necessary.
# Backend
![image](https://github.com/user-attachments/assets/5995ac8c-92de-47eb-88cc-a1e7529abfb2)
1. Click Refresh Releases, and select version and build type for your system.
2. Click Download, and it will be automatically extracted to the `llama_bin` directory, ready for use.
# HuggingFace -> GGUF (unquantized)
![image](https://github.com/user-attachments/assets/8ab55e16-0da0-4cde-90de-5679aabbcfae)
1. Choose model directory with safetensors using Browse
2. Choose output GGUF filename and path
3. Select desired output precision for unquantized GGUF
4. Specify model name (optional, for GGUF metadata), split max size (optional, for splitting models using GGUF split specification automatically, in place)
6. Click Convert HF to GGUF, will be dropped at the output path
# GGUF (unquantized) -> GGUF (quantized)
![image](https://github.com/user-attachments/assets/92710ab7-02f2-445f-8012-7c3d825dd177)
1. Select source/unquantized GGUF model in the model directory, or import from any path on the system.
2. Select quantization types (multiple, if desired).
3. Set parameters (Allow Requantize essential for quantized -> more quantized GGUF, although this is not recommended)
4. Click Quantize Model
5. Quantized model will be dropped in Output Path directory