From ab6912dad2c57c157a2734f4218d74d3a02dec78 Mon Sep 17 00:00:00 2001 From: leafspark <78000825+leafspark@users.noreply.github.com> Date: Wed, 16 Oct 2024 18:20:48 -0700 Subject: [PATCH] Created Quick Start (markdown) --- Quick-Start.md | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 Quick-Start.md diff --git a/Quick-Start.md b/Quick-Start.md new file mode 100644 index 0000000..f8a4de4 --- /dev/null +++ b/Quick-Start.md @@ -0,0 +1,33 @@ +# Install + +1. Download the latest release for your system on GitHub +2. Extract to a suitable folder, or use the installer (if using the -setup variant). +3. Download the .env.example file from here: [file](https://raw.githubusercontent.com/leafspark/AutoGGUF/refs/heads/main/.env.example) +4. Rename it to .env and move it to the AutoGGUF root folder (where the AutoGGUF-x64 binary is). Open in your choice of text editor, configure as necessary. + +# Backend + +![image](https://github.com/user-attachments/assets/5995ac8c-92de-47eb-88cc-a1e7529abfb2) + +1. Click Refresh Releases, and select version and build type for your system. +2. Click Download, and it will be automatically extracted to the `llama_bin` directory, ready for use. + +# HuggingFace -> GGUF (unquantized) + +![image](https://github.com/user-attachments/assets/8ab55e16-0da0-4cde-90de-5679aabbcfae) + +1. Choose model directory with safetensors using Browse +2. Choose output GGUF filename and path +3. Select desired output precision for unquantized GGUF +4. Specify model name (optional, for GGUF metadata), split max size (optional, for splitting models using GGUF split specification automatically, in place) +6. Click Convert HF to GGUF, will be dropped at the output path + +# GGUF (unquantized) -> GGUF (quantized) + +![image](https://github.com/user-attachments/assets/92710ab7-02f2-445f-8012-7c3d825dd177) + +1. Select source/unquantized GGUF model in the model directory, or import from any path on the system. +2. Select quantization types (multiple, if desired). +3. Set parameters (Allow Requantize essential for quantized -> more quantized GGUF, although this is not recommended) +4. Click Quantize Model +5. Quantized model will be dropped in Output Path directory \ No newline at end of file