---
quantized_by: bobchenyx
license: mit
base_model:
- moonshotai/Kimi-K2-Instruct
pipeline_tag: text-generation
base_model_relation: quantized
---

## Llamacpp Quantizations of Kimi-K2-Instruct

Original model: [moonshotai/Kimi-K2-Instruct](https://huggingface.co/moonshotai/Kimi-K2-Instruct).

All quants made based on [bartowski1182-llama.cpp](https://github.com/bartowski1182/llama.cpp).

All quants using imatrix & BF16 convertion from [unsloth/Kimi-K2-Instruct-GGUF/BF16](https://huggingface.co/unsloth/Kimi-K2-Instruct-GGUF/tree/main/BF16).

**IQ1_S : 197.39 GiB (1.65 BPW)**

**IQ1_M : 206.03 GiB (1.72 BPW)**

**IQ2_S : 265.71 GiB (2.22 BPW)**

**Q2_K : 335.39 GiB (2.81 BPW)**

---
## Download(Example)

```
# !pip install huggingface_hub hf_transfer
import os
os.environ["HF_HUB_ENABLE_HF_TRANSFER"] = "1"
from huggingface_hub import snapshot_download
snapshot_download(
    repo_id = "bobchenyx/Kimi-K2-Instruct-GGUF",
    local_dir = "bobchenyx/Kimi-K2-Instruct-GGUF",
    allow_patterns = ["*IQ1_M*"],
)
```