--- license: apache-2.0 base_model: - Writer/palmyra-mini - Writer/palmyra-mini-thinking-a - Writer/palmyra-mini-thinking-b language: - en pipeline_tag: text-generation library_name: transformers tags: - text-generation-inference - code - math - coder --- # **palmyra-mini-thinking-AIO-GGUF** > The palmyra-mini *[models](https://huggingface.co/Writer)* demonstrates exceptional capabilities in complex reasoning and mathematical problem-solving domains. Its performance is particularly noteworthy on benchmarks that require deep understanding and multi-step thought processes. A key strength of the model is its proficiency in grade-school-level math problems, as evidenced by its impressive score of 0.818 on the gsm8k (strict-match) benchmark. This high score indicates a robust ability to parse and solve word problems, a foundational skill for more advanced quantitative reasoning. This aptitude for mathematics is further confirmed by its outstanding performance on the MATH500 benchmark, where it also achieved a score of 0.818. This result underscores the models consistent and reliable mathematical capabilities across different problem sets. The model also shows strong performance on the AMC23 benchmark, with a solid score of 0.6. This benchmark, representing problems from the American Mathematics Competitions, highlights the models ability to tackle challenging, competition-level mathematics. ## Palmyra Mini GGUF Variants | Model Name | Download Link | |-----------------------------------|---------------------------------------------------------------------------------------------------| | **palmyra-mini-GGUF** | [Link](https://huggingface.co/prithivMLmods/palmyra-mini-thinking-AIO-GGUF/tree/main/palmyra-mini-GGUF) | | **palmyra-mini-thinking-a-GGUF** | [Link](https://huggingface.co/prithivMLmods/palmyra-mini-thinking-AIO-GGUF/tree/main/palmyra-mini-thinking-a-GGUF) | | **palmyra-mini-thinking-b-GGUF** | [Link](https://huggingface.co/prithivMLmods/palmyra-mini-thinking-AIO-GGUF/tree/main/palmyra-mini-thinking-b-GGUF) | ## Model Files ### palmyra-mini | File Name | Quant Type | File Size | | - | - | - | | palmyra-mini.BF16.gguf | BF16 | 3.56 GB | | palmyra-mini.F16.gguf | F16 | 3.56 GB | | palmyra-mini.F32.gguf | F32 | 7.11 GB | | palmyra-mini.Q2_K.gguf | Q2_K | 752 MB | | palmyra-mini.Q3_K_L.gguf | Q3_K_L | 980 MB | | palmyra-mini.Q3_K_M.gguf | Q3_K_M | 924 MB | | palmyra-mini.Q3_K_S.gguf | Q3_K_S | 861 MB | | palmyra-mini.Q4_0.gguf | Q4_0 | 1.07 GB | | palmyra-mini.Q4_1.gguf | Q4_1 | 1.16 GB | | palmyra-mini.Q4_K.gguf | Q4_K | 1.12 GB | | palmyra-mini.Q4_K_M.gguf | Q4_K_M | 1.12 GB | | palmyra-mini.Q4_K_S.gguf | Q4_K_S | 1.07 GB | | palmyra-mini.Q5_0.gguf | Q5_0 | 1.26 GB | | palmyra-mini.Q5_1.gguf | Q5_1 | 1.35 GB | | palmyra-mini.Q5_K.gguf | Q5_K | 1.28 GB | | palmyra-mini.Q5_K_M.gguf | Q5_K_M | 1.28 GB | | palmyra-mini.Q5_K_S.gguf | Q5_K_S | 1.26 GB | | palmyra-mini.Q6_K.gguf | Q6_K | 1.46 GB | | palmyra-mini.Q8_0.gguf | Q8_0 | 1.89 GB | ### palmyra-mini-thinking-a | File Name | Quant Type | File Size | | - | - | - | | palmyra-mini-thinking-a.BF16.gguf | BF16 | 3.56 GB | | palmyra-mini-thinking-a.F16.gguf | F16 | 3.56 GB | | palmyra-mini-thinking-a.F32.gguf | F32 | 7.11 GB | | palmyra-mini-thinking-a.Q2_K.gguf | Q2_K | 752 MB | | palmyra-mini-thinking-a.Q3_K_L.gguf | Q3_K_L | 980 MB | | palmyra-mini-thinking-a.Q3_K_M.gguf | Q3_K_M | 924 MB | | palmyra-mini-thinking-a.Q3_K_S.gguf | Q3_K_S | 861 MB | | palmyra-mini-thinking-a.Q4_0.gguf | Q4_0 | 1.07 GB | | palmyra-mini-thinking-a.Q4_1.gguf | Q4_1 | 1.16 GB | | palmyra-mini-thinking-a.Q4_K.gguf | Q4_K | 1.12 GB | | palmyra-mini-thinking-a.Q4_K_M.gguf | Q4_K_M | 1.12 GB | | palmyra-mini-thinking-a.Q4_K_S.gguf | Q4_K_S | 1.07 GB | | palmyra-mini-thinking-a.Q5_0.gguf | Q5_0 | 1.26 GB | | palmyra-mini-thinking-a.Q5_1.gguf | Q5_1 | 1.35 GB | | palmyra-mini-thinking-a.Q5_K.gguf | Q5_K | 1.28 GB | | palmyra-mini-thinking-a.Q5_K_M.gguf | Q5_K_M | 1.28 GB | | palmyra-mini-thinking-a.Q5_K_S.gguf | Q5_K_S | 1.26 GB | | palmyra-mini-thinking-a.Q6_K.gguf | Q6_K | 1.46 GB | | palmyra-mini-thinking-a.Q8_0.gguf | Q8_0 | 1.89 GB | ### palmyra-mini-thinking-b | File Name | Quant Type | File Size | | - | - | - | | palmyra-mini-thinking-b.BF16.gguf | BF16 | 3.09 GB | | palmyra-mini-thinking-b.F16.gguf | F16 | 3.09 GB | | palmyra-mini-thinking-b.F32.gguf | F32 | 6.18 GB | | palmyra-mini-thinking-b.Q2_K.gguf | Q2_K | 676 MB | | palmyra-mini-thinking-b.Q3_K_L.gguf | Q3_K_L | 880 MB | | palmyra-mini-thinking-b.Q3_K_M.gguf | Q3_K_M | 824 MB | | palmyra-mini-thinking-b.Q3_K_S.gguf | Q3_K_S | 761 MB | | palmyra-mini-thinking-b.Q4_0.gguf | Q4_0 | 935 MB | | palmyra-mini-thinking-b.Q4_1.gguf | Q4_1 | 1.02 GB | | palmyra-mini-thinking-b.Q4_K.gguf | Q4_K | 986 MB | | palmyra-mini-thinking-b.Q4_K_M.gguf | Q4_K_M | 986 MB | | palmyra-mini-thinking-b.Q4_K_S.gguf | Q4_K_S | 940 MB | | palmyra-mini-thinking-b.Q5_0.gguf | Q5_0 | 1.1 GB | | palmyra-mini-thinking-b.Q5_1.gguf | Q5_1 | 1.18 GB | | palmyra-mini-thinking-b.Q5_K.gguf | Q5_K | 1.13 GB | | palmyra-mini-thinking-b.Q5_K_M.gguf | Q5_K_M | 1.13 GB | | palmyra-mini-thinking-b.Q5_K_S.gguf | Q5_K_S | 1.1 GB | | palmyra-mini-thinking-b.Q6_K.gguf | Q6_K | 1.27 GB | | palmyra-mini-thinking-b.Q8_0.gguf | Q8_0 | 1.65 GB | ## Quants Usage (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)