Hugging Face's logo

nicoboss
/
hpc-coder-v2-16b-non-MLA-GGUF

Transformers
GGUF
English
code
hpc
parallel
axonn
conversational
Model card Files
xet
Community
nicoboss mradermacher commited on
Commit
998efaa
·
verified ·
0 Parent(s):

Duplicate from mradermacher/hpc-coder-v2-16b-GGUF

Browse files

Co-authored-by: team mradermacher <mradermacher@users.noreply.huggingface.co>

Files changed (13) hide show
  1. .gitattributes +46 -0
  2. README.md +71 -0
  3. hpc-coder-v2-16b.IQ4_XS.gguf +3 -0
  4. hpc-coder-v2-16b.Q2_K.gguf +3 -0
  5. hpc-coder-v2-16b.Q3_K_L.gguf +3 -0
  6. hpc-coder-v2-16b.Q3_K_M.gguf +3 -0
  7. hpc-coder-v2-16b.Q3_K_S.gguf +3 -0
  8. hpc-coder-v2-16b.Q4_K_M.gguf +3 -0
  9. hpc-coder-v2-16b.Q4_K_S.gguf +3 -0
  10. hpc-coder-v2-16b.Q5_K_M.gguf +3 -0
  11. hpc-coder-v2-16b.Q5_K_S.gguf +3 -0
  12. hpc-coder-v2-16b.Q6_K.gguf +3 -0
  13. hpc-coder-v2-16b.Q8_0.gguf +3 -0
.gitattributes ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ hpc-coder-v2-16b.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
37
+ hpc-coder-v2-16b.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
38
+ hpc-coder-v2-16b.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ hpc-coder-v2-16b.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
40
+ hpc-coder-v2-16b.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
41
+ hpc-coder-v2-16b.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
42
+ hpc-coder-v2-16b.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
43
+ hpc-coder-v2-16b.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
44
+ hpc-coder-v2-16b.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
45
+ hpc-coder-v2-16b.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
46
+ hpc-coder-v2-16b.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: hpcgroup/hpc-coder-v2-16b
3
+ datasets:
4
+ - hpcgroup/hpc-instruct
5
+ - ise-uiuc/Magicoder-OSS-Instruct-75K
6
+ - nickrosh/Evol-Instruct-Code-80k-v1
7
+ language:
8
+ - en
9
+ library_name: transformers
10
+ quantized_by: mradermacher
11
+ tags:
12
+ - code
13
+ - hpc
14
+ - parallel
15
+ - axonn
16
+ ---
17
+ ## About
18
+
19
+ <!-- ### quantize_version: 2 -->
20
+ <!-- ### output_tensor_quantised: 1 -->
21
+ <!-- ### convert_type: hf -->
22
+ <!-- ### vocab_type: -->
23
+ <!-- ### tags: -->
24
+ static quants of https://huggingface.co/hpcgroup/hpc-coder-v2-16b
25
+
26
+ <!-- provided-files -->
27
+ weighted/imatrix quants are available at https://huggingface.co/mradermacher/hpc-coder-v2-16b-i1-GGUF
28
+ ## Usage
29
+
30
+ If you are unsure how to use GGUF files, refer to one of [TheBloke's
31
+ READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
32
+ more details, including on how to concatenate multi-part files.
33
+
34
+ ## Provided Quants
35
+
36
+ (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
37
+
38
+ | Link | Type | Size/GB | Notes |
39
+ |:-----|:-----|--------:|:------|
40
+ | [GGUF](https://huggingface.co/mradermacher/hpc-coder-v2-16b-GGUF/resolve/main/hpc-coder-v2-16b.Q2_K.gguf) | Q2_K | 6.5 | |
41
+ | [GGUF](https://huggingface.co/mradermacher/hpc-coder-v2-16b-GGUF/resolve/main/hpc-coder-v2-16b.Q3_K_S.gguf) | Q3_K_S | 7.6 | |
42
+ | [GGUF](https://huggingface.co/mradermacher/hpc-coder-v2-16b-GGUF/resolve/main/hpc-coder-v2-16b.Q3_K_M.gguf) | Q3_K_M | 8.2 | lower quality |
43
+ | [GGUF](https://huggingface.co/mradermacher/hpc-coder-v2-16b-GGUF/resolve/main/hpc-coder-v2-16b.Q3_K_L.gguf) | Q3_K_L | 8.6 | |
44
+ | [GGUF](https://huggingface.co/mradermacher/hpc-coder-v2-16b-GGUF/resolve/main/hpc-coder-v2-16b.IQ4_XS.gguf) | IQ4_XS | 8.7 | |
45
+ | [GGUF](https://huggingface.co/mradermacher/hpc-coder-v2-16b-GGUF/resolve/main/hpc-coder-v2-16b.Q4_K_S.gguf) | Q4_K_S | 9.6 | fast, recommended |
46
+ | [GGUF](https://huggingface.co/mradermacher/hpc-coder-v2-16b-GGUF/resolve/main/hpc-coder-v2-16b.Q4_K_M.gguf) | Q4_K_M | 10.5 | fast, recommended |
47
+ | [GGUF](https://huggingface.co/mradermacher/hpc-coder-v2-16b-GGUF/resolve/main/hpc-coder-v2-16b.Q5_K_S.gguf) | Q5_K_S | 11.2 | |
48
+ | [GGUF](https://huggingface.co/mradermacher/hpc-coder-v2-16b-GGUF/resolve/main/hpc-coder-v2-16b.Q5_K_M.gguf) | Q5_K_M | 12.0 | |
49
+ | [GGUF](https://huggingface.co/mradermacher/hpc-coder-v2-16b-GGUF/resolve/main/hpc-coder-v2-16b.Q6_K.gguf) | Q6_K | 14.2 | very good quality |
50
+ | [GGUF](https://huggingface.co/mradermacher/hpc-coder-v2-16b-GGUF/resolve/main/hpc-coder-v2-16b.Q8_0.gguf) | Q8_0 | 16.8 | fast, best quality |
51
+
52
+ Here is a handy graph by ikawrakow comparing some lower-quality quant
53
+ types (lower is better):
54
+
55
+ ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
56
+
57
+ And here are Artefact2's thoughts on the matter:
58
+ https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
59
+
60
+ ## FAQ / Model Request
61
+
62
+ See https://huggingface.co/mradermacher/model_requests for some answers to
63
+ questions you might have and/or if you want some other model quantized.
64
+
65
+ ## Thanks
66
+
67
+ I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
68
+ me use its servers and providing upgrades to my workstation to enable
69
+ this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
70
+
71
+ <!-- end -->
hpc-coder-v2-16b.IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bc9795f929f8ad914e3a3340526b52f79f91ecfdae79296010c3980b83f1508a
3
+ size 8646805856
hpc-coder-v2-16b.Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8e1cac52bb7a6a7c3842665714cbbe1e2536c371ee26f0cfcca06f899ee898c1
3
+ size 6430466400
hpc-coder-v2-16b.Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:457e8b5905317d85e69c87f83150754e21f910174d6ec8c2a28e33db81a4da97
3
+ size 8459400544
hpc-coder-v2-16b.Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:95e1d1fd2db8c1631a593f34bc906772850de644d681b3fca6b7bd9eed9a1de8
3
+ size 8126608736
hpc-coder-v2-16b.Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d41b67e87d3b5ff2b91791b9b48daa3bcd6a6b342408f041f2e9828a87c5c614
3
+ size 7487665504
hpc-coder-v2-16b.Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9f65ebbf84294b2232f766cceb0caf02cb60c1ebc31fd30ca8893a3a206e05e1
3
+ size 10364418400
hpc-coder-v2-16b.Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:072301c47a1ca73bf521fd51845cf8000dc1a9999f5fb09f927fef1d61be1f02
3
+ size 9533610336
hpc-coder-v2-16b.Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:438e07f78aa82dbed2db37caf06b4ed5bb5efc57fdef9e567230f9d3cb610ff5
3
+ size 11851315552
hpc-coder-v2-16b.Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:209cb2f6d292b3406bd532bb96bfae8b9ce5313a34c4a1872db2f26c92698db2
3
+ size 11143059808
hpc-coder-v2-16b.Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:603c24f54366d74357d3f102a747dbdb4441e102ef3cf20e6ffa6e22761273ab
3
+ size 14066974048
hpc-coder-v2-16b.Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ded996f63a3ca740c09841b50ebbb0be0d942b9c6732e8787240380e97cbf404
3
+ size 16702519648