Spaces:

CoderCowMoo
/

Llama-3-70b-Instruct-Orthoganalized

Runtime error

CoderCowMoo commited on May 5, 2024

Commit

606fa0a

verified ·

1 Parent(s): b17bd82

change device to cuda:0 same as mazaiyahr's repo

Files changed (1) hide show

app.py CHANGED Viewed

@@ -21,7 +21,7 @@ quantize_config = BaseQuantizeConfig(
 model = AutoGPTQForCausalLM.from_quantized(
         model_id,
         use_safetensors=True,
-        device="cuda",
         quantize_config=quantize_config).eval()

 model = AutoGPTQForCausalLM.from_quantized(
         model_id,
         use_safetensors=True,
+        device="cuda:0",
         quantize_config=quantize_config).eval()