Spaces:

CoderCowMoo
/

Llama-3-70b-Instruct-Orthoganalized

Runtime error

CoderCowMoo commited on May 5, 2024

Commit

332c1a1

verified ·

1 Parent(s): 606fa0a

Maybe remove device?

Files changed (1) hide show

app.py CHANGED Viewed

@@ -21,7 +21,6 @@ quantize_config = BaseQuantizeConfig(
 model = AutoGPTQForCausalLM.from_quantized(
         model_id,
         use_safetensors=True,
-        device="cuda:0",
         quantize_config=quantize_config).eval()

 model = AutoGPTQForCausalLM.from_quantized(
         model_id,
         use_safetensors=True,
         quantize_config=quantize_config).eval()