arright so i made it lightweight by manually passing the dtype to match the one i saw when loading and it loads super quick because it's just mmaping memory. > model = AutoDistributedModelForCausalLM.from_pretrained(model_name, > torch_dtype='bfloat16') but then it complains nobody is running the model which matches what the health webpage said. 1646 1654 ok so, at least on my internet, petals is kaput right now no activity in discord their official chat app complains it has no layer providers :s
[sounds familiar, they shoulda put it on a blockchain or used gnunet ;D
