Quantized model shards

[vrojkova1 (Viktoria)]

How to shard quantized model to load to different GPUs?