A request if you are inclined
Hello helpful reaper fellow. I see you are doing the gemma moe. Fantastic! Thank you :)
Any chance you also do a 16-17b reap, like ~35%? Suspect this will fit on fully on 8gb cards like mine without too much additionally lobotomizing quantization.
If not, all good, and have a great day.
Hey! I will be trying a REAP model hopefully within the next 7 days just because of work.
Hey! I will be trying a REAP model hopefully within the next 7 days just because of work.
Sero has done a 20 and a 30. If you do happen to do a 35% reap, let us know! I feel like that's probs going to be optimal for my paltry 8gb. I got excited when I saw the Gemma 4 moe's because I though, just a little reap and that bad boy is going to fit in vram and run smooth.
(And probably be a stupid request given it's for work, but if you happen to heretic that thing afterward, even better, but do not expect anything at all).
Sigh, one day I'll be vram rich, lol. Thanks for chiming in :) Have a good one.