Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Wouldn't the sensible default be to use 80% of available VRAM, or total VRAM minus 2GB, or something along those lines. Something that's a tad conservative but works for 99% of cases, with tuning options for those who want to fly closer to the sun.


2GB is a huge amount - you'd be dropping a dozen layers. Saving a few MB should be sufficient, and a layer is generally going to be orders of megabytes, so unless your model fits perfectly into VRAM (using 100%) you're already going to be leaving at least a few MB / 10s of MBs/ 100s of MBs free.

Your window manager will already have reserved its vRAM upfront so it isn't a big deal to use ~all of the rest.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: