Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Any GPT-3-level fully-opensource models yet?


BLOOM and OPT are both GPT-3 sized models that are open source and similarily sized (175B parameters) but a) text generation is not as good as GPT-3 for whatever reason and b) getting them to run in a cost-effective manner is not easy and with this new price decrease, it may be more economical to use GPT-3 for deployment despite that.


The reason the similarly sized open source models aren’t as good is because gpt-3 has had a 2 year head start of additional development and improvement. The gpt-3 that is available today is Much better than what was initially produced in 2020.


Is this due to entirely to economies of scale?


Not entirely. There are many ways to optimize models for deployment both at the software and hardware levels: some are public such as quantization/fusing, others are not.


Economies of scale and years of optimization, some of which likely have not been publicly shared.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: