George Hotz: Sam Altman wont tell you that GPT-4 has 220B parameters and is a 16-way mixture model with 8 sets of weights? by Peter Xing DataDrivenInvestor
The 1 8T trillion parameters GPT-MoE might be GPT-4 And recent updates made in October even allow phi-1.5 to display multimodality—an ability to interpret images as well as text. Last week Microsoft announced the release of phi-2, a 2.7-billion-parameter follow-up to phi-1.5, which demonstrates even more ability in a still relatively compact package, the company […]