Doing AI at scale is not just a question of how many parameters a model has.
It is how much hardware and energy we need to store those parameters, move them around, and repeatedly compute with them.
At 4–16-bit precision, each weight alone requir...
Akin to the myriad fundamental differences between open core companies and closed core companies (of which I’ve closely studied for > 15 years now)… the differences between biological intelligence and AI are even more profound. This is glossed ove...