Binarized Weights as a Scaling Breakthrough
October 20, 2023

Archived from an original LinkedIn post by Brian Greenforest.

Original Post

One of the most important contributions to computer science, machine learning, and automated digital circuit design genetic programming was made by Yoshua Bengio in late 2015. Geoffrey Hinton gave us BPTT in 1985 that enabled RNN backprop. Bengio gave us binarized weights neural networks that scale. Now the team deploys and trains Transformer LLMs on specialized chips that run 15 times faster in Israel. Microsoft Research China forgot to mention the shoulders of the giants in BitNet paper just published few days ago.

https://lnkd.in/gM2z4Qe5

Links From the Original Post