Title: Demystifying and Generalizing BinaryConnect 


Authors: Tim Dockhorn, Yaoliang Yu, Eyyüb Sari, Mahdi Zolnouri, Vahid
Partovi Nia

Abstract: BinaryConnect (BC) and its many variations have become the de
facto standard for neural network quantization. However, our understanding
of the inner workings of BC is still quite limited. We attempt to close this
gap in four different aspects: (a) we show that existing quantization
algorithms, including post-training quantization, are surprisingly similar
to each other; (b) we argue for proximal maps as a natural family of
quantizers that is both easy to design and analyze; (c) we refine the
observation that BC is a special case of dual averaging, which itself is a
special case of the generalized conditional gradient algorithm; (d)
consequently, we propose ProxConnect (PC) as a generalization of BC and we
prove its convergence properties by exploiting the established connections.
We conduct experiments on CIFAR-10 and ImageNet, and verify that PC achieves
competitive performance.