Analog Compute in Memory and Breaking Digital Number Representations

The acceleration of matrix-vector multiplications (MVMs) operation provided by graphical processing units (GPUs) has been instrumental to the success of deep neural networks (DNNs). By providing high parallelism and large memory buffers to store the weights and activations of DNNs, GPUs remain to this day the main platform to train and deploy DNNs. As most GPUs are designed for 32bit floating-point arithmetic, it is standard for modern DNNs to be executed with 32bit weights and activations.

However, recent quantization solutions for DNNs inference have shown that networks using reduced precision weights and activations can achieve similar accuracies. Exploiting these quantization solutions, digital neural networks accelerators, such has Google TPUs, Microsoft BrainWave or Nvidia Hopper architecture, are able to achieve significant time and energy gains compared to conventional GPUs. However, being tied to the progress of complementary metal-oxide semiconductor (CMOS) technology, the gains achievable by these digital accelerators are unable to match the growth of recent DNNs. To overcome this limitation, unconventional non-Von Neumann architectures for neural networks accelerators are emerging. Among them, MVM accelerators using analog compute-in-memory (ACIM) are a radical example to push DNNs’ performance through a drastic data movement and energy consumption reduction compared to traditional architecture.

Nonetheless, multiple inherent non-idealities as well as lack of explicit algorithmic support for ACIM, have so far prevented analog based neural networks from being as reliable as their digital counterparts. Whether ACIM based neural networks can be as accurate digital equivalents remains uncertain. In this thesis, we develop the argument that beyond ACIM non-idealities, part of this diminished reliability can be tied to the inability of ACIM to conform to the arithmetic assumptions of digital datatypes. By not conforming to those assumptions, ACIM fails to showcase the DNNs it deploys to the full potential exhibited by digital alternatives. Through adapted DNN quantization solutions, we show that correcting these assumptions and properly leveraging the arithmetic capacities of ACIM results in improved DNN performance and increased resiliency to ACIM non-idealities.

Extending this arithmetic view to digital DNNs, we show that bit-slicing, a design strategy commonly used in ACIM design, can be leveraged to reveal bit-level arithmetic structures within pretrained floating-point DNNs. Using post training pruning techniques on these structures, we show that new optimization tracks can be leveraged to increase the efficiency of pretrained DNNs.