Aussie AI

Residue Number System

  • Book Excerpt from "Generative AI in C++"
  • by David Spuler, Ph.D.

Residue Number System

The residue number system is an alternative method for approximating floating-point arithmetic with integers. The method represents numbers as integers using modulo arithmetic with multiple prime numbers. Hence, it is based on the “residues” (remainders) of division by multiple primes. The opportunities for efficiency when applied to AI include integer arithmetic and special properties of non-carry and parallel addition in modulo arithmetic.

Research papers on the residue number system:

  1. JK Lee, L Mukhanov, AS Molahosseini, 2023, Resource-Efficient Convolutional Networks: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques, https://dl.acm.org/doi/abs/10.1145/3587095, PDF: https://dl.acm.org/doi/pdf/10.1145/3587095
  2. Zhi-Gang Liu and Matthew Mattina, 2020, Efficient residue number system based Winograd convolution, In Proceedings of the European Conference on Computer Vision (ECCV’20). 53–6, https://arxiv.org/abs/2007.12216
  3. S. Salamat, M. Imani, S. Gupta, and T. Rosing, 2018, RNSnet: In-memory neural network acceleration using residue number system, In Proceedings of the 2018 IEEE International Conference on Rebooting Computing (ICRC’18), 1–12, https://ieeexplore.ieee.org/document/8638592
  4. N. Samimi, M. Kamal, A. Afzali-Kusha, and M. Pedram, 2020, Res-DNN: A residue number system-based DNN accelerator unit, IEEE Transactions on Circuits and Systems I: Regular Papers 67, 2 (2020), 658–671, https://ieeexplore.ieee.org/document/8903528
  5. G Alsuhli, V Sakellariou, H Saleh, M Al-Qutayri, 2023, Number Systems for Deep Neural Network Architectures: A Survey, https://arxiv.org/abs/2307.05035
  6. CY Hung, B Parhami, 1994, An approximate sign detection method for residue numbers and its application to RNS division, Computers & Mathematics with Applications, Elsevier, https://doi.org/10.1016/0898-1221(94)90052-3, https://www.sciencedirect.com/science/article/pii/0898122194900523
  7. Molahosseini AS, De Sousa LS, Chang C-H, 2017, Embedded systems design with special arithmetic and number systems, Springer. https://doi.org/10.1007/978-3-319-49742-6, https://link.springer.com/book/10.1007/978-3-319-49742-6 (A text that contains multiple papers on LNS and RNS.)
  8. V Arrigoni, B Rossi, P Fragneto, G Desoli, 2017, Approximate operations in Convolutional Neural Networks with RNS data representation, ESANN, 2017, https://www.esann.org/sites/default/files/proceedings/legacy/es2017-30.pdf
  9. Hiroki Nakahara and Tsutomu Sasao. 2015, A deep convolutional neural network based on nested residue number system, In International Conference on Field Programmable Logic and Applications, 2015, https://ieeexplore.ieee.org/document/7293933
  10. Nicholas S Szabo and Richard I Tanaka. 1967, Residue arithmetic and its applications to computer technology, McGraw-Hill, 1967, https://www.amazon.com/Residue-Arithmetic-Application-Computer-Technology/dp/0070626596/

For more research papers on the residue number system, see also https://www.aussieai.com/research/advanced-ai-mathematics#residue.

 

• Next:

• Up: Table of Contents

• Buy: Generative AI in C++: Coding Transformers and LLMs

Generative AI in C++ The new AI programming book by Aussie AI co-founders:
  • AI coding in C++
  • Transformer engine speedups
  • LLM models
  • Phone and desktop AI
  • Code examples
  • Research citations

Get your copy from Amazon: Generative AI in C++