Aussie AI

Posit Number System

  • Book Excerpt from "Generative AI in C++"
  • by David Spuler, Ph.D.

Posit Number System

The posit number system is an alternative floating-point arithmetic using multiple exponent fields, where one of the fields is the “regime” of the exponent. Posit numbers have been used in neural networks as an alternative to floating-point numbers in various research papers, and the area seems close to a breakthrough into more mainstream usage. There are hardware versions of posit number computations and “posit quantization” is also possible.

Research papers on the posit number system:

  1. JK Lee, L Mukhanov, AS Molahosseini, 2023, Resource-Efficient Convolutional Networks: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques, https://dl.acm.org/doi/abs/10.1145/3587095, PDF: https://dl.acm.org/doi/pdf/10.1145/3587095
  2. Z. Carmichael, H. F. Langroudi, C. Khazanov, J. Lillie, J. L. Gustafson, and D. Kudithipudi, 2019, Deep positron: A deep neural network using the posit number system, In Proceedings of the 2019 Design, Automation, and Test in Europe Conference and Exhibition (DATE’19). 1421–1426, https://arxiv.org/abs/1812.01762
  3. Zachariah Carmichael, Hamed F. Langroudi, Char Khazanov, Jeffrey Lillie, John L. Gustafson, and Dhireesha Kudithipudi, 2019, Performance-efficiency trade-off of low-precision numerical formats in deep neural networks, In Proceedings of the 2019 Conference for Next Generation Arithmetic (CoNGA’19), ACM, New York, NY, Article 3, 9 pages, https://doi.org/10.1145/3316279.3316282
  4. G Alsuhli, V Sakellariou, H Saleh, M Al-Qutayri, 2023, Number Systems for Deep Neural Network Architectures: A Survey, https://arxiv.org/abs/2307.05035 (Survey of number systems with good coverage of Posits.)
  5. Jinming Lu, Siyuan Lu, Zhisheng Wang, Chao Fang, Jun Lin, Zhongfeng Wang, Li Du, Sep 2019, Training Deep Neural Networks Using Posit Number System, 2019 32nd IEEE International System-on-Chip Conference (SOCC), https://ieeexplore.ieee.org/abstract/document/9088105/, https://arxiv.org/pdf/1909.03831
  6. Raul Murillo, Alberto A. Del Barrio, Guillermo Botella, Min Soo Kim, HyunJin Kim, Nader Bagherzadeh, R Murillo, 2021, PLAM: A posit logarithm-approximate multiplier, https://arxiv.org/pdf/2102.09262
  7. F. de Dinechin, L. Forget, J.-M. Muller, and Y. Uguen, 2019, Posits: the good, the bad and the ugly, in Proceedings of the Conference for Next Generation Arithmetic 2019. New York, NY, USA: ACM, mar 2019, pp. 1–10. https://dl.acm.org/doi/10.1145/3316279.3316285
  8. R. Murillo, A. A. Del Barrio, and G. Botella, 2020, Deep PeNSieve: A deep learning framework based on the posit number system, Digital Signal Processing, vol. 102, p. 102762, Jul 2020. https://www.sciencedirect.com/science/article/abs/pii/S105120042030107X (An example of “posit quantization”.)
  9. H. F. Langroudi, Z. Carmichael, and D. Kudithipudi, 2019, Deep Learning Training on the Edge with Low-Precision Posits, arXiv e-prints, pp. 1474–1479, Jul 2019. https://arxiv.org/abs/1907.13216
  10. M. K. Jaiswal and H. K. So, 2018, Universal number posit arithmetic generator on FPGA, in 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE), vol. 2018-Janua. IEEE, mar 2018, pp. 1159–1162. https://ieeexplore.ieee.org/document/8342187
  11. R. Chaurasiya et al., 2018, Parameterized Posit Arithmetic Hardware Generator, in 2018 IEEE 36th International Conference on Computer Design (ICCD). IEEE, oct 2018, pp. 334–341. https://ieeexplore.ieee.org/document/8615707
  12. M. K. Jaiswal and H. K. So, 2019, PACoGen: A hardware posit arithmetic core generator, IEEE Access, vol. 7, pp. 74 586–74 601, 2019. https://ieeexplore.ieee.org/document/8731915
  13. Y. Uguen, L. Forget, and F. de Dinechin, 2019, Evaluating the Hardware Cost of the Posit Number System, in 2019 29th International Conference on Field Programmable Logic and Applications (FPL). IEEE, Sep 2019, pp. 106–113. https://ieeexplore.ieee.org/document/8892116
  14. R. Murillo, A. A. Del Barrio, and G. Botella, 2020, Customized posit adders and multipliers using the FloPoCo core generator, in 2020 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, oct 2020, pp. 1–5. https://ieeexplore.ieee.org/document/9180771
  15. M. Cococcioni, F. Rossi, E. Ruffaldi, and S. Saponara, 2020, Fast deep neural networks for image processing using posits and ARM scalable vector extension, Journal of Real-Time Image Processing volume 17, pages 759–771 (2020), https://link.springer.com/article/10.1007/s11554-020-00984-x
  16. Jinming Lu; Chao Fang; Mingyang Xu; Jun Lin; Zhongfeng Wang, 2020, Evaluations on Deep Neural Networks Training Using Posit Number System, IEEE Transactions on Computers, vol. 14, no. 8, pp. 1–1, 2020. https://ieeexplore.ieee.org/document/9066876
  17. M. K. Jaiswal and H. K. So, 2018, Architecture Generator for Type-3 Unum Posit Adder/Subtractor, in 2018 IEEE International Symposium on Circuits and Systems (ISCAS), vol. 2018-May. IEEE, 2018, pp. 1–5. https://ieeexplore.ieee.org/document/8351142
  18. J. L. Gustafson and I. Yonemoto, 2017, Beating Floating-Point at its Own Game: Posit Arithmetic, Supercomputing Frontiers and Innovations, vol. 4, no. 2, pp. 71–86, Jun 2017, https://dl.acm.org/doi/10.14529/jsfi170206, PDF: https://gwern.net/doc/ai/nn/sparsity/low-precision/2017-gustafson.pdf
  19. S. H. F. Langroudi, T. Pandit and D. Kudithipudi, 2018, Deep learning inference on embedded devices: Fixed-point vs posit, Proc. 1st Workshop Energy Efficient Mach. Learn. Cognit. Comput. Embedded Appl. (EMC2), pp. 19-23, Mar. 2018. https://arxiv.org/abs/1805.08624

For more research papers on the posit number system, see also https://www.aussieai.com/research/advanced-ai-mathematics#posit.

 

• Next:

• Up: Table of Contents

• Buy: Generative AI in C++: Coding Transformers and LLMs

Generative AI in C++ The new AI programming book by Aussie AI co-founders:
  • AI coding in C++
  • Transformer engine speedups
  • LLM models
  • Phone and desktop AI
  • Code examples
  • Research citations

Get your copy from Amazon: Generative AI in C++