Aussie AI

9-Bit Quantization (INT9)

  • Book Excerpt from "Generative AI in C++"
  • by David Spuler, Ph.D.

9-Bit Quantization (INT9)

Research papers on 9-bit quantization:

  1. M Giacobbe, TA Henzinger, M Lechner, 2020, How many bits does it take to quantize your neural network?, TACAS 2020, https://link.springer.com/chapter/10.1007/978-3-030-45237-7_5, PDF: https://link.springer.com/content/pdf/10.1007/978-3-030-45237-7_5.pdf (Ran experiments from 6-bit to 10-bit quantization.)
  2. W Jiang, P Liu, F Wen, 2017, An improved vector quantization method using deep neural network, AEU - International Journal of Electronics and Communications, Volume 72, February 2017, Pages 178-183, https://www.sciencedirect.com/science/article/pii/S1434841116313954

See more papers on 9-bit quantization (INT9) at: https://www.aussieai.com/research/quantization#int9

 

• Next:

• Up: Table of Contents

• Buy: Generative AI in C++: Coding Transformers and LLMs

Generative AI in C++ The new AI programming book by Aussie AI co-founders:
  • AI coding in C++
  • Transformer engine speedups
  • LLM models
  • Phone and desktop AI
  • Code examples
  • Research citations

Get your copy from Amazon: Generative AI in C++