Aussie AI

Division Optimization

  • Last Updated 17 September, 2024
  • by David Spuler, Ph.D.

Division is an expensive operation, and has been largely avoided in neural networks (with preference given to multiplication, addition and bitwise operators). However, there is some research in regard to division arithmetic. Related research areas include:

  • Division algorithms: Faster ways to implement division, now mainly for hardware designers.
  • Approximate division algorithms: see below.
  • Power-of-two quantization: Bitshifting can optimize division, as it can for multiplication. Right bitshift is an obvious optimization for integer division involving power-of-2 divisors. This has relevance in relation to logarithmic quantization.
  • Integer division: For some thoughts on the use of general integer division of weights in quantization, see division quantization.
  • Advanced number system division: See dyadic numbers and dyadic quantization for an obscure number system involving power-of-two division.

Division Algorithms and Approximate Division

Some research on fast division algorithms:

More AI Research

Read more about: