Aussie AI
Bitwise Operators for Inference
-
Book Excerpt from "Generative AI in C++"
-
by David Spuler, Ph.D.
Bitwise Operators for Inference
Instead of addition, could any of the bitwise operations be used to replace multiplication? Yes, and there are various possibilities.
Bitshift operators: The bitwise shift operators are good candidates for replacing multiplication (or division) by a power-of-2 integer, as discussed under power-of-two quantization (logarithmic quantization). This is a well-known optimization and has considerable research.
Bitwise-or is a possible candidate, that doesn't seem to be considered in the research. When applied to positive integers, bitwise-or has some properties similar to addition, but its result will either equal or be less than an addition result, but greater than or equal to either of the two operands. This assumes bitwise-or for unsigned weights, such as via integer quantization to positive weights, because bitwise-or on negative signed numbers has various oddities. As such, bitwise-or with quantization could be considered for some of the above algorithms that use addition to replace multiplication. The accumulated sum based on bitwise-or would increase slightly more slowly than with pure addition.
Bitwise-and is another candidate, similar to bitwise-or, as it will also be less than or equal to the addition result, but the result will be less than or equal to either operand. This seems less desirable than bitwise-or, but it's all a bit unclear, and experiments are required.
Bitwise-xor seems too odd for realistic usage. It has rather strange mathematical properties. But, it has been used in various optimizations..
The use of the bitwise operators (or/and/xor) with quantization for non-multiplication inference is an area that needs some exploration. No papers were found yet.
|
• Next: • Up: Table of Contents |
|
The new AI programming book by Aussie AI co-founders:
Get your copy from Amazon: Generative AI in C++ |