Tropical Algebra (Max-Plus)

Tropical Algebra (Max-Plus)

Tropical algebra is a number system based on addition and maximum operations. The “max-plus” tropical algebra can be used to analyze zero-multiplication AI models that use addition and maximum functions; see Max-Plus networks. There is also a “min-plus” tropical algebra using minimum and addition operations. The tropical algebra is also closely related to the “minimax” algebra.

Some other areas of theory are somewhat related to tropical algebra. One method to approximate Logarithmic Number System (LNS) addition is to use maximum and addition. Also related is the calculation of Softmax, so tropical algebra may have relevance to approximation of Softmax using maximum and addition.

Research papers on tropical algebra:

1. L. Zhang, G. Naitzat, and L.-H. Lim, 2018, Tropical geometry of deep neural networks, in Proc. Int’l Conf. on Machine Learning, vol. 80, pp. 5824–5832, PMLR, 2018. https://arxiv.org/abs/1805.07091 (Analysis of neural network architecture using tropical algebra.)
2. P. Maragos, V. Charisopoulos, and E. Theodosis, 2021, Tropical Geometry and Machine Learning, https://ieeexplore.ieee.org/document/9394420
3. G Smyrnis, P Maragos, 2019, Tropical polynomial division and neural networks, arXiv preprint arXiv:1911.12922, https://arxiv.org/abs/1911.12922 (Lots of tropical algebra theory, but also a neural network approximation tested.)
4. V. Charisopoulos and P. Maragos, 2018, A tropical approach to neural networks with piecewise linear activations, arXiv preprint arXiv:1805.08749, https://arxiv.org/abs/1805.08749
5. Diane Maclagan and Bernd Sturmfels. Introduction to tropical geometry, volume 161. American Mathematical Soc., 2015, https://bookstore.ams.org/gsm-161
6. Wikipedia, 2023, Tropical geometry, https://en.wikipedia.org/wiki/Tropical_geometry
7. Smyrnis G and Maragos P., 2020, Multiclass neural network minimization via tropical Newton polytope approximation, Proceedings of the 37th International Conference on Machine Learning. (9068-9077). /doi/10.5555/3524938.3525779

For more research papers on the tropical algebra, see also https://www.aussieai.com/research/advanced-ai-mathematics#tropical.

• Next: • Up: Table of Contents • Buy: Generative AI in C++: Coding Transformers and LLMs

The new AI programming book by Aussie AI co-founders: AI coding in C++ Transformer engine speedups LLM models Phone and desktop AI Code examples Research citations Get your copy from Amazon: Generative AI in C++