Aussie AI

Elements of Memory Optimization

  • Book Excerpt from "Generative AI in C++"
  • by David Spuler, Ph.D.

Elements of Memory Optimization

Before we delve into complex AI memory management optimizations like model compression or kernel tiling, let's look at the fundamental building blocks. How do we represent data in memory inside an AI engine to promote memory efficiency?

Some of the lower-level types of memory optimizations include:

  • Contiguous memory blocks
  • Linearizing multidimensional matrices/tensors
  • C++ memory management optimizations

For the last category, there are also a variety of basic C++ programming techniques to minimize CPU memory usage overall. See chapter 14 for information on C++ memory reduction.

 

• Next:

• Up: Table of Contents

• Buy: Generative AI in C++: Coding Transformers and LLMs

Generative AI in C++ The new AI programming book by Aussie AI co-founders:
  • AI coding in C++
  • Transformer engine speedups
  • LLM models
  • Phone and desktop AI
  • Code examples
  • Research citations

Get your copy from Amazon: Generative AI in C++