Advanced AI in C++

Advanced AI in C++

There's also a lot of new things to learn when coding AI in C++. Here's a list of some of them:

• Vectorization. Many things you've learned about speeding up code in traditional sequential C++ is now wrong. Even half of your multi-threading or multi-process experience with parallel execution is irrelevant. You need to think “simple and parallel” with SIMD architectures on your mind. Modifying algorithms to parallelize properly on GPUs is called “vectorization.”
• Intrinsics. Optimizing AI engines means vectorizing code to work on hardware accelerators. The way you do that in C++ is to use the various intrinsic functions (aka “builtins”) in standard platforms or GPU integration libraries.
• Assembler. Sometimes even intrinsics aren't fast enough and you need to hard-code assembler into C++ source code using asm, __asm or other directives.
• Data structures. The vector is the basis of AI inference, and it's really just an array. Matrices are the 2-D version with two-dimensional arrays. Tensors generalize this to 3-dimensional arrays. For optimizations, lookup tables and bit vectors are the go-to data structures. Hash tables are occasionally used, and also vector hashing. Binary trees and tries, not so much.
• Algorithms. Vector dot product and matrix multiplication are the low-level computational routines for tensors. Vectorizing algorithms for GPU parallelization is job number one. Many other algorithms are more mathematical (e.g. exp and log) or statistical (e.g. variance and standard deviation).
• Memory management. Normal C++ CPU memory management is stretched to gigabyte-sized data structures. And the GPU memory management paradigm is very different to stack or heap memory. You tend to write huge blocks to the GPU, which remain in somewhat static structures with less swapping, and you don't ever access individual bytes.

• Next: • Up: Table of Contents • Buy: Generative AI in C++: Coding Transformers and LLMs

The new AI programming book by Aussie AI co-founders: AI coding in C++ Transformer engine speedups LLM models Phone and desktop AI Code examples Research citations Get your copy from Amazon: Generative AI in C++