Your Desktop AI Engine

Your Desktop AI Engine

This chapter is about using C++ to get AI running on your developer desktop. Native execution of an AI engine on a basic developer PC is already possible, and all of the C++ code runs locally. You can play with all of the C++ code, try to understand it all, and make changes to optimize things.

But don't get too excited: it's not very fast.

The state-of-the-art is that developers can run smaller models locally on a fairly high-spec developer box. The ability to run the better and bigger models directly on a PC is still in the future, and there aren't yet any major software applications that run the AI engine natively on the PC.

But don't get frustrated — it's exciting! We're close to that capability, and in that direction is where the major opportunity for advancement remains. We just need to tweak the C++ code a little.

• Next: • Up: Table of Contents • Buy: Generative AI in C++: Coding Transformers and LLMs

The new AI programming book by Aussie AI co-founders: AI coding in C++ Transformer engine speedups LLM models Phone and desktop AI Code examples Research citations Get your copy from Amazon: Generative AI in C++