Scalability

Scalability

Almost this entire treatise is about scalability of your AI engine. Getting that behemoth to run fast is the biggest challenge.

But the actual engine is not the only scalability concern. There's also the server on which you receive and process requests, sending them on to the AI engine, and collating returned results. This is a piece of software, and it could be an off-the-shelf server, or you could write your own in C++ if you like.

User interfaces are another overlooked point in regard to scalability. Not only must the backend be fast, but the user interface layer must handle all of the requirements in a way that people can cope with. The key point is this:

Humans don't scale.

What that means is that making your human user do anything is a hard problem. People cannot read reams of text fast, they cannot click on a thousand warning messages, and they do dumb things in the interface, like re-clicking the “Load” button a hundred times if it's taking too long. The fact that a human is part of the process flow means that you have to make sure that all of your steps are human-friendly. This is an often-underestimated aspect of scalability.

• Next: • Up: Table of Contents • Buy: Generative AI in C++: Coding Transformers and LLMs

The new AI programming book by Aussie AI co-founders: AI coding in C++ Transformer engine speedups LLM models Phone and desktop AI Code examples Research citations Get your copy from Amazon: Generative AI in C++