The State of AI

The State of AI

There's so much going on in the AI industry that these words are out-of-date the second that I type them. Nevertheless, here are a few general thoughts on where we are:

AI is amazing. I'm still astounded by the capabilities of the latest AI apps, whether it's in creating fluent text or vibrant realistic images. There are so many advances happening in other areas such as speech, vision, animation, and video. The whole industry is evolving rapidly at such speed that I need an AI copilot to help me keep up with all the news.

AI is expensive. Remember the joke about how “boat” stands for “Bring Out Another Thousand”? That's nothing compared to AI. A single GPU costs more than your boat and a typical motherboard has eight of them. And the big companies have been buying these by the thousands. What should LLM stand for? “Lavish Leviathan Mammoth”? That was mine. “Ludicrously Large Mango” was Bing Chat with GPT-4's AI suggestion. Neither are great, which is comforting because it means there's still some work to be done.

AI is not new. The AI-related workload hosting market is many years old. Just because GenAI has blasted into consumer consciousness, and into boardroom discussions as a result, doesn't mean that AI is new. The cloud hosting companies like Amazon AWS, Microsoft Azure, and Google GCP, have been doing AI workloads for many customers, for many years. Instead of using GPUs for GenAI, they've been running workloads in other AI areas like Machine Learning (ML), machine vision (e.g. Tesla autonomous cars), product suggestion feeds, predictive modeling, auto-completion of search queries, and so on. There were already billions of dollars invested in AI long before ChatGPT set the web on fire.

AI Phones. AI is going to be on your phone, and it's going to be a big driver of new phone purchases. There are already low-end AI models that can run on your desktop PC, but it's not really true yet of phones. We're at the start of AI adoption inside phone apps, but there aren't many examples yet. See Chapter 3 for more about AI phones.

AI PCs. AI models and applications are set to make PCs hot again in the near-term. The next generation of laptops and desktops will likely run some AI models natively, and there will also be hybrid architectures with AI workloads offloaded into the cloud. The first generation is likely to include “AI Developer PCs” because software developers typically have high-end PCs, and various existing AI models can already run on desktop PCs. For end user applications, the model still has to run fast to give the user a decent response time, so there are still some significant obstacles for AI models on non-developer PCs, but hybrid cloud architectures will likely hide a lot of the limitations of native AI execution. It is early days in this trend, but it's surely going to be a major technology driver for years to come. See Chapter 4 for how to run a C++ AI engine on your desktop PC.

Green AI. The widespread use of AI makes it a significant contributor to energy consumption, and there is much research on the environmental impact from AI computing. On the positive side, this means that all of the research towards AI improvements is helpful for green AI, since it will also reduce its carbon footprint and environmental impacts. All of those C++ code optimizations to speed up the AI engine are also making things greener overall.

• Next: • Up: Table of Contents • Buy: Generative AI in C++: Coding Transformers and LLMs

The new AI programming book by Aussie AI co-founders: AI coding in C++ Transformer engine speedups LLM models Phone and desktop AI Code examples Research citations Get your copy from Amazon: Generative AI in C++