GenAI Market Evolution

How will the market for generative AI ("genAI"), such as ChatGPT and Bard, evolve over the coming years? Consumers will surely benefit from AI empowerment in numerous ways, but this article looks at how it will play out for businesses.

Update: April 2024

Most of the below analysis is from October, 2023. The new points in the last six months include:

On-device inference. A strong drive to run models on "edge" devices is spawning a lot of research in the area of on-device inference. This is mainly for the "AI Phones" and "AI PCs" trends being driven by phone and PC vendors, but it also affects other "edge" devices such as cars, IoT, and small embedded devices. See on-device inference research.

Training may be less important. There are three factors that indicate that my original comments about training being more important than inference may be incorrect:

• Someone else's model (SEM). There is widespread use of foundation models, either commercial APIs like OpenAI, or open-source models like Meta or Mistral, that mean companies don't need to train their own foundation model.
• RAG architectures vs fine-tuning. The prevailing sentiment seems to be the use of RAG architectures with database lookups rather than fine-tuning. This reduces the amount of training needed for fine-tuning approaches.
• Multi-AI engines. The new big thing is "Mixture-of-Experts" (MoE) architectures, as used by OpenAI GPT-4 and Google Gemini, which involves the use of multiple models at once for a single query. The rise of multi-AI engines leads not only to more training, but has a multiplier effect on the number of inference queries being done, with many executions per user query. It seems likely that the rise in inference will outpace the extra training needed for these architectures.

Custom AIs. There are various projects doing "custom AI apps" such as OpenAI's GPTs or Baidu and others. Expect to see more in the "no-code AI" space.

Agents. There's a lot more about AI "agents" including "autonomous intelligent agents" that use AI models underneath.

And now we return to regular station programming from 2023...

Important Points about the GenAI Market

AI is not new. The AI-related workload hosting market is many years old. Just because GenAI has blasted into consumer consciousness, and into boardroom discussions as a result, doesn't mean that AI is new. The cloud hosting companies like Amazon AWS, Microsoft Azure, and Google GCP, have been doing AI workloads for many customers, for many years. Instead of using GPUs for GenAI, they've been running workloads in other AI areas like Machine Learning (ML), machine vision (e.g. Tesla autonomous cars), product suggestion feeds, predictive modeling, and so on. There was already billions of dollars invested in AI long before ChatGPT set the web on fire.

B2B market opportunity trumps B2C. The massive ramp-up of consumer engagement with ChatGPT has made the consumer side seem hot. However, it's actually more likely to be the business side that makes more money (as usual). Predictions of the billions, maybe trillions, of dollars of benefit to economies through full AI integration into businesses, dwarf the predictions for consumer opportunities. And the B2B segment is where Microsoft has a significant advantage over other B2C players, such as Google and Amazon, although both those companies also have large B2B businesses in their cloud hosting segments (AWS and GCP).

It's a marathon, not a sprint. Consumers may continue to adopt genAI quickly, but that's not the most likely case for businesses. Whereas genAI is a hot topic in boardrooms, most business are still trying to find their feet in the area, with only exploratory projects launching. Small businesses and professionals (e.g. doctor's offices) will take years to adopt genAI, and larger enterprises will take even longer. There will be some early projects, sure, but the bulk of the B2B AI market will evolve more slowly. Projections for the B2B side of AI are over many years, even decades, with high CAGR. We've already seen this in the business adoption of cloud architectures, which is still ongoing, despite having been running since the early 2000's. The B2B AI market is likely to sustain very strong growth through 2030 and probably even into the 2040s and beyond.

Training is the big B2B market. Although fast inference response time matters to users, the underlying cost is likely to be dominated by training. Current training methods take a long time, a lot of GPUs, and cost a great deal. Most business AI projects will involve training, which includes fine-tuning, using proprietry data that the business owns. For example, a support chatbot has to be trained on the businesses products, or an internal HR chatbot needs to use internal policy documents. As such, most business projects will involve more training cost than inference cost. Even the B2C genAI bots need continuous training and fine-tuning, to keep up with current events.

How Will the AI Market Evolve?

What are the possible outcomes? Will there be one winner? Let's consider the existing segments.

Existing Incumbent Players: First, consider all the various big players in AI (in terms of both models and hosting):

• Microsoft Azure and OpenAI ChatGPT. OpenAI led the way with ChatGPT, and Microsoft is a strategic multi-billion dollar investor in OpenAI. It is the second biggest cloud hoster with Microsoft Azure (behind AWS). Microsoft also has a broad range of businesses in both B2C and B2B, including Micrsoft Office software, the Bing search engine, and the XBox gaming division, to name a few.
• Google Bard. Alphabet is the company name, but Google search is still its main business. The B2C business dwarfs its B2B business, but its Google Cloud Platform (GCP) runs a solid third in hosting. Google started a lot of the early AI research, open-sourcing most of it, notably including the Transformer architecture in 2017.
• Amazon AWS/Bedrock/Titan. Although the Amazon B2C e-commerce website is a long-standing Internet icon, its main B2B play is the AWS cloud hosting, which is the leader of the pack. AWS has been hosting cloud workloads for years, and has its own GenAI offerings in terms of Bedrock with Titan, along with partnerships with third-party AI vendors.

And consider the maverick players in this market:

• Meta/Facebook LLama (open-sourced). Meta has a very solid AI play in the LLama models, but it decided to open-source these, rather than attempt to monetize them and compete directly with ChatGPT. Interestingly, it has a partnership with Microsoft to host Llama, even though Microsoft is aligned with OpenAI. Facebook's business is primarily B2C, with revenue from online advertising, with many considering them to be in a duopoly with Google that dominates the online ads market.
• The open-source AI developers. There's an ongoing discussion about whether the open-source AI models will "win" the battle for AI, rather than Microsoft or Google. Certainly, Meta's decision to open-source its models adds fuel to that fire.

Consider the other major cloud hosters, infrastructure providers, and database vendors:

• Oracle. Without a major foundation model in the mix, Oracle is nevertheless a major play on the business side with its cloud platform and established leading database.
• IBM. Even with its global consulting business spun-off as Kyndryl, IBM maintains a solid place in AI, through its Watson AI platform, and major database software.
• Salesforce. As a cloud-first company, Salesforce has a significant play in the hosting arena, with its focus entirely on the B2B segment.
• SAP. The German giant has longstanding database and ERP capabilities, which bring it into the mix of AI for B2B. SAP seems to have a parternship strategy for GenAI, with investments in a trio of AI foundation model companies (Cohere, Anthropic, and Aleph Alpha).
• VMware. As a pioneer of cloud virtualization of software, VMware has a play in terms of hosting for AI platforms, whether virtual or bare metal.

Consider the large companies that already use AI in their products:

• Apple. Despite not yet announcing any foundation model with GenAI capabilities, Apple should not be underestimated. Many of its existing products are underpinned by AI models (e.g. face recognition), and CEO Tim Cook has stated that they are working on a lot of areas in relation to GenAI.
• Adobe. The text generation capabilities of GenAI seem to get more headlines, but there's also plenty of image-related capabilities, which is Adobe's stronghold. They also have a bigger B2B opportunity for the use of AI in their business analytics division.

Consider the top few Chinese AI players:

• Alibaba. A large B2C and B2B player, Alibaba has its e-commerce platform, and also a cloud hosting business.
• Tencent. A large Chinese tech conglomerate, Tencent has not only the massively dominant WeChat app in China, but a significant cloud hosting business, along with its gaming division for B2C. Tencent has also announced that its own GenAI foundation model will be forthcoming in late 2023.
• Baidu. One of the earliest foundation models to come online was from Chinese search giant Baidu, with its Ernie Bot in the B2C market, and later its Wenxin Qianfan platform in the B2B AI market.

New Entrant Companies: There are many new startups entering the various AI market segments, including:

• AI Foundation Models
• AI-specific GPU Hosting (underneath)
• AI-specific applications (on top)
• AI B2C companies (new applications)
• AI B2B companies (helping businesses use AI)

Conclusions

Prediction: Big cloud hosters win. The biggest cloud hosting companies win even if they lose. Even if ChatGPT disappears along with Microsoft's Bing AI, Google Bard fails to gain traction, and Amazon's Bedrock Titan doesn't catch fire, such as if all of them lose out in the long run to free open-source AI technologies, don't worry, they'll be fine. The majority of AI workloads are still likely to be run on AWS, Azure, or GCP, even if the higher levels of the tech stack are some other AI technology. And if any of the smaller GPU hosting entrants get any significant traction, these large companies have the money to buy them.

Prediction: AI hardware companies win. Similarly, the hardware-level AI chip companies underneath, notably NVidia with the lead at present, will benefit, no matter how AI market evolution plays out. There will be a battle for market share and technological dominance, but breakthroughs in hardware take longer than software.

Prediction: AI Application Companies Emerge as a Market Segment. Just as a new generation of companies arose as a layer above the cloud hosting companies (e.g. Databricks, Snowflake, Nutanix), so too will a new group of AI application companies. This will occur in both the B2B and B2C spaces, and company sizes will vary depending on the sub-markets they serve. There will be B2C AI companies in numerous use cases, and B2B AI companies in GPU hosting, "AI data" companies, and integration/consulting companies. Some incumbents will achieve this with products, too, such as Microsoft 365 Copilot, but mostly they will be in the current crop of new AI startups. They'll be at the top of the AI tech stack, and there will be significant IPOs for such new companies, but they won't be making the most money from AI. That accolade will go to their main suppliers, the big AI hosting companies and hardware chip companies.

Prediction: American AI companies will struggle in China. There are several major Chinese companies already strong in AI (e.g. TenCent, Alibaba), enjoying some significant defensive barriers in their home market, so companies from America or other Western nations will struggle to gain traction in the Chinese market. Similarly in reverse, American AI companies will dominate the US market, and Chinese AI companies will struggle with market share in USA.

Prediction: European AI market evolution is unclear. The European market is fragmented compared to the US market, and the end results for AI market share are unclear. Some EU countries already have a solid AI industry, notably France, but every country is doing something in the space. Non-English markets are not always a guaranteed win for American companies, but interestingly, ChatGPT and Bard already speak multiple foreign languages, so the future remains to be resolved.

Prediction: Asian AI market evolution is unclear. Other than China, predictions for the massive markets in India, Japan and South Korea and other nearby markets are unclear. The current crop of GenAI technology doesn't work as well in Asian languages, which have a very large set of characters (rather than only 26 in English, and only a small number more for European languages). However, India is less technologically problematic with its many English speakers, although it has a vast and innovative technology industry.

Prediction: Google can defend search. Google is a little late to the party, with Bard behind ChatGPT at present, but it is working hard to bring all of that online. And it has very powerful defensive barriers in its brand mindshare, Android phone involvement, Chrome web browser market share, deal with Apple for iPhone search, and much more. And if AI search bots can answer queries more quickly, the resulting decline in pages-per-user is more likely to damage downstream websites than the search engine itself. And the challenge of monetizing AI-specific answer formats is in its early stages.

Prediction: Microsoft 365 Copilots boom with consumers. It seems likely that ChatGPT integration into the paid Office products from Microsoft will be the first big B2C revenue win for large businesses. The first B2B win is obviously already happening in the GPU hosting market (i.e. AWS, Azure, GCP) and the underlying chips (i.e. NVidia). Microsoft seems likely to make the earliest "big money" from GenAI consumers. This seems likely to beat OpenAI's monetization attempts, though presumably OpenAI will be getting a cut of Microsoft's Copilot revenues.

Prediction: Meta's Facebook ecosystem benefits. Facebook doesn't need to monetize AI and will do fine using AI in its B2C ad-supported business model, using the capabilities within its own ecosystem to retain its place in the duopoly of online advertising. Its launch of Llama v2 models for free, with commercial usage allowed, will mean advancements to its own AI technology will benefit from the open source movement.

Prediction: Amazon benefits from AWS. Amazon will surely share in the AI boom via AWS GPU hosting. Benefits in its B2C business seem less clear. AI seems less likely to rocket its e-commerce business for the simple reason that it's already been using AI to get people to buy in its own tech stack. GenAI doesn't seem to be a breakthrough for areas such as product suggestion engines. Writers are already using GenAI to bombard Amazon with new e-books, but that over-supply has already been going on for years, with human writers pumping out formulaic novels.

Prediction: Apple continues AI projects successfully. Apple has been coy about its AI, even in earnings calls, but it's been doing AI behind the scenes in many of its products. Since it's already had AI since before GenAI took off, why make a big deal of it? For example, Apple's had facial recognition technology in its products for years. As such, Apple will focus on the consumer experience, and likely deliver some more amazing products with AI underneath (and likely not really talk about it as much as analysts would prefer).

Prediction: Integrators benefit from B2B. Integration and consulting companies (e.g. IBM, Kyndryl, HPE) will benefit from new B2B GenAI projects, just as they have been benefiting from B2B cloud migration projects. It seems unlikely that IBM's Watson booms from GenAI, but stranger things have happened. IBM has a longer history with AI technology, and will benefit from the B2B AI product needs of their business clients, including integration and consulting projects, regardless of AI's evolution.

Prediction: Database vendors benefit from AI data. AI has made data very valuable, especially proprietary data. This gives an opportunity to companies such as long-standing database vendors such as Oracle and IBM, and new entrants such as Snowflake and Databricks, to leverage data capabilities into lucrative business AI projects.

Prediction: Multi-AI is the next big shift. If you think one AI engine is good, think about two. Or a dozen. There are a lot of amazing things that multiple AI engines could do in tandem, as a voting "committee", or in a swarm. This "multi-AI" capability is still held back by cost somewhat at present, but it's surely coming. There are already training regimes that use multiple AI engines, and "big-small" architectures that use two engines. Even if the overall cost of training (and inference) of AI engines drops significantly, the opportunity to leverage higher-level functionality using multi-AI will rise up to fill the demand. As such, AI hosting and GPU usage demand seems sure to rise for years or decades, no matter how many boffins try to optimize it.

Disclaimer: YMMV

I hope you enjoyed the above discussion, but it's just a set of opinions of what might happen, and certainly contains numerous errors and omissions. Don't rely on this as financial advice or for your business strategy.