Financial Optimizations

Financial Optimizations

An AI project is expensive in terms of the hardware, the software, and the people you need. There are some considerations that can reduce the cost somewhat.

Use existing assets. What internal data assets do you possess? Can you re-purpose any of your company's existing hardware assets? And can you “re-purpose” any of your staff, too?

Buy vs rent. If it's floating, flying, or foundational modeling: rent, don't buy! Similarly, do you need to buy your own servers and GPUs? The decision may be different for the different phases of a project:

• Development and testing
• Training the model
• Inference (live execution)

For example, you might want to buy for training phases and rent for the inference phase. This depends on how much training you need, the size of your model, and whether you plan to avoid fine-tuning for proprietary data by using RAG instead. The cost of inference depends on the user counts, which is significantly different if it's an internal employee project versus a live public user application.

Idle VMs and GPUs. Watch out for virtual machines and rented GPUs being idle early in the project. You're paying money for nothing in such cases. This can occur in the development phases and in the early live deployment when user levels are low.

Scrimp on developer models. During the development and testing phases, there's no need for gold-plated AI models. The cost of development and testing of your AI application can be reduced by using low-end models for simple testing. Many of the components needed are not dependent on whether the AI engine returns stellar results. Initial development, prototyping, and ongoing regression testing of these parts of the system can proceed with small models.

There is also vendor support for testing on lower-end models. There are various other AI platforms that offer interfaces that mimic OpenAI's API, but at a lower cost, so you can test on these platforms, and then do final testing on the live commercial platform.

• Next: • Up: Table of Contents • Buy: Generative AI in C++: Coding Transformers and LLMs

The new AI programming book by Aussie AI co-founders: AI coding in C++ Transformer engine speedups LLM models Phone and desktop AI Code examples Research citations Get your copy from Amazon: Generative AI in C++