Aussie AI

Model Selection Algorithms

  • Book Excerpt from "Generative AI in C++"
  • by David Spuler, Ph.D.

Model Selection Algorithms

Model selection algorithms are dynamic inference optimizations where a choice is made between two or more models for execution. The hottest area of such research is Mixture-of-Experts, because of the GPT-4's rumored architecture. Another example is “big-little” architectures, where a heuristic attempts to send “easy” queries to a faster “little” model. Various other ensemble architectures are possible with multiple models.

Another practical example of a different type of model selection is the deployment architecture, which may be deciding which server to send the request to, where each server may have different models or multiple instances of the same model. Other areas of research with similar aims include cascades and collaborative inference.

 

• Next:

• Up: Table of Contents

• Buy: Generative AI in C++: Coding Transformers and LLMs

Generative AI in C++ The new AI programming book by Aussie AI co-founders:
  • AI coding in C++
  • Transformer engine speedups
  • LLM models
  • Phone and desktop AI
  • Code examples
  • Research citations

Get your copy from Amazon: Generative AI in C++