Aussie AI

Commercialized SOTA Research

  • Book Excerpt from "Generative AI in C++"
  • by David Spuler, Ph.D.

Commercialized SOTA Research

The whole of the AI industry is based on commercialization of State-of-the-Art (SOTA) research, from the basic Transformer architecture to the often-used optimizations of quantization and pruning. The areas where recent research from the last year or two is starting to appear in industry models and open source frameworks, with kudos paid to the innovative researchers, includes:

  • Flash Attention (linearized attention algorithm)
  • Flash Decoding
  • RoPE (positional encoding for long contexts)
  • Long context window research

Just my opinion, but the areas that seem ripe for greater inclusion in commercial and open source industry AI work include:

  • Early exit (dynamic layer pruning)
  • Integer-only arithmetic quantization models (end-to-end integers)

 

• Next:

• Up: Table of Contents

• Buy: Generative AI in C++: Coding Transformers and LLMs

Generative AI in C++ The new AI programming book by Aussie AI co-founders:
  • AI coding in C++
  • Transformer engine speedups
  • LLM models
  • Phone and desktop AI
  • Code examples
  • Research citations

Get your copy from Amazon: Generative AI in C++