Aussie AI

Inputs, Outputs and Dimensions

  • Book Excerpt from "Generative AI in C++"
  • by David Spuler, Ph.D.

Inputs, Outputs and Dimensions

The input to a normalization component is a vector of logits (probability-like values). The output is the same vector, but with all values changed into a normalized scale, one way or another (e.g. in the range [0..1], inclusive). Hence, the operation is a many-to-many vector operation, which can require multiple scans of every element of the vector. The dimension of the vector, both input and output, is the model's dimension size.

Typically, a model's hidden dimension will be chosen as a multiple of a power-of-two that is large enough to allow easy vectorization. Hence, the size of the normalization input and output vectors should be amenable to vectorization without any extra cases.

 

• Next:

• Up: Table of Contents

• Buy: Generative AI in C++: Coding Transformers and LLMs

Generative AI in C++ The new AI programming book by Aussie AI co-founders:
  • AI coding in C++
  • Transformer engine speedups
  • LLM models
  • Phone and desktop AI
  • Code examples
  • Research citations

Get your copy from Amazon: Generative AI in C++