What are Tensorsand?

What are Tensors?

Tensors are terrifying at first! I avoided learning about them for ages. All those nested loops are scary. But eventually it dawned on me that they're just three-dimensional arrays, and the computations are nothing harder than multiplication and addition.

An important point is that “tensors” in Computer Science are much different to the mathematical forms used in Physics. AI tensors are used in “linear algebra” and are much more basic than the 4-D space-time tensors in Einstein's theory of general relativity. Which may explain why all those brainy physicists are so smug, despite being unable to predict if it'll rain tomorrow.

Tensors in AI are simply multi-dimensional arrays, and are usually 3-dimensional in AI engines. Each slice of a 3-D tensor is a two-dimensional matrix. And like vectors and matrices, tensors have these basic properties:

(a) Each element stores a single number (i.e. no strings or objects).

(b) All elements have the same data type (e.g. int or float).

(c) Elements may be positive, negative or zero.

(d) There are no missing elements. The concept of “missing” can only be represented by zero in a normal tensor.

There are exceptions, of course. There are “sparse tensors” that can represent elements as missing. Also, you can technically store strings or objects in a C++ three-dimensional array, but then it's more of a misuse of a tensor. Numbers are where it's at.

• Next: • Up: Table of Contents • Buy: Generative AI in C++: Coding Transformers and LLMs

The new AI programming book by Aussie AI co-founders: AI coding in C++ Transformer engine speedups LLM models Phone and desktop AI Code examples Research citations Get your copy from Amazon: Generative AI in C++