Open Source C++ Transformer Engines

Open Source C++ Transformer Engines

You don't need to create a whole C++ Transformer engine from scratch. There are multiple fully coded AI engines, with C++ source code to download, that are available under permissive open source licenses.

Some of the useful C++ engines that are great to experiment with on your desktop PC, or even a laptop, include these options:

• GGML: https://github.com/ggerganov/ggml (MIT License)
• Llama.cpp: https://github.com/ggerganov/llama.cpp (MIT License)
• StarCoder from BigCode: https://github.com/bigcode-project/starcoder (Apache 2.0 License)
• Intel Extension for Transformers: https://github.com/intel/intel-extension-for-transformers (Apache 2.0 License)
• gemma.cpp: https://github.com/google/gemma.cpp (Apache 2.0 License and BSD-3-clause license)

You should check the current license status of these projects yourself, since many of these are not only free, but allow commercial usage. However, even permissive licenses have restrictions and obligations.

• Next: • Up: Table of Contents • Buy: Generative AI in C++: Coding Transformers and LLMs

The new AI programming book by Aussie AI co-founders: AI coding in C++ Transformer engine speedups LLM models Phone and desktop AI Code examples Research citations Get your copy from Amazon: Generative AI in C++