Aussie AI Blog

Planning Your AI Business Project

September 23, 2024

by David Spuler, Ph.D.

Planning an AI Project

Writing a planning document is an important part of an AI project. It's likely to be so long that your fingertips get sore from tapping away at the keyboard. But at least, if you've got to this point, you're hopefully past the "paralysis by analysis" that can occur in deciding what to build.

Some general points about planning:

A strong plan can make the day.
General project planning methods apply — it's just another type of tech project.
Vendors will over-promise in RFP responses and under-deliver in reality.
Open source components are what they always are, if you know what I mean.

And one final AI-specific point:

Hurry! AI clocks run faster.

Expediting AI Projects

Here are some suggestions for early actions to take that can speed up your overall time-to-market for an AI project:

Approval process
Audit existing ML projects
Consulting advice
Staff skills assessment
Data inventory and cleaning
Legal department review

And one overarching requirement is to invest time and energy into learning all this new stuff. You're on the right track in reading this book, so your next step should be to buy everyone in your division a copy of this book. Or alternatively, you could try to find an online AI training data set that contains a pirated copy.

Approvals

As with most IT projects, your generative AI project will need to go through a formal approval process within your company. It probably needs rather a lot of signatures, going up the chain, kind of like the way that autoregressive decoding slows down AI engines, where each step has to await completion of the prior one.

In the AI world, here's how it goes:

You: I'm vaguely thinking about doing something in generative AI.

CEO: I'll send the private jet to pick you up. The Board can fly coach.

On the other hand, you might discover that ten other Vice Presidents have been pitching the same thing for their divisions. Or there might be a new Chief AI Officer (CAIO) in your org chart now who has overall say on the direction to take. In any case, it's best to figure out the new process for AI project approvals.

Auditing Existing AI Projects

AI isn't new, but it used to be called "Machine Learning" or ML. You've certainly used AI in Amazon product recommendations, Netflix movie suggestions, or iPhone face recognition. Any medium to large organization will already have ML projects happening in the IT division.

Hence, audit them! Gather the information about what capabilities already exist. Early projects won't be using generative AI, but will be based on "predictive AI".

Predictive and generative AI are quite complementary technologies. Predictive AI is great with numbers and patterns, whereas generative AI is great with words and natural language. There may be opportunities to re-package some of your prior ML projects into generative AI projects by using an LLM as a natural language wrapper around the core predictive engine, in two basic ways:

Input — an LLM could accept natural language queries against a data set, but use the predictive AI engine to analyze trends, and/or
Output — an LLM could output the results from a predictive AI analysis in more readable natural language, or with greater personalization to the audience.

Even if there's no clear win from your existing ML projects, at least you can identify some staff with AI competence. And they've probably been playing with the ChatGPT API for a while.

Consulting Advice

A number of the large consultant companies have been crowing about all the extra revenue they've been getting for AI projects, especially generative AI. Although that doesn't sound like great news for you when trying to hire them, at least it shows that many other businesses are seeking help to get these AI projects off the ground. There are two main things you could ask for:

Advice
Coding

It's not clear how much of each the big consultants are doing, but it's certainly some of both.

Seeking advice is not a bad place to start the ball rolling. Some of the questions on which you might need some further advice may include:

Use cases
Budgeting and ROI analysis
Data review
Vendor pricing review

There are also some issues that might require legal review:

Regulatory issues
Privacy issues
License issues

On the other hand, you won't need any help with planning, design, or achitecture, because you have this book, which costs about the same as 3 minutes of a consultant's time.

General Legal Issues

The company has a few lawyers, right? Another early stop in AI project planning is the legal department. It's good advice to get started early with their involvement, because legal signoff is a common bottleneck in launching AI projects.

There are plenty of billable hours up for grabs in analyzing your AI project:

Regulatory compliance
Privacy compliance
Copyright & copyrightability
Patentability
Responsible AI policies

AI regulations are an area of ongoing change, so you will need to review this with someone who went to Harvard or Yale. There's more stuff happening on this in Europe than the USA at the moment, but don't worry, there's plenty of US laws to talk about. You can have a nice, pleasant conversation about "compliance": SOC, SOC2, SOX, CAN SPAM, HIPAA, GPDR, COPA, DMCA, FSOC, FINRA, COPPA, CalOPPA, TWEA, CISA, and so much more.

Finally, it's not just the policies of various governments that you need to worry about. Rather, there are those big tech companies that make more money than many sovereign governments, which have their own policies for access to their platform. If you want to use their infrastructure or publish your application to their users, then you also need to comply with various corporate policies. Some examples:

Apple's responsible AI and privacy policies
OpenAI's responsible AI policies

More finally, you will actually need some documents created by the legal department to use with your AI projects. You need to consider the terms of use and privacy policy issues for online AI projects, or for less public interfaces, what license are you giving your users or business customers? There's all the usual issues, and a few more thanks to AI:

Rights to use data for AI training or disclosure that you won't.
Disclosure of AI-created anything may be needed.
Who owns the output from the AI engine?
Disclaimers about copyrightability of AI-created anything.
Responsible AI usage policy issues.
Disclosure and other requirements of all those MIT and Apache 2 licenses you're using in your project.
Indemnification of your users against future legal issues (Microsoft did this viz some AI output).
Disclaimers about inaccuracies, hallucinations, and other issues with LLM outputs.

The good news in this whole section is that these are your problems, not mine (!), but at least you can foist them onto the legal department. But don't worry about the laywers, they'll be fine, and they can punish you for it.

The whole project will stop dead when someone in legal hears a rumor that you did a great job building your AI project and they want to patent this stuff, and you don't get that 12-month grace period after going live for foreign patents (only US patent law is sensible), and we need that priority date against our foreign competitors, and you can't lodge a provisional patent without the full description, and you have to describe all of the algorithms in excruciating detail because of enablement, and then writing a patent specification takes months of back-and-forth between the programmers who designed it and the patent prosecution associates trying to write in words how an LLM actually works.

Did I mention that legal signoff is a common bottleneck?

Buy Projects

A lot of early Ai projects are of the "buy" type, such as purchasing copilots for writers or programmers. There is no shortage of vendors to choose from, and there's only about 65,000 AI startups that have launched in the last few years.

Planning for the integration of a vendor-purchased AI product involves all of the normal project planning steps:

Requirements analysis
RFP posting and review
Internal testing and comparison
Proof-of-concept and pilot phase
Go live
Review and repeat

Using bought AI technology is a way to get a quicker win than building. It's also a useful stepping stone in terms of organizational readiness for more advanced AI projects.

Build Projects

Building an AI project is not easy. There are several ways to approach it:

Customize an existing AI platform
No-code AI platforms
Open source AI platforms
Build from scratch

Note that when I say "build from scratch" I don't necessarily mean training your own foundation model from scratch, but coding all of the other application infrastructure around it. In fact, both the LLM and the AI engine are probably not something you need to build yourself. You can either rent one by the token, or use fully-coded open source versions.

Generally speaking, there are four main parts to a basic AI project:

Front-end UI
Back-end infrastructure
Model and AI engine
Prompt engineering

The front-end is completely up to you. If you're stuck for suggestions, no doubt ChatGPT would be happy to help. Or there's a few consultants out there willing to help.

The back-end of an AI application considers of all the usual stuff plus the AI part, which consists of the LLM and an engine. The LLM is all data, and the engine is all code (usually written in Python at the top-level and C++ deep in the guts, but you didn't need to know that and I only mentioned it so you can go buy my other book).

You can use commercial LLM services or open source LLMs, depending on your budget. For the engine, you can host it on premises or use cloud services (or maybe on-device). If it's a pre-trained LLM of either type, the main way to add business-specific logic to your application is via prompt engineering (and the user interface), so this is an extra AI-only module that's quite important.

The non-AI backend components are all the usual things you need to serve apps to users. For example, a web-based application needs server computers, network capacity, HTTPD web servers (e.g., Apache or Ngiinx), DNS, domain names, user login management, and lots more waffle.

If you're going to do some more advanced things with data, there are two main ways, each of which adds some additional development tasks and software components:

Fine-tuning
RAG

But there's another whole chapter on architecture that's all about that, so I'll stop here.

Data

Data is such an important aspect of an AI project that there's a whole chapter on it (Chapter ??). To get the full benefit of an LLM that's specialized to your particular type of business, you want and need some proprietary data. This means you can use this data in either a fine-tuning or RAG project, and this offers more specialized results for your users, whether they're external customers or internal staff.

Assuming your project needs data, there are various extra developmental tasks:

Data inventory — finding it.
Data cleaning
Legal review
Data ingesting phase — fine-tuning or RAG chunking.

Not all AI projects need such data. You can use a "dataless" architecture in various ways to get some quite significant benefits via prompt engineering alone.

AI-Free Zone

Despite what you might have heard, software programs used to actually work just fine without AI. I'm here to help you along in your career, and you might find yourself in the horrible situation where a developer has coded an app without an LLM in it.

Here's how to save your job:

You: We've finished the AI version of our word counter app.

CEO: What sort of AI does it use?

You: It's using an iterative decision logic amplifier based on a vocabulary size of 256.

CEO: The analysts will be impressed!

Here are some additional recalibration suggestions:

Database — tabular index-enabled RAG chunk retrieval engine.
Integer variable — 32-bit quantized bit-parallel datastore.
If-then statement — bifurcating model selection cascade pathway.
Floating point — exponent-scaled mantissa-based power-of-two data.

Feel free to suggest your own!