AI Basics for Business Owners — Lesson 6
Fine-Tuning, Model Choice, and AI Adoption
Learning Objectives
- 1Know when fine-tuning is worth the investment versus using prompts or RAG.
- 2Create an AI adoption framework for business decisions.
- 3Evaluate AI vendors and implementation partners.
When to fine-tune
Fine-tuning adjusts a pre-trained model using your own examples to improve performance on a specific task. It can make a model better at your company writing style, your industry terminology, your classification categories, or your specific output format.
Fine-tuning is appropriate when prompt engineering and RAG are not sufficient — when you have many examples of the desired input-output behavior, the task is repeated frequently enough to justify the investment, and the improvement in quality or consistency materially affects the business outcome.
For most business use cases, start with prompts, then try RAG, and consider fine-tuning only if neither achieves acceptable quality. Fine-tuning requires technical expertise, quality training data, ongoing maintenance as the model and business needs evolve, and significantly more cost than prompting.
Choosing models and vendors
Model choice involves tradeoffs between capability, cost, speed, privacy, and vendor dependency. The most capable models are more expensive and often require sending data to third-party servers. Less capable models may be sufficient for the task and can sometimes run locally, keeping data private.
Key evaluation criteria: Does the model perform well on our specific task? What data do we need to share with the vendor? What are the privacy and security guarantees? What is the cost at our expected volume? How does the vendor handle data retention and training on customer inputs? What happens if the model is deprecated or pricing changes?
Avoid locking into a single AI vendor when possible. Use abstraction layers that allow model switching. The AI landscape is changing rapidly — the best model today may not be the best model in six months. Flexibility to switch models is valuable.
AI adoption as a business process
AI adoption is not a technology project — it is a change management process. The most common failure mode is not technical problems but organizational ones: unclear goals, lack of training, resistance to workflow changes, or deploying AI without adjusting processes around it.
A practical adoption framework: Start with a specific, measurable problem. Run a small pilot with willing team members. Measure the impact against the baseline. Address concerns and adjust the approach based on pilot learning. Expand gradually with training and documentation.
Set expectations honestly. AI will change how people work, and that creates anxiety. Be clear about what AI will handle, what humans will still do, and how roles may evolve. Transparency and training reduce resistance more effectively than mandates.
Case Study
The premature fine-tuning project
Situation
A legal services company spent $40,000 fine-tuning a model to summarize legal documents. After deployment, the fine-tuned model performed only marginally better than a well-crafted prompt with examples (few-shot prompting) that cost nothing beyond API usage. The improvement was not sufficient to justify the investment or the ongoing maintenance.
Analysis
The team jumped to fine-tuning without first optimizing their prompts. Few-shot prompting — including 3-5 examples of desired input-output pairs in the prompt — achieved 90% of the improvement for zero additional cost. Fine-tuning should have been explored only after prompting reached its limits.
Takeaway
Always optimize prompts before investing in fine-tuning. The progression is: basic prompt → detailed prompt with examples → RAG → fine-tuning. Each step is more expensive and complex.
Reflection Questions
- 1. For an AI use case you are considering, which approach makes the most sense: prompting, RAG, or fine-tuning?
- 2. If your organization adopted an AI tool tomorrow, what training would your team need?
Key Takeaways
- ✓Fine-tune only after prompting and RAG prove insufficient — it is the last option, not the first.
- ✓Evaluate AI vendors on privacy, cost at scale, data handling, and flexibility to switch.
- ✓AI adoption is change management — start small, measure impact, expand gradually.
- ✓Avoid vendor lock-in by using abstraction layers that allow model switching.