As engineering managers, CTOs and tech leads, we see the hype every day.
People ask me what their teams should learn to “get into AI”, how to build proper AI products, or which course will turn a developer into an AI engineer in 30 days.
Honestly, I am tired of seeing people sold a lie.
The internet is full of content promising that anyone can “master Claude”, build an agent overnight, or become an AI engineer by learning a handful of prompts and connecting an API key to a web page.
That is not how serious engineering works.
A clever prompt can create an impressive demo. It can even create a useful prototype. But a prompt, an API key and a chat interface do not automatically create a durable product, a defensible business, or an AI engineering capability.
Equally, we should not swing too far in the other direction and tell every software engineer that they need a PhD in mathematics before they can contribute.
Both views are wrong.
The real skill is learning how to build reliable systems around probabilistic models. Systems with useful data, clear user value, proper evaluation, secure boundaries, human accountability and operational discipline.
AI is not magic. But it is not simply autocomplete either.
It is software engineering, data engineering, product judgement, statistics, security, operations and organisational change, all meeting in one messy place.
If you want to build a serious AI capability in your engineering organisation, I believe there are six foundations that matter.
1. Start with the problem, not the model
The first question should not be:
“Which model should we use?”
It should be:
“What user problem, decision or workflow would genuinely improve here?”
Too many teams start with a model and go looking for a use case. They build a chatbot because chatbots are easy to demo. They build an agent because every vendor is talking about agents. They add AI to a product because investors, competitors or senior leadership expect it.
That is how you end up with expensive features nobody trusts or uses.
A serious AI product starts with a clear problem:
What job is the user trying to do?
Where are they losing time, making errors or getting stuck?
What does good look like?
What happens when the system is wrong?
Should this be automated, assisted or kept entirely human-led?
Could a better process, search experience, workflow or rules engine solve this more reliably?
AI is valuable when it improves a real outcome, not when it generates an impressive screenshot.
For example, an internal support assistant should not be judged by whether its answers sound intelligent. It should be judged by whether it helps people resolve issues more quickly, points them to authoritative sources, reduces repetitive work and safely escalates when it is uncertain.
If you cannot define the user outcome and how you will measure it, you are not ready to choose a model.
2. Learn the fundamentals at the right depth
There is a persistent argument about whether people need maths to work in AI.
The honest answer is: it depends on what kind of work you want to do.
If you are researching models, training models, fine-tuning them, designing retrieval systems, improving ranking, managing inference infrastructure or working close to the machine-learning layer, then yes, mathematics matters a great deal.
You need to understand concepts such as:
Linear algebra: vectors, matrices, embeddings and similarity.
Probability and statistics: uncertainty, distributions, sampling, confidence and evaluation.
Optimisation: how models learn, how loss functions work and why training can fail.
Neural networks and deep learning: what a model is doing when it processes data.
You do not need to become a pure mathematician. But you do need enough understanding to know what is happening beneath the abstraction.
However, not every person building an AI-enabled product needs to derive matrix multiplication by hand.
A strong application engineer can build valuable AI features by understanding the practical behaviour of models, their limitations, their failure modes and how to engineer reliable systems around them.
The important thing is intellectual honesty.
Do not pretend an LLM is magic. Do not treat it as a human brain. And do not assume it knows what is true simply because it says something confidently.
At a basic level, language models predict likely next tokens based on patterns in their training data and the context they are given. That apparently simple mechanism can produce remarkable results across language, code, reasoning-like tasks and multimodal inputs.
But it does not make the output inherently factual, safe or appropriate for your organisation.
That is your engineering job.
3. Data is usually the real product
Most organisations do not have an AI problem.
They have a data problem.
Their information is duplicated, stale, scattered across tools, inconsistently named, inaccessible to the people who need it, or full of conflicting advice. Then they expect an LLM to somehow make it clean, current and trustworthy.
It will not.
If your AI product depends on company knowledge, customer information, policies, contracts, technical documentation or operational data, then data quality becomes a core engineering concern.
Your team needs to understand:
Where the source data comes from.
Who owns it and who is accountable for keeping it current.
Whether it is accurate, complete and authoritative.
Which version is the source of truth when documents conflict.
Who is allowed to access it.
Whether the data contains personal, confidential, regulated or commercially sensitive information.
How data is retained, deleted, audited and protected.
This is where retrieval-augmented generation, usually called RAG, can be useful.
RAG allows a system to retrieve relevant information at runtime and provide that information to the model as context. Instead of relying entirely on the model’s training data, the system can search internal documentation, policies, product catalogues, support records or knowledge bases before generating an answer.
That can improve relevance, freshness and grounding.
But RAG does not “fix hallucinations”.
It can still retrieve the wrong documents. It can retrieve incomplete or stale information. It can expose content the user should not see if permissions are badly designed. The model can misunderstand the evidence, ignore it or confidently invent an answer around it.
Good RAG is not just “put documents in a vector database”.
It requires source quality, document ownership, chunking, metadata, access controls, retrieval evaluation, relevance ranking, citations, clear uncertainty handling and continuous monitoring.
And do not throw away traditional search.
Keyword search remains excellent for exact error codes, policy clauses, product identifiers, customer names, technical acronyms and rare terms. In many real systems, the right approach is hybrid retrieval: combining keyword search, semantic search, metadata filters and reranking.
Choose based on evidence and evaluation, not ideology.
4. Build Systems, Not Prompt Demos
Prompting matters.
Good system instructions, examples, context design, tool descriptions and output constraints can make a substantial difference to quality. Prompt engineering is a real part of modern AI application work.
But prompting alone is not a durable technical strategy.
A thin interface over a general-purpose model is easy to copy. It is also exposed to vendor pricing changes, rate limits, model-version changes and the simple fact that the next model may make your basic feature available to everyone.
That does not mean using an API is a bad business model.
It means your value cannot be only the API call.
Durable AI products create value through things that are harder to replicate:
Deep integration into a real user workflow.
Proprietary or carefully governed domain data.
A strong user experience.
Reliable automation and orchestration.
Trust, safety and auditability.
Better operational processes.
Distribution and customer relationships.
A clear understanding of the domain problem.
This is also where conventional software engineering still matters enormously.
You need proper APIs, authentication, authorisation, databases, queues, retries, rate limiting, caching, error handling, observability, resilient fallbacks and sensible user interfaces.
You need to design for uncertainty.
A conventional system may fail because a server is down or a query times out. An AI system can fail while appearing completely healthy. It can return fluent, plausible, well-structured nonsense.
That is a different class of failure, and it needs a different engineering mindset.
5. Evaluation, Security and Governance Are Not Optional
This is the part too many AI discussions skip.
You cannot look at five impressive examples and conclude that a system works.
A proper AI product needs evaluation before it reaches users, and it needs monitoring once it is in production.
Your team should build representative test sets from real user tasks, known edge cases and previous failures. Then measure the things that matter for the use case:
Is the answer accurate?
Is it grounded in approved sources?
Does it cite or link to the right evidence?
Does it follow the required policy?
Does it select the right tool or workflow?
Does it know when to abstain or escalate?
Is it fast enough?
Is it affordable enough to operate?
Does it behave safely with hostile or misleading input?
Every change can affect performance.
A new prompt, model version, retrieval method, chunking strategy, system instruction, tool description or policy can improve one set of examples while making another worse.
That is why prompts, configurations and model choices should be versioned and tested like production code.
Structured output is also useful. If a model needs to pass information into an existing system, do not ask for loosely formatted prose and hope it parses correctly. Use schemas and structured fields where possible.
But valid JSON is not the same as a correct decision.
A system can return perfectly valid structured data that is factually wrong, operationally unsafe or unauthorised. Your application must still validate business rules and enforce permissions independently of the model.
Security matters just as much.
Teams need to think about:
Prompt injection, including malicious content hidden inside retrieved documents.
Data leakage through prompts, logs, tools or external model providers.
Excessive permissions for agents and tools.
Broken access controls in internal knowledge systems.
Unsafe automation of production, finance, customer or security actions.
Audit logging and incident response.
Human approval for consequential actions.
Clear kill switches and rollback plans.
The moment an AI system can take action rather than merely suggest one, the risk profile changes significantly.
Treat it accordingly.
6. MLOps, LLMOps and Reliable Operations
I used to think that teams had to master massive GPU clusters and highly specialised infrastructure before they could do anything meaningful with AI.
That is not true for every organisation.
Many teams can build useful AI applications with managed model APIs and existing cloud platforms. They do not need to train a foundation model or run their own inference fleet.
But they still need operational discipline.
For teams using hosted models, the important concerns often include:
Model and prompt version management.
Evaluation pipelines.
Request tracing and observability.
Token consumption and cost control.
Latency and timeout management.
Rate limits and provider outages.
Retrieval quality and document freshness.
Safety monitoring.
Tool-call reliability.
Vendor lock-in and model portability.
Incident response when an AI feature starts failing or behaving dangerously.
This is often described as LLMOps.
For teams training, fine-tuning or self-hosting models, the scope becomes broader. They may need traditional MLOps capabilities too: data pipelines, experiment tracking, model registries, training infrastructure, GPU scheduling, inference optimisation, model deployment, drift monitoring and capacity planning.
Neither replaces cloud engineering, SRE, security or platform engineering.
MLOps and LLMOps build on those disciplines.
It does not matter how good your model is if it is too slow, too expensive, unavailable during peak demand, impossible to debug, leaking sensitive data or making untraceable decisions.
Reliable AI is still reliable engineering.
There is no single AI career
One final point for people trying to build a career in this space.
“AI engineer” is too broad to be useful on its own.
There are several career paths, and they require different strengths:
AI application engineers build user-facing AI features, workflows, integrations and tools.
Machine-learning engineers build, train, evaluate and deploy ML systems.
Data engineers create the reliable, governed data foundations that AI systems depend on.
MLOps and platform engineers make AI development, deployment, security and operations repeatable at scale.
Research engineers and applied scientists work more deeply on modelling, optimisation and new capabilities.
Product leaders and engineering leaders choose use cases, create operating models, manage risk and help organisations adopt AI responsibly.
You do not need to be great at all of these.
But you do need to understand how they fit together.
The best organisations will not be staffed entirely by prompt engineers. Nor will they be staffed entirely by researchers.
They will have people who understand users, products, data, software, infrastructure, security, governance and the limitations of the models themselves.
You do not need to be an AI engineer to use AI well
Not everyone needs to become an AI engineer.
Most people will use AI in the same way they use spreadsheets, search engines, collaboration tools and office software: to make everyday work faster, clearer and less repetitive.
You do not need to understand neural networks to ask AI to improve an email, structure a plan, explain a difficult topic, summarise notes or create a first draft.
But you do need to understand the limits.
AI is good at generating, restructuring, summarising and suggesting. It is not automatically good at knowing what is true, what is current, what is confidential or what is appropriate in your organisation.
The basic skill is not learning secret prompts. It is learning how to give a clear brief.
Tell it what you are trying to achieve. Give it the relevant context. Specify the audience, tone, length and format. Ask it to show assumptions, identify uncertainty and suggest what needs human checking.
Then use your own judgement.
Do not paste sensitive data into tools your organisation has not approved. Do not accept important claims without checking them. Do not let an agent send, spend, delete, publish or change something consequential without appropriate controls.
Start with small, reversible tasks.
Use AI to draft the email, not automatically send it. Ask it to organise the meeting notes, not make personnel decisions. Let it suggest an incident checklist, not make production changes without review.
The more access and autonomy an AI system has, the more carefully it must be designed, governed and supervised.
That is AI literacy.
It is not about turning everyone into a machine-learning specialist. It is about helping people use powerful tools with enough understanding to get value from them without giving away their judgement, their data or their accountability.
A practical prompt formula
For everyday users, I would give a simple framework rather than talking about “prompt engineering” as though it were a mysterious technical skill:
Context + Task + Constraints + Output + Check
For example:
I am preparing an update for a non-technical executive team about an engineering incident.
Turn the notes below into a calm, factual update. Keep it under 250 words. Avoid jargon. Include: what happened, customer impact, what we have done, what remains uncertain and the next update time. Do not invent details. Flag anything that needs confirmation.
Notes: [paste approved, non-sensitive notes]
That is enough to make most people significantly better users of AI.
The goal is not to write magical prompts. The goal is to communicate clearly, retain ownership of the work and know when an answer needs checking.
In the end.
The real capability is learning to build dependable systems around uncertain models.
Start with a real problem. Understand the data. Learn the fundamentals at the right depth. Build proper software around the model. Evaluate continuously. Secure everything. Keep humans accountable for meaningful decisions. Operate the whole thing like a production system.
The hot tool will change.
The model provider will change.
The framework will change.
But the organisations that understand how to turn AI capability into trustworthy, measurable user value will keep winning.
About the Author
Diamantino Almeida is a tech leader, coach, and writer reshaping how we think about leadership in a burnout-driven world. With over 20 years at the intersection of engineering, DevOps, and team culture, he helps humans lead consciously from the inside out. When he’s not challenging outdated norms, he’s plotting how to make work more human, one verb at a time.
An excellent article. Thank you.