Becoming an AI Engineer

"AI engineer" covers a family of jobs, from people who invent new training methods to people who ship a retrieval assistant into a customer-support workflow. Knowing which role you are aiming for decides what to study, what to build and how you will be interviewed. This lesson maps the roles and their skills, turns this program into a 12-month plan, and covers portfolios, interviews and how to keep up once you are in.

The intuition: a hospital, not a single job

A hospital runs on surgeons, anaesthetists, nurses, pharmacists, radiographers and administrators. They share a base of medical knowledge, but nobody expects a pharmacist to operate. AI teams work the same way: everyone needs a shared foundation (Python, data, the ML workflow, evaluation), and then each role goes deep in a different direction. The common beginner mistake is to try to learn everything at equal depth. A better strategy is "T-shaped": broad enough to talk to every role, deep in one.

The roles

Job titles are inconsistent between companies, so read job descriptions for the actual duties. The table describes the typical centre of each role.

The boundary between ML engineer and AI engineer is the newest and blurriest. Roughly: ML engineers more often train models on the company's own data; AI engineers more often compose pretrained models, retrieval and tools, and spend much of their effort on evaluation.

Skills map per role

Rate each skill as F (foundation: working knowledge), D (depth: you are the person others ask) or blank (helpful but optional).

Two skills appear in every column: evaluation and communication. Teams hire people who can tell whether a model is actually good and explain that clearly.

A 12-month learning plan using this program

The plan assumes about 10 to 12 hours a week, which adds up to roughly 500 to 600 hours over a year. Adjust the pace, but keep the order: each block depends on the one before. Every block ends with something you build and write up.

Data scientists should spend extra time on statistics and experiment design in months 3–4; research-engineer candidates should add distributed training (Clustering and Multi-GPU) and read papers weekly from month 5.

Building a portfolio

A portfolio exists to answer one question for a hiring manager: can this person take an ambiguous problem to a working, measured result? Three to five strong projects beat fifteen tutorials.

Project ideas that signal real skill

• A domain-specific RAG assistant over a public corpus (court opinions, standards documents, a software project's documentation), with a labelled evaluation set of 100 or more questions, retrieval metrics, and an analysis of failure cases.
• A tabular model with a decision attached: churn or fraud on public data, where you set the threshold using a cost matrix and report the expected cost, not just AUROC.
• A fine-tuned small model that beats a prompted larger model on one narrow task, with cost and latency compared.
• A production-style service: a model behind an API, containerised, with tests, logging, a monitoring dashboard and a load test.
• A reproduction: re-implement a result from a paper at small scale and report what matched and what didn't.

Writing up results

Each project needs a README or short report with: the problem and why it matters; data and its limitations; a baseline; the method; results with variance (mean and spread across folds or seeds); an error analysis with examples; and what you would do next. Honest negative results ("the fine-tuned model did not beat retrieval plus prompting, and here's why") are a strong signal of maturity.

The habit that underlies all of this is reproducible experiments. A minimal template:

import json, platform, time
import numpy as np, sklearn
from sklearn.datasets import load_breast_cancer
from sklearn.dummy import DummyClassifier
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import StratifiedKFold, cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

SEED = 0
X, y = load_breast_cancer(return_X_y=True)
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=SEED)
candidates = {
    "majority-class baseline": DummyClassifier(strategy="most_frequent"),
    "logistic regression": make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)),
    "gradient boosting": HistGradientBoostingClassifier(random_state=SEED),
}
results = []
for name, model in candidates.items():
    t0 = time.perf_counter()
    s = cross_val_score(model, X, y, cv=cv, scoring="roc_auc")
    results.append({"model": name, "auroc_mean": round(s.mean(), 4),
                    "auroc_sd": round(s.std(ddof=1), 4),
                    "seconds": round(time.perf_counter() - t0, 2)})
    print(f"{name:26s} AUROC {s.mean():.3f} +/- {s.std(ddof=1):.3f}")

record = {"seed": SEED, "cv": "stratified 5-fold", "python": platform.python_version(),
          "sklearn": sklearn.__version__, "results": results}
with open("results.json", "w") as f:
    json.dump(record, f, indent=2)

It fixes the random seed, compares against a trivial baseline, reports spread across folds, records library versions and timing, and saves results as data rather than screenshots. On this dataset logistic regression and gradient boosting both score around 0.99 AUROC with overlapping spreads, so the honest write-up says "no meaningful difference; prefer the simpler model", which is exactly the judgement interviewers look for.

Open-source contributions

Contributing to a widely used library (scikit-learn, PyTorch, Hugging Face libraries, evaluation frameworks, inference servers) shows you can read a large codebase, follow review norms and communicate in writing. Start with documentation fixes, reproducible bug reports and issues labelled for newcomers; read the project's contributing guide first; and keep pull requests small. One merged, substantive pull request is worth more than many trivial ones.

Interviewing

Most AI engineering loops combine: a coding round (data structures and practical Python), an ML fundamentals round (bias–variance, regularisation, metrics, leakage, how transformers and attention work), an ML system design round, a project deep dive on your own work, and a behavioural round. Research roles add paper discussion; product and security roles add case studies.

ML system design: a worked outline

Prompt: "Design an AI assistant that answers customer-support questions from our help-centre articles and past tickets."

1. Clarify requirements (5 minutes). Who uses it: customers directly, or agents as a drafting tool? Languages? Can it take actions (refunds) or only answer? What counts as success: deflection rate, resolution, customer satisfaction? Constraints on latency, cost and privacy? Assume: customer-facing, English, answer-only, escalate to a human when unsure.

2. Estimate scale. Suppose 50,000 conversations a day, 6 turns each: 300,000 model calls a day. Average load is $300{,}000 / 86{,}400 \approx 3.5$ requests per second; with a peak-to-average factor of 3, plan for about 10 per second. Each call sends about 3,000 input tokens (instructions 200, five retrieved chunks of 400 tokens = 2,000, conversation history 800) and generates about 300 output tokens. Daily volume: $300{,}000 \times 3{,}000 = 900$ million input tokens and 90 million output tokens. At illustrative prices of 1 USD per million input tokens and 4 USD per million output tokens, that is $900 + 360 = 1{,}260$ USD per day, roughly 38,000 USD per 30-day month. Cutting retrieval to three chunks (input 2,200 tokens) drops input to 660 million tokens and the bill to about 1,020 USD per day: cost is a design variable, not an afterthought.

3. Architecture. Ingestion (parse articles and resolved tickets, remove personal data, chunk, embed, index with metadata such as product and date) → query path (rewrite the question using conversation context, hybrid keyword-plus-vector retrieval, re-rank, generate with citations to sources) → guardrails (refuse out-of-scope topics, detect prompt injection in retrieved content, confidence-based escalation to a human) → logging of every request, retrieved documents, answer and feedback.

4. Latency budget. Retrieval about 100 ms, re-ranking about 150 ms, time to first token about 500 ms; stream the answer so the user sees text within a second, even if the full reply takes several seconds.

5. Evaluation. Offline: a labelled set of real questions with reference answers and source documents; measure retrieval recall@k, answer correctness, faithfulness to sources and escalation precision. Online: A/B test against the current help centre on resolution rate and satisfaction, with guardrail metrics such as wrong-answer reports.

6. Risks and iteration. Stale articles (re-index on change), hallucinated policies (require citations; refuse when retrieval is weak), privacy (redact before indexing), drift in question mix (monitor topics). Close by naming what you would build first: a narrow pilot on the top 20 question types, with human review of a sample of answers each week.

Interviewers care less about the "right" architecture than about clarifying requirements, quantifying, choosing metrics and naming trade-offs.

Project deep dives

Expect "why this metric?", "what was the baseline?", "what failed?" and "what would you change?". Prepare a two-minute summary and a ten-minute walkthrough of your best project, and know its numbers cold.

Staying current

• Papers: most ML papers appear first as preprints on arXiv; ICLR and many workshops publish reviews openly on OpenReview. Read abstracts widely and a few papers deeply; for each, write three lines: claim, evidence, limitation.
• Conferences: NeurIPS (Conference on Neural Information Processing Systems, usually December) and ICML (International Conference on Machine Learning, usually July) cover machine learning broadly; ICLR (International Conference on Learning Representations, usually spring) focuses on deep learning; ACL (the Annual Meeting of the Association for Computational Linguistics) is the flagship for natural language processing; CVPR (Conference on Computer Vision and Pattern Recognition, usually June) leads in computer vision. Their proceedings are freely available, and recorded talks and tutorials are an efficient way to catch up.
• Engineering sources: release notes and documentation of the libraries you use, model cards, and the engineering blogs of teams that publish post-mortems and evaluation details.
• Practice: re-run a new method on your own data before believing a leaderboard; benchmarks are covered in Benchmarks and Evaluation.

Certifications

Major cloud providers offer machine-learning and AI certifications that test their own platforms' services for training, deploying and monitoring models, at foundational, associate and professional or specialty levels. Other organisations offer vendor-neutral credentials, and universities offer certificate programmes. Treat them for what they are: evidence that you know a platform's tooling, which can help in roles centred on that cloud or in consulting, and a structured study plan. They rarely substitute for a portfolio in technical hiring. Exam names, content and availability change frequently and some are retired, so check the provider's official certification page for the current list before you study.

Common pitfalls

• Tutorial loops: completing courses without building anything original.
• Chasing every new model instead of mastering evaluation, data and software engineering, which change slowly.
• Portfolio projects without baselines or error analysis, which look like demos rather than engineering.
• Ignoring software engineering: untested notebooks are a red flag for ML and AI engineering roles.
• Over-claiming: inflated results in a README are easy for interviewers to probe and destroy trust.
• Applying to every title alike instead of tailoring to the role's real duties.

In practice

• Choose a target role from the table and write down the three skills where you need depth.
• Follow the 12-month plan, and publish one write-up per block.
• Keep an experiment log and reproducible code for every project.
• Practise system design aloud with the six-step outline and back-of-envelope numbers.
• Read one paper a week and attend (or watch) one major conference a year.
• See AI and Your Career for how AI is changing work more broadly.