AI Quality Engineering

No. This is written for QA and SDET engineers, not ML engineers. You are testing the behaviour of a system you did not train, which is the job you already do, applied to outputs that change between runs. There is no model training, no maths, and no research-paper reading anywhere in the course.

That is the central question of the course and it takes four modules to answer properly. You build a golden dataset of cases with known-good characteristics, write assertions on properties rather than exact strings (must-mention checks, score ranges, structural rules), use a second model as a judge where a property is genuinely fuzzy, and then test the judge itself so you are not trusting an unvalidated grader.

Yes. It is at github.com/aston-cook/asserthired-evals and you can clone and run it. It is a reference implementation over synthetic cases and a sanitized prompt snapshot, not a suite running against live production traffic. Module 9 walks through one of its runs, which scored 50.8 percent against a CI floor of 80 percent, and what was changed to bring it to 98.5 percent. The write-up is in FINDINGS.md in that repository, so every number in the module can be checked against its source.

The eval suite and the code examples are JavaScript and TypeScript, run with Node. The concepts (golden datasets, judge validation, red teaming, CI gating) are language-neutral, so the patterns transfer if your stack is Python or Java.

The final module is about exactly that: what to put in a portfolio, how to talk about eval work in an interview, and what the role actually looks like today. The strongest thing you leave with is not the certificate, it is a working eval suite you can demonstrate and a failed run you can explain honestly, which is a much rarer interview answer than a course completion.

About four hours of video across 31 lessons, plus the time you spend running the suite yourself. There is no deadline and no expiry: you get lifetime access and every future update at no extra cost.