Generative AI evaluation is difficult because many important qualities are not captured by one accuracy number. A response can be fluent but unsupported, relevant but unsafe, correct for one customer but wrong for another, or acceptable in development but unstable after a model or prompt update. Databricks MLflow 3 provides tracing, evaluation datasets, scorers, LLM judges, human feedback, and production monitoring so teams can make quality measurable across the lifecycle of a GenAI application.
The Databricks Generative AI Engineer Associate certification includes evaluation and monitoring because a production AI application needs evidence that it behaves acceptably. The goal is not to produce a decorative scorecard. Evaluation should guide design choices, block unsafe releases, and reveal regressions after deployment.
Start by defining quality dimensions
Different applications require different definitions of success. A support assistant may prioritize groundedness and policy compliance, while a summarization workflow may emphasize completeness and factual consistency. Define the dimensions before choosing metrics so the team does not optimize whichever score happens to be easiest to calculate.
Separate quality dimensions where possible. A single composite score can hide a safety failure behind strong relevance. Release criteria should make critical requirements visible, especially when one failure mode is unacceptable regardless of performance elsewhere.
Instrument applications with traces
Tracing records the sequence of model calls, retrieval steps, tool invocations, and outputs that produced a result. This gives evaluators evidence beyond the final answer. If a response is wrong, traces can reveal whether retrieval missed the right document, a tool returned stale data, or the model ignored useful context.
Trace data should include version metadata for prompts, models, indexes, and important configuration. Reproducing a bad request is much easier when the system state is known rather than reconstructed from memory.
Build representative evaluation datasets
An evaluation dataset should contain normal requests, important edge cases, known production failures, ambiguous questions, safety challenges, and examples from different user segments. It can include expected answers or facts where ground truth is available, but useful testing is still possible for cases without a single canonical response.
Maintain a small high-confidence regression set that every release must pass, plus broader sampled datasets that explore behavior across the problem space. As production reveals new failure modes, add them to future evaluation so the same problem does not quietly return.
Use built-in judges where they fit
MLflow provides built-in LLM judges for dimensions such as relevance, safety, correctness, and retrieval quality. These can accelerate evaluation because teams do not need to design every rubric from scratch. Use them as measured components, not unquestioned truth.
Judge outputs should be calibrated against human review for high-impact use cases. If domain experts consistently disagree with an automated scorer, refine the criteria or use a custom judge that reflects the application’s actual standard.
Add code-based scorers for deterministic rules
Some requirements should not be judged probabilistically. A response may need to contain a required disclaimer, follow a JSON schema, cite at least one retrieved source, avoid a prohibited field, or return a tool result within a defined range. Code-based scorers are appropriate for deterministic checks like these.
Combining deterministic checks with LLM judges produces a more complete test suite. The system can verify structure exactly while using semantic evaluation for qualities such as groundedness or helpfulness.
Evaluate retrieval separately in RAG systems
For RAG applications, measure whether the retrieved context is relevant before scoring the final answer. A model cannot reliably ground an answer in evidence it never received. Retrieval relevance, context coverage, and source freshness should therefore be first-class evaluation targets.
When a final answer fails, inspect whether the issue belongs to ingestion, chunking, embeddings, filtering, ranking, prompt construction, or generation. This component-level view prevents teams from changing the model when the real defect is in the knowledge pipeline.
Use human feedback strategically
Human review is valuable for nuanced quality dimensions, domain correctness, and scorer calibration. It is also expensive. Use experts where their judgment changes the standard: difficult edge cases, regulated decisions, ambiguous answers, and examples used to align automated evaluation.
Do not rely solely on thumbs-up or thumbs-down signals. Capture reason codes or structured feedback when possible so the team can distinguish factual problems, missing context, tone issues, unsafe output, and retrieval failures.
Make evaluation part of release engineering
Run evaluation when prompts, models, retrieval configuration, tools, or application logic change. Compare candidate versions against a stable baseline using the same dataset. A release should improve or preserve required dimensions rather than simply produce an interesting demo.
The discipline resembles software testing and the principles behind version-controlled development: changes are reviewed, measured, and associated with identifiable versions. Evaluation results should be saved so teams can explain why a release was approved.
Continue scoring production behavior
Offline evaluation cannot represent every real user or data condition. Sample production traces and apply the same scorers used during development where appropriate. Monitor quality trends, latency, tool failures, safety events, and cost so degradation is detected before it becomes widespread.
The Databricks Machine Learning Associate and Machine Learning Professional paths are adjacent because experimentation, model lifecycle, and production monitoring share the same need for measurable evidence. Strong GenAI evaluation turns subjective impressions into an engineering loop: trace behavior, score it, review failures, change the system, and prove that the next version is better.
Evaluation data should be versioned alongside the application. If the dataset changes between two experiments, score differences may reflect the changed test set rather than a better model or prompt. Record the dataset version, scorer definitions, model version, and application commit for every important comparison.
Golden sets are intentionally small, stable collections of must-pass examples. They are useful for release gates because failures are easy to inspect and explain. Broader benchmark sets can change more frequently as teams discover new use cases, but golden examples should represent business behaviors that the application is never allowed to regress.
Ground truth can take several forms. Some tasks have one expected answer, while others have required facts, acceptable ranges, or a reference policy. Choose the representation that matches the task. Forcing open-ended work into one exact reference string can penalize valid answers and teach the team to optimize for an artificial test.
LLM judges should have clear rubrics. A vague instruction such as “rate quality” produces scores that are difficult to interpret. Define what a pass means, include examples where appropriate, and separate dimensions such as correctness, completeness, relevance, and style so a reviewer can understand why a result changed.
Judge consistency should be measured. Run repeated samples or compare judge output with expert labels for important tasks. If the judge is unstable, treat small score movements cautiously. Evaluation becomes more useful when the team knows which metrics are precise and which are directional.
Human disagreement is informative. When experts disagree about the right answer or policy, the problem may be an ambiguous requirement rather than model quality. Capture that uncertainty instead of forcing a false consensus into the benchmark.
Production monitoring should use sampling intentionally. Scoring every trace with expensive judges may be unnecessary, while sampling too little can miss rare high-impact failures. Adjust sampling by traffic volume, risk, user segment, or detected anomaly so evaluation resources focus where they produce useful evidence.
Drift in GenAI applications can come from more than the model. Source documents change, retrieval indexes refresh, tools return different data, prompts are edited, and user populations evolve. Monitoring metadata should make these changes visible so teams can correlate a quality shift with the most likely cause.
Latency and cost belong in evaluation alongside semantic quality. A candidate configuration may improve groundedness slightly while doubling response time. Record operational metrics during experiments so release decisions reflect the full user and business experience.
Safety tests should include adversarial and boundary cases, not only ordinary prompts. Evaluate jailbreak attempts, requests for restricted information, sensitive-data handling, and tool misuse where relevant. Passing normal question-answer examples does not demonstrate that an application is safe under pressure.
Evaluation results should drive concrete action. Define which failures block release, which create follow-up work, and which are accepted trade-offs. A dashboard full of scores without decision thresholds becomes reporting rather than engineering.
The most effective MLflow evaluation practice creates continuity from development to production. The same concepts—traces, versioned datasets, scorers, human review, and measurable thresholds—support both pre-release comparison and ongoing monitoring. That continuity turns evaluation into part of the application’s operating system rather than a one-time certification exercise.
Evaluation datasets should be balanced across important user segments. An assistant can score well overall while failing consistently for one language, product line, document type, or customer tier. Segment metrics reveal weaknesses that disappear inside a global average.
Use confidence intervals or repeated evaluation where score variance matters. Small test sets can produce apparently large improvements from only one or two examples. Treat tiny changes cautiously unless the dataset is large enough and the judge behavior is stable enough to support the conclusion.
Error taxonomy improves remediation. Label failures such as retrieval miss, hallucination, incomplete answer, unsafe tool use, formatting error, and latency timeout. Trends by category help the team decide whether to improve the knowledge base, prompt, model, tool, or infrastructure.
Regression analysis should compare more than average scores. Identify examples that improved and examples that worsened. A candidate version with the same overall average may fix high-impact failures while introducing minor style regressions, or the reverse. Example-level comparison supports better release judgment.
Production feedback should be sampled back into offline evaluation carefully. Highly active or dissatisfied users may be overrepresented in explicit ratings. Combine user-submitted examples with random or risk-based samples so the benchmark reflects both common behavior and important failures.
Automated judges can also become dependencies that change over time. Record judge model and configuration where possible. If the evaluation system changes, rerun baselines so score shifts are not mistaken for application improvement or regression.
Evaluation cost can be controlled through tiering. Run cheap deterministic checks on every candidate, broader LLM judging on selected builds, and expensive expert review for major releases or high-risk changes. This preserves rigor without making iteration prohibitively slow.
Latency tests should include realistic concurrency. A model configuration that is fast for one request may slow substantially at peak traffic. Combine semantic evaluation with load testing when release decisions affect user-facing production systems.
For RAG, include source freshness in the benchmark. Ask questions whose answers changed recently and verify the system retrieves the current evidence rather than a stale but semantically similar document. This tests the indexing pipeline as well as the model.
For agents, evaluate tool selection and side effects separately from final wording. An agent can produce a polite answer after calling the wrong tool or changing the wrong record. Trace-based evaluation makes those hidden failures visible before they become real incidents.
Human review programs should rotate examples and reviewers where practical. Familiarity with a fixed benchmark can unconsciously encourage overfitting, while multiple reviewers expose ambiguous rubrics. Use disagreement to improve the specification instead of merely averaging it away.
Evaluation maturity is reached when release conversations change from “this version feels better” to “this version improved the required quality dimensions, preserved safety gates, met latency targets, and did not regress critical examples.” MLflow provides the machinery, but disciplined criteria and representative data make that machinery meaningful.
Release thresholds should reflect business risk. A creative writing assistant may tolerate occasional stylistic variation, while a compliance assistant may require near-perfect citation and refusal behavior for restricted requests. Avoid one universal quality threshold across unrelated applications.
When a scorer fails, preserve the trace and surrounding evidence for triage. Engineers should be able to move from a red metric to the exact request, retrieved context, model output, and tool calls that produced it. Evaluation without diagnosability slows improvement.
Benchmark maintenance is an ongoing task. Remove obsolete examples, add newly discovered failure modes, and keep a record of major benchmark revisions. Otherwise a test suite can drift away from current product behavior while continuing to produce reassuring numbers.
MLflow evaluation is most valuable when it changes decisions. It should tell the team whether to release, what to fix, and whether production behavior remains within the accepted envelope. Scores are useful only when they connect directly to those operational choices.
Evaluation ownership should be explicit. Product owners define acceptable behavior, domain experts validate difficult examples, engineers maintain instrumentation and scorers, and security teams define critical safety gates. Shared ownership prevents the benchmark from becoming disconnected from real business risk.
When metrics disagree, return to examples. A strong aggregate score cannot explain whether one specific failure is acceptable. Trace-level review keeps evaluation grounded in actual application behavior and helps teams improve the system rather than optimize a number in isolation.
Keep the benchmark understandable to humans. Every critical scorer should have a short explanation of what it measures, what data it needs, and what threshold matters. This makes evaluation maintainable when team members change and prevents release decisions from depending on opaque metrics that nobody can confidently interpret.
That discipline makes evaluation reproducible enough to support release decisions, incident analysis, and continuous improvement rather than one-time model demos.
When that process is repeatable, evaluation becomes a durable quality system rather than an occasional test performed only when a model is first launched.