Databricks Updated Databricks-Generative-AI-Engineer-Associate Exam Questions and Answers by nuala

Option B preserves the relationships between each user query, generated response, and expert assessment. That context is necessary to turn production feedback into reusable evaluation cases. The engineer can curate representative examples from the Delta table, map them into MLflow’s evaluation schema, and retain the dataset for consistent comparisons between agent versions. Where correctness scoring is required, experts should provide corrected answers or expected facts; a star rating alone is not a ground-truth answer. Keeping only written comments loses essential request context. Selecting only five-star interactions biases the dataset and excludes failure cases that future versions should improve. Deploying highly rated responses is not a substitute for evaluating an updated agent against a stable, representative benchmark. Databricks documentation

The AI Gateway usage table is designed to capture operational consumption information for SQL-based reporting. Its records include identifiers for the workspace, endpoint, and request, together with token-usage information and timestamps. The engineer can aggregate consumption by day and endpoint, then join those results to the company’s cost-center mapping. Request counting should respect the table’s request and invocation identifiers because a single logical request can involve multiple inference calls. Served-entity metadata describes deployed entities rather than providing the primary consumption records. Inference tables emphasize request and response payloads, while MLflow runs organize experiments and evaluations. Token consumption supports cost allocation, but calculating actual monetary charges may additionally require applicable pricing or billing records. Databricks documentation

When working within a Databricks notebook, several high-level SDKs are " Databricks-aware. " The MLflow Deployments SDK (C) and the Databricks Python SDK (D) are designed to automatically look for the DATABRICKS_HOST and DATABRICKS_TOKEN environment variables provided by the notebook context. The OpenAI client (A), when configured for Databricks via Mosaic AI Gateway, also typically handles authentication via workspace integration in recent versions. However, the REST API via the requests library (B) is a generic Python HTTP client. It has no intrinsic knowledge of the Databricks environment. To use it, an engineer must manually extract the token (e.g., via dbutils.notebook.entry_point...) and explicitly pass it in the Authorization: Bearer < token > header of the request. Without this manual step, the requests library will fail with a 401 Unauthorized error.

When deploying an agent via the Mosaic AI Agent Framework (which leverages Databricks Model Serving), operational metrics are captured automatically by default. These include system-level telemetry such as the number of requests per second (volume), the time taken for the model to respond (latency), and the rate of 4xx/5xx HTTP errors. These are essential for monitoring Service Level Agreements (SLAs). However, Quality metrics (B), such as correctness, groundedness, or adherence to custom guidelines, cannot be determined " automatically " by the serving infrastructure because they require either human feedback or an LLM-as-a-judge evaluation (using Databricks Agent Evaluation). While Databricks makes it easy to generate quality metrics using the mlflow.evaluate API or the inference table, they are not " default operational metrics " that appear without additional evaluation configuration.