A context engineer is evaluating a Databricks customer support agent that handles complex multi-turn conversations involving subscription changes, billing adjustments, and service cancellation requests.
The agent uses Lakebase-backed session storage to maintain conversation history. MLflow 3 tracing is enabled, and each inference trace includes a session identifier.
During testing, the engineer observes the following behavior:
Turn 1: The customer requests cancellation of an existing subscription.
Turn 2: The agent correctly identifies the subscription and explains the cancellation procedure.
Turn 4: The customer specifies that cancellation must occur at the end of the current billing cycle.
Turn 6: The agent retrieves the relevant cancellation policy and correctly answers a billing question.
Turn 8: The agent generates a final confirmation stating that the subscription will be canceled immediately, overlooking the customer ' s earlier instruction to cancel at the end of the billing cycle.
Individual response evaluations show high correctness and relevance scores. However, the operations team suspects that the agent is failing to preserve important information across conversational turns.
The engineer needs an evaluation mechanism that detects failures involving information introduced earlier in a conversation, rather than scoring each response independently.
Which MLflow evaluation approach should the context engineer implement?

• A. Group traces by their conversation session identifiers and run MLflow ' s multi-turn KnowledgeRetention judge to evaluate whether information introduced in earlier turns is correctly maintained throughout the conversation.
• B. Run the built-in Correctness scorer independently on each response trace, increasing the number of test questions to improve statistical confidence in the results.
• C. Increase the number of MLflow traces collected per conversation and compare average token usage, tool-call count, and latency between successful and unsuccessful sessions.
• D. Evaluate only the final response against the latest retrieved policy document, since the agent ' s correctness should depend primarily on the most recent authoritative source.

A context engineer is developing a RAG pipeline for a pharmaceutical knowledge base. Document chunks from clinical trial summaries are stored in a Unity Catalog-governed Delta table with precomputed embedding vectors. Pure vector similarity search retrieves topically related chunks effectively but misses chunks that depend on precise drug compound names and dosage identifiers in the search string, because those tokens are underrepresented in the embedding space. The engineer wants to recover those exact-term chunks while preserving the semantic retrieval that already works, without standing up a separate keyword index or adding a downstream merge step.
Which configuration should the engineer use to close this retrieval gap?

• A. Query the index using a filter with a LIKE operator in conjunction with approximate nearest neighbors (ANN) to return a single ranking while recovering the specific exact-term chunks being ignored in the original search.
• B. Query the index with hybrid search, using approximate nearest neighbors (ANN) and full-text search in one call and returning a single fused ranking.
• C. Query the index using full-text search, then leverage the Databricks reranker with approximate nearest neighbors (ANN) to return a single hybrid ranking.
• D. Query the index using approximate nearest neighbors (ANN), then filter the returned chunks to those containing the exact drug and dosage terms, resulting in a single ranked result set.

A context engineer is tuning a compaction prompt for a Databricks agent that summarizes customer support ticket histories before sending them to a downstream LLM. The trace being compacted spans 40 turns, and the compacted output must fit within the downstream LLM's context window. During initial evaluation against a benchmark of escalated support cases, the engineer observes that the compaction output consistently omits several prior resolutions and key escalation events, even though it fits well within the downstream LLM's context window.
What tuning step should the engineer take next?

• A. Add retention rules and pruning rules in the same iteration so recall and precision are tuned together rather than over separate passes.
• B. Add a recency rule that keeps the most recent turns in full and compresses the earlier turns, since the latest exchanges usually carry the most relevant context.
• C. Broaden the compaction prompt's retention rules to capture all events, resolutions, and escalation signals, accepting a longer summary for now and tightening verbosity in a later iteration.
• D. Add a token-count cap to the compaction prompt to shorten the summary and lower downstream inference cost and latency, then check afterward whether the key events were retained.

A context engineer is investigating an AI agent deployed on Databricks that handles customer support operations. The agent has two MCP tools registered: one described as "Searches customer support tickets and open case records for a given customer", and another described as "Looks up customer issue history and unresolved case details by customer ID".
When users ask whether a customer has any open issues, the agent randomly chooses between the two tools across repeated identical requests. When users ask about a customer's past resolved issues, the agent similarly misfires between the two tools. The engineer has reviewed the tool call logs and confirmed that the agent cannot reliably distinguish which tool to invoke.
Which overlap between the two tool descriptions is causing the agent to select tools ambiguously?

• A. Both descriptions omit the data source or system name, so the agent cannot determine which backend database each tool queries when forming a retrieval plan.
• B. Both descriptions use action verbs (Searches and Looks up) that belong to the same semantic category, causing the agent to treat the tools as interchangeable retrieval operations.
• C. Both descriptions reference a customer identifier as the lookup parameter, making the input signature indistinguishable to the agent when resolving tool selection.
• D. Both descriptions claim the same functional scope: each covers open cases (open case records and unresolved case details) and historical records (support tickets and issue history), leaving no functional differentiator.