Question 290
An assistant makes confident factual claims when relevant documents cannot be retrieved. Which architecture change should be approved first?
- For this production path, instrument no-evidence paths and enforce abstention or human escalation on missing support.
- For the intended users, benchmark a more capable model against the same difficult holdout before adopting it.
- At this decision point, diff prompt revisions and replay known failing cases using pinned component versions.
- As a candidate control, inspect ingestion, indexing and retrieval results before rewriting the system prompt.
- For the primary workstream, inspect tool schemas, examples and argument-validation errors at the invocation boundary.
Explanation
Correct Answer
A:The failure is unsupported synthesis; more assertive wording would worsen the defect; its effect can be measured when empty-retrieval tests yield acknowledged uncertainty rather than fabricated facts.
Incorrect Answers
B:Persistent reasoning failure after evidence and prompt checks can be a model-capability mismatch; this technique fits where a task requires complex multi-document reconciliation, and a small model misses constraints despite clean prompts, rather than the condition that an assistant makes confident factual claims when relevant documents cannot be retrieved.
C:Controlled replay distinguishes prompt regression from random variation or data changes; the trigger would be that a production model begins giving different answers after a silent prompt template edit, while the immediate question involves an assistant makes confident factual claims when relevant documents cannot be retrieved.
D:A time-correlated corpus update points first to source and evidence pipeline changes; the design helps when a RAG assistant deteriorates immediately after a large document refresh, but prompts did not change, but the present constraint is that an assistant makes confident factual claims when relevant documents cannot be retrieved.
E:Precise parameter contracts control tool execution more directly than general answer-quality tuning; that fits cases where an agent chooses the correct API but sends invalid arguments for a nested object, whereas the current issue is that an assistant makes confident factual claims when relevant documents cannot be retrieved.
Question 291
A task requires complex multi-document reconciliation, and a small model misses constraints despite clean prompts. What should be incorporated into the approved design?
- In the proposed design, instrument no-evidence paths and enforce abstention or human escalation on missing support.
- Under the reviewed approach, inspect tool schemas, examples and argument-validation errors at the invocation boundary.
- For this production path, benchmark a more capable model against the same difficult holdout before adopting it.
- For the intended users, diff prompt revisions and replay known failing cases using pinned component versions.
Explanation
Correct Answer
C:Persistent reasoning failure after evidence and prompt checks can be a model-capability mismatch; acceptance evidence must confirm that blind comparisons confirm improved constraint satisfaction without unacceptable cost.
Incorrect Answers
A:The failure is unsupported synthesis; more assertive wording would worsen the defect; the other workload is one where an assistant makes confident factual claims when relevant documents cannot be retrieved, not the risk where a task requires complex multi-document reconciliation, and a small model misses constraints despite clean prompts.
B:Precise parameter contracts control tool execution more directly than general answer-quality tuning; this applies when an agent chooses the correct API but sends invalid arguments for a nested object, not this situation where a task requires complex multi-document reconciliation, and a small model misses constraints despite clean prompts.
D:Controlled replay distinguishes prompt regression from random variation or data changes; it is justified if a production model begins giving different answers after a silent prompt template edit, but here a task requires complex multi-document reconciliation, and a small model misses constraints despite clean prompts.
Question 292
A production model begins giving different answers after a silent prompt template edit. Which approach should the design team choose?
- As a candidate control, diff prompt revisions and replay known failing cases using pinned component versions.
- In the initial solution, benchmark a more capable model against the same difficult holdout before adopting it.
- In the proposed design, inspect ingestion, indexing and retrieval results before rewriting the system prompt.
- Under the reviewed approach, evaluate grounding and enforce claim-to-source consistency during generation.
Explanation
Correct Answer
A:Controlled replay distinguishes prompt regression from random variation or data changes; the audit trail can establish that old-version replay restores prior behavior and identifies the problematic instruction change.
Incorrect Answers
B:Persistent reasoning failure after evidence and prompt checks can be a model-capability mismatch; the other case is when a task requires complex multi-document reconciliation, and a small model misses constraints despite clean prompts, but the actual risk is that a production model begins giving different answers after a silent prompt template edit.
C:A time-correlated corpus update points first to source and evidence pipeline changes; this makes sense when a RAG assistant deteriorates immediately after a large document refresh, but prompts did not change, but the requested boundary concerns a production model begins giving different answers after a silent prompt template edit.
D:Retrieval success alone does not guarantee faithful synthesis of the retrieved text; this technique fits where a retrieval bot finds the correct passage but the answer invents numbers not in that passage, not the present case where a production model begins giving different answers after a silent prompt template edit.
Question 293
A retrieval bot finds the correct passage but the answer invents numbers not in that passage. Where should the solution architect focus the implementation?
- In the proposed implementation, check schema instructions and output validation before changing model capability tiers.
- At this decision point, instrument no-evidence paths and enforce abstention or human escalation on missing support.
- As a candidate control, benchmark a more capable model against the same difficult holdout before adopting it.
- In the initial solution, evaluate grounding and enforce claim-to-source consistency during generation.
Explanation
Correct Answer
D:Retrieval success alone does not guarantee faithful synthesis of the retrieved text; the go-live check should establish that numeric-claim checks reject values absent from the cited source span.
Incorrect Answers
A:A formatting failure may reflect an underspecified interface rather than inadequate reasoning; it serves a setting where a model repeatedly returns valid prose but violates a strict JSON consumer contract, rather than the condition that a retrieval bot finds the correct passage but the answer invents numbers not in that passage.
B:The failure is unsupported synthesis; more assertive wording would worsen the defect; that is defensible if an assistant makes confident factual claims when relevant documents cannot be retrieved, while the immediate question involves a retrieval bot finds the correct passage but the answer invents numbers not in that passage.
C:Persistent reasoning failure after evidence and prompt checks can be a model-capability mismatch; the other workload is one where a task requires complex multi-document reconciliation, and a small model misses constraints despite clean prompts, but the present constraint is that a retrieval bot finds the correct passage but the answer invents numbers not in that passage.
Question 294
After a corpus rebuild, retrieval no longer surfaces the right clause although model and prompt versions are unchanged. What result discriminates a working safeguard from a demo?
- The reviewer observes that a before-and-after trace finds missing or stale answer-bearing passages.
- The release report verifies that schema stress cases isolate missing fields and invalid values reproducibly.
- The controlled replay verifies that traces show which required argument or type failed and prevent external side effects.
- A reproducible run shows that blind comparisons confirm improved constraint satisfaction without unacceptable cost.
Explanation
Correct Answer
A:A time-correlated corpus update points first to source and evidence pipeline changes; the condition under test is that a RAG assistant deteriorates immediately after a large document refresh, but prompts did not change.
Incorrect Answers
B:A formatting failure may reflect an underspecified interface rather than inadequate reasoning; it is relevant if a model repeatedly returns valid prose but violates a strict JSON consumer contract, whereas this scenario involves after a corpus rebuild, retrieval no longer surfaces the right clause although model and prompt versions are unchanged.
C:Precise parameter contracts control tool execution more directly than general answer-quality tuning; it serves conditions where an agent chooses the correct API but sends invalid arguments for a nested object, but here after a corpus rebuild, retrieval no longer surfaces the right clause although model and prompt versions are unchanged.
D:Persistent reasoning failure after evidence and prompt checks can be a model-capability mismatch; the other case is when a task requires complex multi-document reconciliation, and a small model misses constraints despite clean prompts, instead of when after a corpus rebuild, retrieval no longer surfaces the right clause although model and prompt versions are unchanged.
Question 295
The structured-output consumer rejects an otherwise helpful explanation because a required JSON field is absent. What outcome must the reviewer see before proceeding?
- A production-like test shows that empty-retrieval tests yield acknowledged uncertainty rather than fabricated facts.
- The release check confirms that a before-and-after trace finds missing or stale answer-bearing passages.
- The reviewer observes that schema stress cases isolate missing fields and invalid values reproducibly.
- The release report verifies that old-version replay restores prior behavior and identifies the problematic instruction change.
Explanation
Correct Answer
C:A formatting failure may reflect an underspecified interface rather than inadequate reasoning; this is relevant for cases where a model repeatedly returns valid prose but violates a strict JSON consumer contract.
Incorrect Answers
A:The failure is unsupported synthesis; more assertive wording would worsen the defect; this control is needed when an assistant makes confident factual claims when relevant documents cannot be retrieved, but the actual risk is that the structured-output consumer rejects an otherwise helpful explanation because a required JSON field is absent.
B:A time-correlated corpus update points first to source and evidence pipeline changes; that would be useful if a RAG assistant deteriorates immediately after a large document refresh, but prompts did not change, but the requested boundary concerns the structured-output consumer rejects an otherwise helpful explanation because a required JSON field is absent.
D:Controlled replay distinguishes prompt regression from random variation or data changes; that is defensible if a production model begins giving different answers after a silent prompt template edit, while this problem involves the structured-output consumer rejects an otherwise helpful explanation because a required JSON field is absent.
Question 296
A tool call passes schema name validation but fails on a nested payload's required data type. Which result should the pilot review prioritize?
- The observed outcome is that old-version replay restores prior behavior and identifies the problematic instruction change.
- The regression suite verifies that empty-retrieval tests yield acknowledged uncertainty rather than fabricated facts.
- A production-like test shows that numeric-claim checks reject values absent from the cited source span.
- The release check confirms that traces show which required argument or type failed and prevent external side effects.
Explanation
Correct Answer
D:Precise parameter contracts control tool execution more directly than general answer-quality tuning; the evidence must be robust when an agent chooses the correct API but sends invalid arguments for a nested object.
Incorrect Answers
A:Controlled replay distinguishes prompt regression from random variation or data changes; it targets cases where a production model begins giving different answers after a silent prompt template edit, while the immediate question involves a tool call passes schema name validation but fails on a nested payload's required data type.
B:The failure is unsupported synthesis; more assertive wording would worsen the defect; the approach helps when an assistant makes confident factual claims when relevant documents cannot be retrieved, but the present constraint is that a tool call passes schema name validation but fails on a nested payload's required data type.
C:Retrieval success alone does not guarantee faithful synthesis of the retrieved text; that belongs to situations where a retrieval bot finds the correct passage but the answer invents numbers not in that passage, whereas the current issue is that a tool call passes schema name validation but fails on a nested payload's required data type.
Question 297
A factual assistant invents authority when its retrieval layer returns no relevant passages. What result would directly validate the necessary safeguard?
- The scenario test demonstrates that blind comparisons confirm improved constraint satisfaction without unacceptable cost.
- The end-to-end check confirms that schema stress cases isolate missing fields and invalid values reproducibly.
- The observed outcome is that a before-and-after trace finds missing or stale answer-bearing passages.
- The regression suite verifies that numeric-claim checks reject values absent from the cited source span.
- The negative-case evaluation confirms that empty-retrieval tests yield acknowledged uncertainty rather than fabricated facts.
Explanation
Correct Answer
E:The failure is unsupported synthesis; more assertive wording would worsen the defect; this checks the situation in which an assistant makes confident factual claims when relevant documents cannot be retrieved.
Incorrect Answers
A:Persistent reasoning failure after evidence and prompt checks can be a model-capability mismatch; the design helps when a task requires complex multi-document reconciliation, and a small model misses constraints despite clean prompts, whereas this scenario involves a factual assistant invents authority when its retrieval layer returns no relevant passages.
B:A formatting failure may reflect an underspecified interface rather than inadequate reasoning; that fits cases where a model repeatedly returns valid prose but violates a strict JSON consumer contract, but here a factual assistant invents authority when its retrieval layer returns no relevant passages.
C:A time-correlated corpus update points first to source and evidence pipeline changes; this control is needed when a RAG assistant deteriorates immediately after a large document refresh, but prompts did not change, instead of when a factual assistant invents authority when its retrieval layer returns no relevant passages.
D:Retrieval success alone does not guarantee faithful synthesis of the retrieved text; that would be useful if a retrieval bot finds the correct passage but the answer invents numbers not in that passage, not the case in which a factual assistant invents authority when its retrieval layer returns no relevant passages.
Question 298
A compact model continues to miss a long dependency chain after prompt wording and retrieval have been corrected. What evidence would reveal whether the control truly works?
- A controlled trial finds that traces show which required argument or type failed and prevent external side effects.
- The measured evidence confirms that blind comparisons confirm improved constraint satisfaction without unacceptable cost.
- The scenario test demonstrates that numeric-claim checks reject values absent from the cited source span.
- The end-to-end check confirms that old-version replay restores prior behavior and identifies the problematic instruction change.
Explanation
Correct Answer
B:Persistent reasoning failure after evidence and prompt checks can be a model-capability mismatch; this is relevant for cases where a task requires complex multi-document reconciliation, and a small model misses constraints despite clean prompts.
Incorrect Answers
A:Precise parameter contracts control tool execution more directly than general answer-quality tuning; it is justified if an agent chooses the correct API but sends invalid arguments for a nested object, but the requested boundary concerns a compact model continues to miss a long dependency chain after prompt wording and retrieval have been corrected.
C:Retrieval success alone does not guarantee faithful synthesis of the retrieved text; it targets cases where a retrieval bot finds the correct passage but the answer invents numbers not in that passage, while this problem involves a compact model continues to miss a long dependency chain after prompt wording and retrieval have been corrected.
D:Controlled replay distinguishes prompt regression from random variation or data changes; the approach helps when a production model begins giving different answers after a silent prompt template edit, and not where a compact model continues to miss a long dependency chain after prompt wording and retrieval have been corrected.
Question 299
A nightly prompt deployment shifts refusal behavior even though no model update was recorded. Which result proves the requested control, not just configuration?
- The representative case test finds that schema stress cases isolate missing fields and invalid values reproducibly.
- The negative-case evaluation confirms that old-version replay restores prior behavior and identifies the problematic instruction change.
- A controlled trial finds that numeric-claim checks reject values absent from the cited source span.
- The measured evidence confirms that traces show which required argument or type failed and prevent external side effects.
Explanation
Correct Answer
B:Controlled replay distinguishes prompt regression from random variation or data changes; the reviewer is concerned with situations where a production model begins giving different answers after a silent prompt template edit.
Incorrect Answers
A:A formatting failure may reflect an underspecified interface rather than inadequate reasoning; this technique fits where a model repeatedly returns valid prose but violates a strict JSON consumer contract, but the present constraint is that a nightly prompt deployment shifts refusal behavior even though no model update was recorded.
C:Retrieval success alone does not guarantee faithful synthesis of the retrieved text; the design helps when a retrieval bot finds the correct passage but the answer invents numbers not in that passage, whereas the release criterion addresses a nightly prompt deployment shifts refusal behavior even though no model update was recorded.
D:Precise parameter contracts control tool execution more directly than general answer-quality tuning; that fits cases where an agent chooses the correct API but sends invalid arguments for a nested object, while the required safeguard concerns a nightly prompt deployment shifts refusal behavior even though no model update was recorded.
Question 300
A deployment has passing prelaunch tests but user complaints rise over time. How should the architect revise the proposed solution?
- For this workflow, add behavioral outcome metrics alongside infrastructure availability and timing.
- For the affected service, run scheduled regression probes with version tags and groundedness measures.
- In the project scope, monitor rate-limit and retry telemetry with saturation and backoff behavior.
- Within the rollout design, monitor sampled production correctness and data drift with reviewed feedback queues.
Explanation
Correct Answer
D:Offline validation must continue as actual inputs and source material change; the intended outcome is that a drift alert links emerging failure categories to newly observed query patterns.
Incorrect Answers
A:Healthy servers do not prove acceptable task performance or appropriate human review workload; this applies when a support system has a rising escalation rate with no change in average response latency, but here a deployment has passing prelaunch tests but user complaints rise over time.
B:Longitudinal quality monitoring requires comparable, labeled cases across releases; the alternative helps if a vendor model update changes output style but existing metrics cannot tell whether correctness changed, instead of when a deployment has passing prelaunch tests but user complaints rise over time.
C:Rate limiting is a capacity interaction that cannot be diagnosed from answer quality alone; it is justified if a model interface experiences intermittent 429 responses during a traffic surge, not the case in which a deployment has passing prelaunch tests but user complaints rise over time.