Question 230
At peak time, a ranking stage adds large tail delays despite generating accurate source-grounded answers. Which verified outcome most directly addresses the reported risk?
- A reproducible run shows that role-change probes cannot read a prior user's cached result.
- The signed test record confirms that a trace benchmark shows improved tail latency with bounded quality degradation.
- The sign-off record demonstrates that a sample trace attributes elapsed time to each stage and identifies the dominant component.
- The acceptance evidence shows that budgeted retrieval meets answer recall targets while reducing input tokens per query.
Explanation
Correct Answer
B:Targeted reranking preserves difficult-query accuracy without imposing the same cost on every request; the test is necessary where a support assistant returns accurate answers but p95 latency spikes during complex reranking.
Incorrect Answers
A:Latency optimization must not bypass authorization or freshness boundaries; this makes sense when an engineer proposes caching all user responses even when access changes frequently, rather than the risk that at peak time, a ranking stage adds large tail delays despite generating accurate source-grounded answers.
C:Stage-level telemetry directs optimization to the actual bottleneck rather than speculative tuning; the trigger would be that a service records slow end-to-end responses but cannot tell whether search or generation dominates, while the immediate question involves at peak time, a ranking stage adds large tail delays despite generating accurate source-grounded answers.
D:Context pruning can reduce inference latency and cost when it preserves answer-bearing evidence; the design helps when an answer generator receives many low-ranked chunks and uses most of its time parsing them, but the present constraint is that at peak time, a ranking stage adds large tail delays despite generating accurate source-grounded answers.
Question 231
An updated privacy notice is live but cached responses still quote yesterday's retention policy. What independently verifiable observation is needed?
- The release report verifies that a sample trace attributes elapsed time to each stage and identifies the dominant component.
- The controlled replay verifies that high-stakes cases retain citation quality while routine cases satisfy their response SLA.
- A reproducible run shows that after a policy replacement, subsequent requests cite the new version inside the required interval.
- The regression suite verifies that a trace benchmark shows improved tail latency with bounded quality degradation.
Explanation
Correct Answer
C:Caching reduces latency only when its staleness window matches the business obligation; it demonstrates the required control when a business query is served from a stale cache after a policy update.
Incorrect Answers
A:Stage-level telemetry directs optimization to the actual bottleneck rather than speculative tuning; the other workload is one where a service records slow end-to-end responses but cannot tell whether search or generation dominates, not the risk where an updated privacy notice is live but cached responses still quote yesterday's retention policy.
B:Risk-adaptive search applies expensive verification only when correctness stakes justify the delay; this applies when a critical legal workflow benefits from reranking but casual FAQ answers do not, not this situation where an updated privacy notice is live but cached responses still quote yesterday's retention policy.
D:Targeted reranking preserves difficult-query accuracy without imposing the same cost on every request; it is justified if a support assistant returns accurate answers but p95 latency spikes during complex reranking, but here an updated privacy notice is live but cached responses still quote yesterday's retention policy.
Question 232
A narrow question sends twenty marginal chunks to the model and crowds out the best evidence. What outcome would demonstrate this requirement in operation?
- The release check confirms that after a policy replacement, subsequent requests cite the new version inside the required interval.
- The reviewer observes that budgeted retrieval meets answer recall targets while reducing input tokens per query.
- The release report verifies that a trace benchmark shows improved tail latency with bounded quality degradation.
- The end-to-end check confirms that high-stakes cases retain citation quality while routine cases satisfy their response SLA.
Explanation
Correct Answer
B:Context pruning can reduce inference latency and cost when it preserves answer-bearing evidence; the observed protection matters when an answer generator receives many low-ranked chunks and uses most of its time parsing them.
Incorrect Answers
A:Caching reduces latency only when its staleness window matches the business obligation; it serves conditions where a business query is served from a stale cache after a policy update, but this decision concerns a narrow question sends twenty marginal chunks to the model and crowds out the best evidence.
C:Targeted reranking preserves difficult-query accuracy without imposing the same cost on every request; this makes sense when a support assistant returns accurate answers but p95 latency spikes during complex reranking, but the requested boundary concerns a narrow question sends twenty marginal chunks to the model and crowds out the best evidence.
D:Risk-adaptive search applies expensive verification only when correctness stakes justify the delay; this technique fits where a critical legal workflow benefits from reranking but casual FAQ answers do not, not the present case where a narrow question sends twenty marginal chunks to the model and crowds out the best evidence.
Question 233
A lightweight FAQ request pays for expensive reranking despite already finding an exact answer. Which observed test outcome would justify sign-off?
- The regression suite verifies that high-stakes cases retain citation quality while routine cases satisfy their response SLA.
- A production-like test shows that role-change probes cannot read a prior user's cached result.
- A controlled trial finds that after a policy replacement, subsequent requests cite the new version inside the required interval.
- The reviewer observes that a trace benchmark shows improved tail latency with bounded quality degradation.
- The scenario test demonstrates that a sample trace attributes elapsed time to each stage and identifies the dominant component.
Explanation
Correct Answer
A:Risk-adaptive search applies expensive verification only when correctness stakes justify the delay; the test is necessary where a critical legal workflow benefits from reranking but casual FAQ answers do not.
Incorrect Answers
B:Latency optimization must not bypass authorization or freshness boundaries; that is defensible if an engineer proposes caching all user responses even when access changes frequently, while the immediate question involves a lightweight FAQ request pays for expensive reranking despite already finding an exact answer.
C:Caching reduces latency only when its staleness window matches the business obligation; the other workload is one where a business query is served from a stale cache after a policy update, but the present constraint is that a lightweight FAQ request pays for expensive reranking despite already finding an exact answer.
D:Targeted reranking preserves difficult-query accuracy without imposing the same cost on every request; this applies when a support assistant returns accurate answers but p95 latency spikes during complex reranking, whereas the current issue is that a lightweight FAQ request pays for expensive reranking despite already finding an exact answer.
E:Stage-level telemetry directs optimization to the actual bottleneck rather than speculative tuning; the alternative helps if a service records slow end-to-end responses but cannot tell whether search or generation dominates, whereas the release criterion addresses a lightweight FAQ request pays for expensive reranking despite already finding an exact answer.
Question 234
Tracing shows slow results but no separate timing for lexical search, reranking and generation. Which sign-off finding best checks the actual boundary?
- The end-to-end check confirms that budgeted retrieval meets answer recall targets while reducing input tokens per query.
- The observed outcome is that after a policy replacement, subsequent requests cite the new version inside the required interval.
- The regression suite verifies that a sample trace attributes elapsed time to each stage and identifies the dominant component.
- A production-like test shows that peak-load trials meet both retrieval recall and request-time targets.
Explanation
Correct Answer
C:Stage-level telemetry directs optimization to the actual bottleneck rather than speculative tuning; the test covers the original risk that a service records slow end-to-end responses but cannot tell whether search or generation dominates.
Incorrect Answers
A:Context pruning can reduce inference latency and cost when it preserves answer-bearing evidence; that belongs to situations where an answer generator receives many low-ranked chunks and uses most of its time parsing them, not this situation where tracing shows slow results but no separate timing for lexical search, reranking and generation.
B:Caching reduces latency only when its staleness window matches the business obligation; it is relevant if a business query is served from a stale cache after a policy update, whereas this scenario involves tracing shows slow results but no separate timing for lexical search, reranking and generation.
D:Search infrastructure pressure may damage ranking or timeout behavior before outright availability fails; the other case is when a RAG system scales to more simultaneous users and begins returning fewer relevant chunks, instead of when tracing shows slow results but no separate timing for lexical search, reranking and generation.
Question 235
An agent succeeds only 70% of the time and the team sees only a final success flag. Which option supplies the needed architectural boundary?
- For the current integration, define service-level indicators, capacity budgets and autoscaling tests for peak traffic.
- Under the system plan, correlate groundedness evaluations, retrieved evidence and prompt-model-index versions.
- As the design priority, propagate end-to-end correlation IDs across asynchronous and synchronous boundaries.
- In the implementation sequence, record correlated spans for model requests, tool calls, retrieval and downstream effects.
Explanation
Correct Answer
D:Distributed traces locate failure stages and support reproducible diagnosis of multi-step workflows; the practical acceptance condition is that a failed request exposes the exact failing tool and correlated model invocation.
Incorrect Answers
A:Production scaling needs measured saturation behavior rather than extrapolation from a small pilot; this control is needed when a pilot is stable at ten users but becomes unpredictable at thousands of concurrent sessions, but the actual risk is that an agent succeeds only 70% of the time and the team sees only a final success flag.
B:HTTP success is not content correctness; quality drift needs semantic and provenance instrumentation; that would be useful if a team sees a sudden rise in hallucination complaints but logs only HTTP status codes, but the requested boundary concerns an agent succeeds only 70% of the time and the team sees only a final success flag.
C:Shared identifiers are necessary to reconstruct one user journey through independently instrumented services; it serves a setting where a request crosses three services but each issues independent trace IDs, not the present case where an agent succeeds only 70% of the time and the team sees only a final success flag.
Question 236
A production service logs full customer prompts and exposes private information to broad staff access. Which action addresses the root concern without unnecessary scope?
- Within the rollout design, publish security, quality and failure metrics alongside cost and latency signals.
- For the next architecture revision, define service-level indicators, capacity budgets and autoscaling tests for peak traffic.
- For the current integration, correlate groundedness evaluations, retrieved evidence and prompt-model-index versions.
- Under the system plan, redact sensitive fields and apply retention and role controls to observability data.
Explanation
Correct Answer
D:Useful debugging cannot justify uncontrolled duplication of confidential customer inputs; one measurable outcome is that audit probes verify restricted log access and no forbidden raw identifiers in routine traces.
Incorrect Answers
A:An observability program should expose risk and correctness, not merely expenditure; it targets cases where an operations team monitors token usage but not the rate of unauthorized tool denials, while the immediate question involves a production service logs full customer prompts and exposes private information to broad staff access.
B:Production scaling needs measured saturation behavior rather than extrapolation from a small pilot; the approach helps when a pilot is stable at ten users but becomes unpredictable at thousands of concurrent sessions, but the present constraint is that a production service logs full customer prompts and exposes private information to broad staff access.
C:HTTP success is not content correctness; quality drift needs semantic and provenance instrumentation; that belongs to situations where a team sees a sudden rise in hallucination complaints but logs only HTTP status codes, whereas the current issue is that a production service logs full customer prompts and exposes private information to broad staff access.
Question 237
A request crosses three services but each issues independent trace IDs. What is the most appropriate initial architecture decision?
- For the affected service, add bounded retries with jitter, circuit breaking and saturation alerts.
- In the project scope, define service-level indicators, capacity budgets and autoscaling tests for peak traffic.
- Within the rollout design, propagate end-to-end correlation IDs across asynchronous and synchronous boundaries.
- For the next architecture revision, redact sensitive fields and apply retention and role controls to observability data.
- For the current integration, publish security, quality and failure metrics alongside cost and latency signals.
Explanation
Correct Answer
C:Shared identifiers are necessary to reconstruct one user journey through independently instrumented services; validation needs to show that a queued retry can be joined to its original request and downstream outcome.
Incorrect Answers
A:Resilience controls prevent transient faults from becoming a cascading traffic storm; the design helps when a tool endpoint experiences intermittent failures and causes retries to overwhelm dependencies, whereas this scenario involves a request crosses three services but each issues independent trace IDs.
B:Production scaling needs measured saturation behavior rather than extrapolation from a small pilot; that fits cases where a pilot is stable at ten users but becomes unpredictable at thousands of concurrent sessions, but here a request crosses three services but each issues independent trace IDs.
D:Useful debugging cannot justify uncontrolled duplication of confidential customer inputs; that would be useful if a production service logs full customer prompts and exposes private information to broad staff access, not the case in which a request crosses three services but each issues independent trace IDs.
E:An observability program should expose risk and correctness, not merely expenditure; it serves a setting where an operations team monitors token usage but not the rate of unauthorized tool denials, whereas the problem presented is that a request crosses three services but each issues independent trace IDs.
Question 238
An operations team monitors token usage but not the rate of unauthorized tool denials. Which architecture change should be approved first?
- For the primary workstream, record correlated spans for model requests, tool calls, retrieval and downstream effects.
- For this workflow, correlate groundedness evaluations, retrieved evidence and prompt-model-index versions.
- For the affected service, publish security, quality and failure metrics alongside cost and latency signals.
- In the project scope, add bounded retries with jitter, circuit breaking and saturation alerts.
Explanation
Correct Answer
C:An observability program should expose risk and correctness, not merely expenditure; the practical acceptance condition is that dashboards show denied actions, safety escalations, answer quality and availability trends.
Incorrect Answers
A:Distributed traces locate failure stages and support reproducible diagnosis of multi-step workflows; it is justified if an agent succeeds only 70% of the time and the team sees only a final success flag, but the requested boundary concerns an operations team monitors token usage but not the rate of unauthorized tool denials.
B:HTTP success is not content correctness; quality drift needs semantic and provenance instrumentation; its scope includes cases where a team sees a sudden rise in hallucination complaints but logs only HTTP status codes, not the present case where an operations team monitors token usage but not the rate of unauthorized tool denials.
D:Resilience controls prevent transient faults from becoming a cascading traffic storm; the approach helps when a tool endpoint experiences intermittent failures and causes retries to overwhelm dependencies, and not where an operations team monitors token usage but not the rate of unauthorized tool denials.
Question 239
A tool endpoint experiences intermittent failures and causes retries to overwhelm dependencies. What should be incorporated into the approved design?
- Within the delivery plan, record correlated spans for model requests, tool calls, retrieval and downstream effects.
- At the system boundary, define service-level indicators, capacity budgets and autoscaling tests for peak traffic.
- For the primary workstream, add bounded retries with jitter, circuit breaking and saturation alerts.
- For this workflow, publish security, quality and failure metrics alongside cost and latency signals.
Explanation
Correct Answer
C:Resilience controls prevent transient faults from becoming a cascading traffic storm; one measurable outcome is that a fault-injection test caps retry amplification and recovers when the dependency stabilizes.
Incorrect Answers
A:Distributed traces locate failure stages and support reproducible diagnosis of multi-step workflows; this technique fits where an agent succeeds only 70% of the time and the team sees only a final success flag, but the present constraint is that a tool endpoint experiences intermittent failures and causes retries to overwhelm dependencies.
B:Production scaling needs measured saturation behavior rather than extrapolation from a small pilot; the trigger would be that a pilot is stable at ten users but becomes unpredictable at thousands of concurrent sessions, whereas the current issue is that a tool endpoint experiences intermittent failures and causes retries to overwhelm dependencies.
D:An observability program should expose risk and correctness, not merely expenditure; that fits cases where an operations team monitors token usage but not the rate of unauthorized tool denials, while the required safeguard concerns a tool endpoint experiences intermittent failures and causes retries to overwhelm dependencies.
Question 240
A pilot is stable at ten users but becomes unpredictable at thousands of concurrent sessions. Which approach should the design team choose?
- For this production path, correlate groundedness evaluations, retrieved evidence and prompt-model-index versions.
- For the intended users, record correlated spans for model requests, tool calls, retrieval and downstream effects.
- Within the delivery plan, define service-level indicators, capacity budgets and autoscaling tests for peak traffic.
- At the system boundary, publish security, quality and failure metrics alongside cost and latency signals.
Explanation
Correct Answer
C:Production scaling needs measured saturation behavior rather than extrapolation from a small pilot; one operational check should confirm that peak-load tests meet availability and tail-latency thresholds without runaway queue growth.
Incorrect Answers
A:HTTP success is not content correctness; quality drift needs semantic and provenance instrumentation; this applies when a team sees a sudden rise in hallucination complaints but logs only HTTP status codes, but here a pilot is stable at ten users but becomes unpredictable at thousands of concurrent sessions.
B:Distributed traces locate failure stages and support reproducible diagnosis of multi-step workflows; the alternative helps if an agent succeeds only 70% of the time and the team sees only a final success flag, instead of when a pilot is stable at ten users but becomes unpredictable at thousands of concurrent sessions.
D:An observability program should expose risk and correctness, not merely expenditure; its scope includes cases where an operations team monitors token usage but not the rate of unauthorized tool denials, whereas the problem presented is that a pilot is stable at ten users but becomes unpredictable at thousands of concurrent sessions.