Anthropic CCAR-P (Claude Certified Architect - Professional) Practice Test 25

Question 240

A pilot is stable at ten users but becomes unpredictable at thousands of concurrent sessions. Which approach should the design team choose?

  1. For this production path, correlate groundedness evaluations, retrieved evidence and prompt-model-index versions.
  2. For the intended users, record correlated spans for model requests, tool calls, retrieval and downstream effects.
  3. Within the delivery plan, define service-level indicators, capacity budgets and autoscaling tests for peak traffic.
  4. At the system boundary, publish security, quality and failure metrics alongside cost and latency signals.

Explanation

Correct Answer

C:

Production scaling needs measured saturation behavior rather than extrapolation from a small pilot; one operational check should confirm that peak-load tests meet availability and tail-latency thresholds without runaway queue growth.

Incorrect Answers

A:

HTTP success is not content correctness; quality drift needs semantic and provenance instrumentation; this applies when a team sees a sudden rise in hallucination complaints but logs only HTTP status codes, but here a pilot is stable at ten users but becomes unpredictable at thousands of concurrent sessions.

B:

Distributed traces locate failure stages and support reproducible diagnosis of multi-step workflows; the alternative helps if an agent succeeds only 70% of the time and the team sees only a final success flag, instead of when a pilot is stable at ten users but becomes unpredictable at thousands of concurrent sessions.

D:

An observability program should expose risk and correctness, not merely expenditure; its scope includes cases where an operations team monitors token usage but not the rate of unauthorized tool denials, whereas the problem presented is that a pilot is stable at ten users but becomes unpredictable at thousands of concurrent sessions.

Question 241

A team sees a sudden rise in hallucination complaints but logs only HTTP status codes. Where should the solution architect focus the implementation?

  1. In the proposed design, correlate groundedness evaluations, retrieved evidence and prompt-model-index versions.
  2. Under the reviewed approach, add bounded retries with jitter, circuit breaking and saturation alerts.
  3. For this production path, publish security, quality and failure metrics alongside cost and latency signals.
  4. For the intended users, define service-level indicators, capacity budgets and autoscaling tests for peak traffic.

Explanation

Correct Answer

A:

HTTP success is not content correctness; quality drift needs semantic and provenance instrumentation; production testing should demonstrate that incident review connects the regression to a specific retrieval or prompt release.

Incorrect Answers

B:

Resilience controls prevent transient faults from becoming a cascading traffic storm; this makes sense when a tool endpoint experiences intermittent failures and causes retries to overwhelm dependencies, while this problem involves a team sees a sudden rise in hallucination complaints but logs only HTTP status codes.

C:

An observability program should expose risk and correctness, not merely expenditure; this technique fits where an operations team monitors token usage but not the rate of unauthorized tool denials, and not where a team sees a sudden rise in hallucination complaints but logs only HTTP status codes.

D:

Production scaling needs measured saturation behavior rather than extrapolation from a small pilot; the trigger would be that a pilot is stable at ten users but becomes unpredictable at thousands of concurrent sessions, rather than the risk that a team sees a sudden rise in hallucination complaints but logs only HTTP status codes.

Question 242

A complex agent run ends with failure, yet operators cannot tell whether planning, retrieval or a tool caused it. What result discriminates a working safeguard from a demo?

  1. The signed test record confirms that a failed request exposes the exact failing tool and correlated model invocation.
  2. The sign-off record demonstrates that a queued retry can be joined to its original request and downstream outcome.
  3. The acceptance evidence shows that peak-load tests meet availability and tail-latency thresholds without runaway queue growth.
  4. The pilot evaluation finds that a fault-injection test caps retry amplification and recovers when the dependency stabilizes.

Explanation

Correct Answer

A:

Distributed traces locate failure stages and support reproducible diagnosis of multi-step workflows; the reviewer is concerned with situations where an agent succeeds only 70% of the time and the team sees only a final success flag.

Incorrect Answers

B:

Shared identifiers are necessary to reconstruct one user journey through independently instrumented services; the other workload is one where a request crosses three services but each issues independent trace IDs, whereas the release criterion addresses a complex agent run ends with failure, yet operators cannot tell whether planning, retrieval or a tool caused it.

C:

Production scaling needs measured saturation behavior rather than extrapolation from a small pilot; this applies when a pilot is stable at ten users but becomes unpredictable at thousands of concurrent sessions, while the required safeguard concerns a complex agent run ends with failure, yet operators cannot tell whether planning, retrieval or a tool caused it.

D:

Resilience controls prevent transient faults from becoming a cascading traffic storm; the alternative helps if a tool endpoint experiences intermittent failures and causes retries to overwhelm dependencies, not the risk where a complex agent run ends with failure, yet operators cannot tell whether planning, retrieval or a tool caused it.

Question 243

A privacy audit discovers internal staff can search raw customer prompts in observability dashboards. What outcome must the reviewer see before proceeding?

  1. The controlled replay verifies that a failed request exposes the exact failing tool and correlated model invocation.
  2. A reproducible run shows that peak-load tests meet availability and tail-latency thresholds without runaway queue growth.
  3. The signed test record confirms that a fault-injection test caps retry amplification and recovers when the dependency stabilizes.
  4. The sign-off record demonstrates that audit probes verify restricted log access and no forbidden raw identifiers in routine traces.

Explanation

Correct Answer

D:

Useful debugging cannot justify uncontrolled duplication of confidential customer inputs; the control should hold even if a production service logs full customer prompts and exposes private information to broad staff access.

Incorrect Answers

A:

Distributed traces locate failure stages and support reproducible diagnosis of multi-step workflows; it is relevant if an agent succeeds only 70% of the time and the team sees only a final success flag, instead of when a privacy audit discovers internal staff can search raw customer prompts in observability dashboards.

B:

Production scaling needs measured saturation behavior rather than extrapolation from a small pilot; it serves conditions where a pilot is stable at ten users but becomes unpredictable at thousands of concurrent sessions, not the case in which a privacy audit discovers internal staff can search raw customer prompts in observability dashboards.

C:

Resilience controls prevent transient faults from becoming a cascading traffic storm; the other case is when a tool endpoint experiences intermittent failures and causes retries to overwhelm dependencies, whereas the problem presented is that a privacy audit discovers internal staff can search raw customer prompts in observability dashboards.

Question 244

A request passes through multiple microservices, and its log fragments cannot be joined to one business transaction. Which result should the pilot review prioritize?

  1. The reviewer observes that a failed request exposes the exact failing tool and correlated model invocation.
  2. The release report verifies that peak-load tests meet availability and tail-latency thresholds without runaway queue growth.
  3. The controlled replay verifies that dashboards show denied actions, safety escalations, answer quality and availability trends.
  4. A reproducible run shows that audit probes verify restricted log access and no forbidden raw identifiers in routine traces.
  5. The signed test record confirms that a queued retry can be joined to its original request and downstream outcome.

Explanation

Correct Answer

E:

Shared identifiers are necessary to reconstruct one user journey through independently instrumented services; it tests the operating case in which a request crosses three services but each issues independent trace IDs.

Incorrect Answers

A:

Distributed traces locate failure stages and support reproducible diagnosis of multi-step workflows; that would be useful if an agent succeeds only 70% of the time and the team sees only a final success flag, while this problem involves a request passes through multiple microservices, and its log fragments cannot be joined to one business transaction.

B:

Production scaling needs measured saturation behavior rather than extrapolation from a small pilot; it serves a setting where a pilot is stable at ten users but becomes unpredictable at thousands of concurrent sessions, and not where a request passes through multiple microservices, and its log fragments cannot be joined to one business transaction.

C:

An observability program should expose risk and correctness, not merely expenditure; that is defensible if an operations team monitors token usage but not the rate of unauthorized tool denials, rather than the risk that a request passes through multiple microservices, and its log fragments cannot be joined to one business transaction.

D:

Useful debugging cannot justify uncontrolled duplication of confidential customer inputs; the other workload is one where a production service logs full customer prompts and exposes private information to broad staff access, rather than the condition that a request passes through multiple microservices, and its log fragments cannot be joined to one business transaction.

Question 245

An agent's monitoring system reports costs and HTTP errors, but security reviewers cannot see failed restricted-tool attempts and users are reporting unsupported claims after a recent content change. Which TWO demonstrated observability outcomes would make both failure types diagnosable? Choose TWO.

  1. A production-like test shows that dashboards show denied actions, safety escalations, answer quality and availability trends.
  2. The release check confirms that audit probes verify restricted log access and no forbidden raw identifiers in routine traces.
  3. The reviewer observes that incident review connects the regression to a specific retrieval or prompt release.
  4. The release report verifies that a failed request exposes the exact failing tool and correlated model invocation.
  5. The controlled replay verifies that a queued retry can be joined to its original request and downstream outcome.

Explanation

Correct Answers

A:

An observability program should expose risk and correctness, not merely expenditure; the evidence must be robust when an operations team monitors token usage but not the rate of unauthorized tool denials; another obligation arises when a team sees a sudden rise in hallucination complaints but logs only HTTP status codes.

C:

HTTP success is not content correctness; quality drift needs semantic and provenance instrumentation; an acceptance test needs to cover cases where a team sees a sudden rise in hallucination complaints but logs only HTTP status codes; another obligation arises when an operations team monitors token usage but not the rate of unauthorized tool denials.

Incorrect Answers

B:

Useful debugging cannot justify uncontrolled duplication of confidential customer inputs; that belongs to situations where a production service logs full customer prompts and exposes private information to broad staff access, while the required safeguard concerns the dashboard reports usage spend but cannot identify suspicious denied write attempts; another obligation arises when a team sees a sudden rise in hallucination complaints but logs only HTTP status codes.

D:

Distributed traces locate failure stages and support reproducible diagnosis of multi-step workflows; it serves conditions where an agent succeeds only 70% of the time and the team sees only a final success flag, not this situation where the dashboard reports usage spend but cannot identify suspicious denied write attempts; another obligation arises when a team sees a sudden rise in hallucination complaints but logs only HTTP status codes.

E:

Shared identifiers are necessary to reconstruct one user journey through independently instrumented services; the other case is when a request crosses three services but each issues independent trace IDs, whereas this scenario involves users report unsupported answers while infrastructure graphs remain green and contain no grounding indicators; another obligation arises when an operations team monitors token usage but not the rate of unauthorized tool denials.

Question 246

A loan-policy assistant is judged by user satisfaction even when it makes incorrect eligibility statements. What intervention best matches the stated boundary?

  1. In the implementation sequence, set percentile latency and timeout metrics alongside completion and token cost.
  2. For the operating team, track tool-policy violations and approval bypass attempts as independent safety metrics.
  3. For this deployment, stratify evaluation by risk, query type and frequency instead of relying on aggregate accuracy.
  4. As a scoped first step, define policy-decision correctness as a primary metric with safety-critical error categories.

Explanation

Correct Answer

D:

Satisfaction can reward fluency while overlooking harmful violations of binding eligibility criteria; a representative trial should prove that a labeled case set reports false approvals and false denials separately by severity.

Incorrect Answers

A:

Averages conceal long-tail delays that may break user experience obligations; that fits cases where a real-time assistant averages one second but occasionally blocks users for thirty seconds, not the case in which a loan-policy assistant is judged by user satisfaction even when it makes incorrect eligibility statements.

B:

Answer correctness cannot substitute for limits on what external side effects the agent may produce; this control is needed when a tool-using agent makes accurate decisions but sometimes performs unauthorized actions, whereas the problem presented is that a loan-policy assistant is judged by user satisfaction even when it makes incorrect eligibility statements.

C:

A high overall score can conceal failures in infrequent but consequential cases; that would be useful if a knowledge assistant is accurate on common questions but misses rare safety procedures, but this decision concerns a loan-policy assistant is judged by user satisfaction even when it makes incorrect eligibility statements.

Question 247

A knowledge assistant is accurate on common questions but misses rare safety procedures. Which implementation decision is the strongest fit?

  1. Under the system plan, measure net handling time, escalation rate and correction effort against the manual baseline.
  2. As the design priority, define policy-decision correctness as a primary metric with safety-critical error categories.
  3. In the implementation sequence, track tool-policy violations and approval bypass attempts as independent safety metrics.
  4. For the operating team, use a weighted evaluation scorecard with explicit hard safety gates for critical errors.
  5. Within the rollout design, stratify evaluation by risk, query type and frequency instead of relying on aggregate accuracy.

Explanation

Correct Answer

E:

A high overall score can conceal failures in infrequent but consequential cases; the implementation should produce evidence that release metrics show per-stratum recall and a separately enforced critical-case threshold.

Incorrect Answers

A:

Value must include downstream verification and rework, not just generation time; its scope includes cases where an internal ticket agent claims to save time without counting extra reviewer effort, and not where a knowledge assistant is accurate on common questions but misses rare safety procedures.

B:

Satisfaction can reward fluency while overlooking harmful violations of binding eligibility criteria; it targets cases where a loan-policy assistant is judged by user satisfaction even when it makes incorrect eligibility statements, rather than the risk that a knowledge assistant is accurate on common questions but misses rare safety procedures.

C:

Answer correctness cannot substitute for limits on what external side effects the agent may produce; the approach helps when a tool-using agent makes accurate decisions but sometimes performs unauthorized actions, rather than the condition that a knowledge assistant is accurate on common questions but misses rare safety procedures.

D:

Multidimensional acceptance stops inexpensive or popular outputs from hiding severe failures; that belongs to situations where a stakeholder requests one universal score for a regulated assistant across all tasks, while the immediate question involves a knowledge assistant is accurate on common questions but misses rare safety procedures.

Question 248

A customer-support assistant seems to finish tickets faster, but rework consumes staff time and occasional requests become extremely slow. What TWO metrics must be part of the release scorecard to avoid overstating efficiency and hiding the experience tail? Choose TWO.

  1. For the next architecture revision, define policy-decision correctness as a primary metric with safety-critical error categories.
  2. For the current integration, stratify evaluation by risk, query type and frequency instead of relying on aggregate accuracy.
  3. Under the system plan, use a weighted evaluation scorecard with explicit hard safety gates for critical errors.
  4. As the design priority, set percentile latency and timeout metrics alongside completion and token cost.
  5. In the implementation sequence, measure net handling time, escalation rate and correction effort against the manual baseline.

Explanation

Correct Answers

D:

Averages conceal long-tail delays that may break user experience obligations; acceptance evidence must confirm that peak-hour monitoring reports p95/p99 latency and failed-request percentages; another obligation arises when an internal ticket agent claims to save time without counting extra reviewer effort.

E:

Value must include downstream verification and rework, not just generation time; a suitable release result is that a controlled pilot captures total labor minutes per fully resolved ticket; another obligation arises when a real-time assistant averages one second but occasionally blocks users for thirty seconds.

Incorrect Answers

A:

Satisfaction can reward fluency while overlooking harmful violations of binding eligibility criteria; the trigger would be that a loan-policy assistant is judged by user satisfaction even when it makes incorrect eligibility statements, while the required safeguard concerns an internal ticket agent claims to save time without counting extra reviewer effort; another obligation arises when a real-time assistant averages one second but occasionally blocks users for thirty seconds.

B:

A high overall score can conceal failures in infrequent but consequential cases; the design helps when a knowledge assistant is accurate on common questions but misses rare safety procedures, not the risk where a real-time assistant averages one second but occasionally blocks users for thirty seconds; another obligation arises when an internal ticket agent claims to save time without counting extra reviewer effort.

C:

Multidimensional acceptance stops inexpensive or popular outputs from hiding severe failures; that fits cases where a stakeholder requests one universal score for a regulated assistant across all tasks, not this situation where an internal ticket agent claims to save time without counting extra reviewer effort; another obligation arises when a real-time assistant averages one second but occasionally blocks users for thirty seconds.

Question 249

A tool-using agent makes accurate decisions but sometimes performs unauthorized actions. Which response follows from the actual operational constraint?

  1. In the project scope, stratify evaluation by risk, query type and frequency instead of relying on aggregate accuracy.
  2. Within the rollout design, score citation entailment and source correctness separately from answer relevance.
  3. For the next architecture revision, track tool-policy violations and approval bypass attempts as independent safety metrics.
  4. For the current integration, set percentile latency and timeout metrics alongside completion and token cost.

Explanation

Correct Answer

C:

Answer correctness cannot substitute for limits on what external side effects the agent may produce; a representative trial should prove that adversarial action tests report zero unauthorized commits under the release gate.

Incorrect Answers

A:

A high overall score can conceal failures in infrequent but consequential cases; the alternative helps if a knowledge assistant is accurate on common questions but misses rare safety procedures, whereas the problem presented is that a tool-using agent makes accurate decisions but sometimes performs unauthorized actions.

B:

The presence of a citation does not prove that the cited text supports a specific claim; it is justified if a retrieval assistant cites a source but sometimes attributes the wrong conclusion to it, but this decision concerns a tool-using agent makes accurate decisions but sometimes performs unauthorized actions.

D:

Averages conceal long-tail delays that may break user experience obligations; it targets cases where a real-time assistant averages one second but occasionally blocks users for thirty seconds, but the requested boundary concerns a tool-using agent makes accurate decisions but sometimes performs unauthorized actions.

Question 250

A retrieval assistant cites a source but sometimes attributes the wrong conclusion to it. Which change targets the constraint rather than a proxy metric?

  1. For this workflow, score citation entailment and source correctness separately from answer relevance.
  2. For the affected service, define policy-decision correctness as a primary metric with safety-critical error categories.
  3. In the project scope, track tool-policy violations and approval bypass attempts as independent safety metrics.
  4. For the intended users, stratify evaluation by risk, query type and frequency instead of relying on aggregate accuracy.

Explanation

Correct Answer

A:

The presence of a citation does not prove that the cited text supports a specific claim; its effect can be measured when a reviewer can verify every sampled claim directly against the cited evidence span.

Incorrect Answers

B:

Satisfaction can reward fluency while overlooking harmful violations of binding eligibility criteria; this technique fits where a loan-policy assistant is judged by user satisfaction even when it makes incorrect eligibility statements, rather than the condition that a retrieval assistant cites a source but sometimes attributes the wrong conclusion to it.

C:

Answer correctness cannot substitute for limits on what external side effects the agent may produce; the trigger would be that a tool-using agent makes accurate decisions but sometimes performs unauthorized actions, while the immediate question involves a retrieval assistant cites a source but sometimes attributes the wrong conclusion to it.

D:

A high overall score can conceal failures in infrequent but consequential cases; the design helps when a knowledge assistant is accurate on common questions but misses rare safety procedures, but the present constraint is that a retrieval assistant cites a source but sometimes attributes the wrong conclusion to it.

Claude Certified Architect - Professional (CCAR-P) Practice Tests