A model can return syntactically perfect JSON and still give an incorrect business answer. It can also extract the right facts from a document in prose that a downstream program cannot safely parse. Reliable systems must address both problems: making the output obey a defined structure and checking whether its contents are supported by evidence.<\/p>\n
The distinction is central to the Prompt Engineering & Structured Output domain of Claude Certified Architect \u2013 Foundations, commonly called CCA-F and formally coded CCAR-F. The domain represents 20 percent of Anthropic’s published blueprint. The practical challenge is to specify a task clearly, constrain what software needs to consume, and know when instructions alone are insufficient.<\/p>\n
A weak extraction prompt says \u201cread this contract and summarize it.\u201d A production-grade specification identifies which clauses matter, how citations should be recorded, which dates or parties must be extracted, and what the system should do when information is absent. That difference reduces guesswork without pretending that ambiguity has disappeared.<\/p>\n
Separate permanent instructions from the material being analyzed. The system can define the model’s role, permitted tools and output requirements. A user task can specify the particular document and question. A retrieved document should be treated as evidence, not as an opportunity to rewrite the assistant’s higher-priority instructions. The clearer those boundaries are, the easier it becomes to evaluate failure.<\/p>\n
Examples help when the expected interpretation is difficult to describe abstractly. A few carefully chosen examples can show what an unresolved clause, a missing field or a conflicting date should look like in output. Examples should include genuine edge cases rather than repeating the same easy pattern. They guide the model’s interpretation; they are not a guarantee that future cases will be classified correctly.<\/p>\n
XML-style tags and other clear delimiters can make complex prompts easier to read and maintain, particularly when mixing instructions, background context and documents. The important property is unambiguous separation, not the belief that a particular tag magically prevents mistakes.<\/p>\n
For a conversational explanation, exact machine-readable formatting may be unnecessary. For an application that loads extracted fields into a database, a missing key or malformed data type can cause a failed transaction or an unintended fallback. The architecture should specify a schema and reject invalid output before it reaches downstream systems.<\/p>\n
Structured-output facilities and strict tool schemas can help constrain response shape. They solve the transport and syntax problem: for example, the application expects a list of invoices with a date, a currency and a total. But a schema cannot establish whether the amount came from the correct invoice, whether the date refers to issuance rather than payment, or whether the document has been altered.<\/p>\n
That requires a second layer of validation. Software might compare extracted totals with independently parsed line items, check dates against known billing periods, and preserve a reference to the original document segment. Where the source itself is ambiguous, the correct output may be an explicit unresolved state instead of a guessed value.<\/p>\n
Suppose an extraction returns A schema might require a string Consider two invoices with the same supplier name and similar reference numbers. A model could emit a valid JSON object that merges one invoice’s amount with the other’s date. A strict schema would accept its shape. A careful pipeline includes immutable document IDs, matched source spans, duplication checks and a rule that mixed evidence cannot be promoted to an accepted record. If a source field is missing, a nullable value with a documented reason is safer than an invented placeholder that passes validation.<\/p>\n Prompt examples should demonstrate both normal and difficult cases: absent fields, contradictory text, OCR errors and adversarial instructions embedded in the document. Their job is to define the task and response boundaries, not to prove an output correct after the fact. Treat prompt, schema and validator changes as releases with regression cases so that improving one field does not silently degrade another.<\/p>\n Real documents are imperfect. A scanned agreement may contain illegible sections. A customer email may mention two possible delivery dates. A policy update may contradict an older procedure. A useful prompt tells the assistant how to signal those conditions rather than insisting that every field must be filled.<\/p>\n For example, the data contract can distinguish Retrieved text may also include commands addressed to the assistant. A malicious supplier document could instruct the system to suppress an adverse finding, ignore prior directions or call an unrelated export tool. This is an instruction-in-data attack, not a legitimate change in business policy. A resilient design isolates the source, constrains tool access and validates any high-impact operation independently.<\/p>\n Prompt injection in LLM applications<\/a> exploits this confusion between untrusted source data and instructions. For Claude architects, the crucial lesson is that prompt wording improves behavior but does not replace trusted-system authorization.<\/p>\n A good evaluation set should deliberately include adversarial and messy examples: repeated identifiers, contradictory paragraphs, tables with wrapped lines, placeholder values and statements that attempt to redirect the assistant. These cases expose failures that polished demonstration documents will never reveal.<\/p>\n More examples do not necessarily improve a prompt. A long collection of almost identical examples may waste context and encourage the model to match superficial patterns. Instead, choose examples that cover the meaningful distinctions behind the task. For a support classifier, those might include routine inquiries, genuine safety escalations, ambiguous requests and requests that must be refused for policy reasons.<\/p>\n Each example should demonstrate both the desired output and the governing interpretation. If the model should mark a missing termination date as absent rather than infer it from a nearby date, include that edge case explicitly. If multiple invoice dates appear, show which one should populate the target field and why.<\/p>\n Examples also need maintenance. When business terminology changes, old examples may teach the wrong schema or policy. Prompt changes are application changes; they deserve version control, review and regression tests<\/a>. The set of examples that worked for one model version should not be assumed to transfer perfectly to a different version without comparison.<\/p>\n Testing must evaluate the underlying correctness as well as format compliance. Count exact extraction errors, unsupported assertions, ambiguous cases handled properly and downstream exceptions. A result can satisfy the schema and still fail the user.<\/p>\n A model may use a tool to retrieve a record and then produce an answer. The tool-call arguments<\/a> need their own validation and authorization. It is dangerous to assume that a correctly shaped final answer proves the intermediate calls were safe.<\/p>\n Consider a tool named Conversely, an application may need a stable final response even when a tool fails. Define explicit statuses such as Tool results that have source IDs, timestamps or version fields can support provenance. The model can use those identifiers when explaining an answer, while software can independently verify that they refer to real records.<\/p>\n Prompts consume tokens and can create additional latency. An application that repeats large instruction blocks and documents in every turn may become expensive before any useful tool runs. Identify reusable guidance, cache or retrieve relevant portions where supported, and avoid giving the model entire corpora when a narrowly scoped retrieval result is sufficient.<\/p>\n At the same time, aggressive trimming can remove evidence necessary for correct extraction. A document’s heading or preceding paragraph may determine which date a number belongs to. Context selection<\/a> should preserve the relationships that determine meaning, not just the string that looks like the target field.<\/p>\n Prompt versioning is operationally important. If a change improves average performance but causes new failures on rare high-risk cases, the deployment should not rely on averages alone. Track outputs for known hard examples and separate schema errors from factual errors. Prompt management for production generative AI<\/a> requires versioned changes and repeatable evaluation regardless of whether the implementation uses Claude directly or a cloud-managed platform.<\/p>\n Sometimes a deterministic parser is a better choice. If a document has a stable, formal machine-readable format, conventional software can often extract required fields more cheaply and reliably. Use a language model where interpretation of messy language is actually valuable, and surround that interpretation with checks where errors have consequences.<\/p>\n{status: \"approved\"}<\/code> in a valid object. If the workflow uses that word to trigger a payment, the system has confused model interpretation with approval authority. Only a verified approval process should authorize money movement. The schema should carry the result of that process, not allow the model to impersonate it.<\/p>\nStructured syntax cannot substitute for verified source data<\/h3>\n
invoice_id<\/code>, numeric amount<\/code>, optional due_date<\/code> and a source_span<\/code> pointing to the original document. Syntactic validation can confirm that amount<\/code> is a number and that required keys exist. It cannot prove the number came from the correct invoice, that its currency is right or that the stated due date belongs to this supplier. For critical records, follow parsing with source-grounding checks and business validation, then return an explicit unresolved state if confidence or supporting evidence is insufficient.<\/p>\nHandle missing, contradictory and adversarial source content<\/h2>\n
not_present<\/code>, ambiguous<\/code>, conflicting<\/code> and found<\/code>. That is more meaningful than using a single null for every uncertainty or inventing a value to satisfy a required field. The downstream workflow can route conflicts to human review<\/a> while allowing straightforward cases to proceed.<\/p>\nFew-shot examples need coverage, not volume<\/h2>\n
Tool use is a separate contract from response formatting<\/h2>\n
query_accounts<\/code> that accepts an arbitrary free-text query. Even if the assistant’s final JSON is valid, a poor query could expose unrelated customer records. Restrict the tool to verified identifiers and server-side access checks. The final response can then include only the fields actually needed by the caller.<\/p>\ncomplete<\/code>, partial<\/code> and requires_review<\/code>, and include machine-readable error details that support handling. A forced \u201csuccess\u201d state merely to keep the schema neat hides reality. The application should decide which errors permit retry, which need user clarification and which require stopping.<\/p>\nCost, latency and consistency affect prompt architecture<\/h2>\n
An extraction pipeline that can be reviewed<\/h2>\n