Structured Output: Typed JSON From an LLM

A small company adds an assistant to its internal calendar. On day one, a manager types:

Set up a project review with Ravi tomorrow at 3 PM for 45 minutes.

The assistant replies:

Sure! I've noted a 45 minute project review with Ravi for tomorrow at 3 PM.
Let me know if you'd like me to add an agenda.

The manager is happy. The calendar is empty. Nobody wrote anything into it, because the calendar service does not read sentences. It needs a date, a start time, a length and a person, each in its own field, each in a format it can check. The reply had all four facts and the code could use none of them.

This note is about closing that gap. We ask the model for an object instead of a paragraph. Then we treat that object like any input from outside our system: parse it, check it, and reject it or try again when it is wrong.

Note 5 touched this idea in its section on why "reply in JSON" is not enough, and then moved on to tools. Notes 11 and 12, on retrieval augmented generation (RAG), taught the model to answer from your own documents. That answer was still a paragraph for a person. Here the reader of the answer is a program. The code is a meeting extractor in Spring Boot, Node.js and Python under code/13-meeting-extractor/.

Core Path

The safe path from model text to application data has four steps:

1. Define the fields and types your code expects.
2. Ask the provider for output constrained to that schema.
3. Parse the response into the language's normal typed object.
4. Validate what the values mean before any side effect.

The main rule is: a schema guarantees shape, not meaning. It can require date to be a string. It cannot prove that 2026-02-30 exists, that the date is in the future, that the user may book it, or that a 4,500 minute meeting is allowed. Those checks stay in ordinary application code.

Sections 1–14 explain the contract and the three validation gates. Sections 15–19 cover dates, missing values, defaults and retries. Sections 20–26 are deeper production detail: tool calling, the three language versions, evaluation, and when structured output is the wrong choice.

Reference: terms used in this chapter

Return to this table as each term appears; it is a lookup, not a prerequisite.

1. Who Reads the Answer?

Every LLM answer has a reader. Up to now that reader was a person: a customer in a support chat, a developer asking about a bug.

People are very good at reading loose text. Give them "tomorrow at 3 PM with Ravi, 45 minutes" and they pick out four facts without effort. They ignore "Sure!" and they know that "3 PM" and "15:00" are the same time.

Code has none of that skill. To code, the whole reply is one string.

Same answer, two readers. The top path works. The bottom one is where most real GenAI features live: the model sits in the middle of a pipeline, and the next step is code.

2. What the Code Wants to Do

Once the meeting is data, normal programming takes over. The calendar code wants to write things like this:

if (meeting.durationMinutes() > 60) {
    warnAboutLongMeeting(meeting);
}

if (calendar.isBusy(meeting.attendee(), meeting.date(), meeting.time())) {
    suggestAnotherSlot(meeting);
}

calendar.book(meeting);

None of that works on a sentence. You cannot compare "45 minutes" with 60. You cannot look up "tomorrow" in a calendar table.

You could try to pull the facts out of the reply with string tricks. Search for "PM", grab the number before it, look for "with" and take the next word. That works for the first ten test messages and breaks on the eleventh: "set it for half past three", "with Ravi and Meera", "Ravi wants 3 PM but make it 4". Note 5 showed the same trap with if (message.contains("add")).

3. Human-Readable vs Machine-Usable

Here is the same meeting written twice:

Sure! I've noted a 45 minute project review with Ravi for tomorrow at 3 PM.

{
  "title": "Project review",
  "attendee": "Ravi",
  "date": "2026-10-10",
  "time": "15:00",
  "durationMinutes": 45
}

The object has the same facts. What changed is that each fact sits in a named slot, in a known format. That is what code needs.

LLMs naturally produce language. Applications naturally work with fields, numbers, dates and flags. We need a bridge between the two. That bridge is structured output.

4. Structured Output in One Line

Structured output means: you define the shape of the answer before the model says anything, and the model fills in that shape.

You already do this with every REST API. A GET /users/42 endpoint does not return "The user is Ravi, his email is ravi@example.com". It returns:

{ "id": 42, "name": "Ravi", "email": "ravi@example.com" }

The frontend and the backend agreed on that shape. Structured output is the same agreement, but the other side is a model.

In both boxes, one side publishes a shape and the other side fills it. The difference is trust. You wrote the backend, so you trust its JSON. You did not write the model, and it chooses each word from probabilities, so it can get things wrong. So on the LLM side you also check what comes back. Most of this note is about that check.

5. It Is Not Really About JSON

People often say "structured output means making the LLM return JSON". That is close, but it points at the wrong thing.

JSON is how the answer travels. What your code wants at the end is an object in its own language: a Java record, a Python class, a JavaScript object with known fields.

The JSON in the middle is a step, not the goal. What you care about is that meeting.date() returns a real date.

This changes how you judge a solution. "The model returned valid JSON" is not success. "My code received a MeetingDetails it can trust" is success.

6. What a Schema Is: a Form With Fixed Boxes

A schema is a description of what the data must look like. Which fields exist, what type each one is, and which ones must be present.

Think of two ways to collect details from a visitor at an office reception.

The first way is a blank sheet of paper. "Please write who you are and why you are here." You get handwriting in every direction. Some people forget their phone number. One person writes a poem.

The second way is a printed form:

Name:            ____________
Meeting with:    ____________
Date:            __ / __ / ____
Time:            __ : __

The visitor still chooses the values. But the boxes are fixed. Nobody can add a box, skip the structure, or write the date as "next week sometime".

Structured output is the printed form. We define the boxes. The LLM fills in the values.

Here is our form as a JSON Schema. You will rarely write this by hand, because each stack can generate it from a class. It helps to see it once:

{
  "type": "object",
  "properties": {
    "title":           { "type": "string", "description": "A short title of 2 to 6 words..." },
    "attendee":        { "type": "string", "description": "The person to meet, as the user wrote it..." },
    "date":            { "type": "string", "description": "Meeting date as yyyy-MM-dd..." },
    "time":            { "type": "string", "description": "Start time in 24 hour HH:mm..." },
    "durationMinutes": { "type": "integer", "description": "Length in minutes. 0 if the user did not say how long." }
  },
  "required": ["title", "attendee", "date", "time", "durationMinutes"],
  "additionalProperties": false
}

Read it top to bottom. It is an object. It has exactly five properties. Four are strings, one is a whole number. All five must be there. additionalProperties: false means nothing else may appear: no notes field, no meetingTime with a typo.

7. Four Ways to Get JSON Out of a Model

There is more than one way to ask for structure, and they are not equally safe. From weakest to strongest:

Read it from the bottom up. Each step removes one kind of failure. Only the top step moves the shape from "please" to "must".

Way 3 is worth a closer look, because it is what Spring AI's .entity() does when you do not ask for more. It adds text like this to your prompt:

Your response should be in JSON format.
Do not include any explanations, only provide a RFC8259 compliant JSON response
following this format without deviation.
Do not include markdown code blocks in your response.
Remove the ```json markdown from the output.
Here is the JSON Schema instance your output must adhere to: ...

Then it strips code fences (the triple backtick marks) and stray whitespace from the reply before parsing it. That clean-up step exists because models do add fences. It works well with good models. It is still a request. Section 23 shows the one line that switches Spring AI to way 4.

8. Why "Reply in JSON" Breaks

For a demo, way 1 is fine. Run it on 10,000 real messages and you meet the rare, odd cases. Here are failures you will see from prompt-only JSON, each one from a model that was told "reply only with JSON":

Sure! Here is the meeting:
{"title": "Project review", ...}

```json
{"title": "Project review", ...}
```

{"meetingTitle": "Project review", "meetingTime": "3 PM", "with": "Ravi"}

{"title": "Project review", "attendee": "Ravi", "durationMinutes": "45 minutes"}

{"title": "Project review", "attendee": "Ravi", "date": "2026-10-10", "ti

In order: chatty text before the JSON, a markdown fence, made-up key names, a number written as a string, and an answer cut off halfway because it hit the output token limit (the most text the model may write in one answer).

A person shrugs at all five. A JSON parser fails on the first, second and fifth. Your code fails on the third and fourth even after parsing. In the third, meeting.durationMinutes is undefined because the key is missing. In the fourth, it is the string "45 minutes", not a number.

How often? It depends on the model and the prompt. With a small model and prompt-only JSON, something like 1 to 5 bad answers per 100 is a rough guess (an illustrative figure, not a measured one). That sounds small. At 20,000 meeting requests a day it is 200 to 1,000 broken requests, every day, each one a user who typed a clear sentence and got an error.

A prompt instruction is a request. A schema sent as the response format is a contract. Production code needs the contract.

9. How Strict Mode Works: Constrained Decoding

From note 2 you know how a model writes. At each step it gives a score (the logits) to every token it knows. A token is a small piece of text. It turns the scores into probabilities with softmax, and picks one.

Strict mode adds one step in the middle. Before picking, the API removes every token that would break the schema at this point in the answer. The model can only choose from what is left.

Say the model has written {"title": "Project review", "attendee": "Ravi", " so far. The schema says the next key must be one of the remaining required keys. A token like meetingTime is not allowed here, so its probability is set to zero before the pick.

That loop runs for every token of the answer. This is called constrained decoding. The model still decides the values. It cannot decide the shape.

When the model finishes its answer, does not refuse, and supports your schema, you get three guarantees:

• The output should parse as JSON. A cut-off or refused answer still needs error handling.
• The keys match the schema. No invented names, and no required field is left out.
• The types match the schema. durationMinutes is an integer, not "45 minutes". But the integer can still be the wrong value.

These guarantees hold only for the shapes your provider and model support. They do not mean the service is always up, that the values are right for your business, or that every JSON Schema feature works.

There are costs. The provider turns your schema into a grammar (a set of rules for which text is allowed) before it can use it. The first request with a new schema can take noticeably longer while that happens, and later requests reuse the prepared grammar. So keep the schema fixed. Building a slightly different schema on every request throws that work away.

10. The Rules Strict Mode Puts on Your Schema

Strict mode does not accept every JSON Schema. The exact list changes, so check your provider's documentation, but the rules that shape this note are:

The first rule raises a real design question. If every field must be in every answer, how does the model say "the user never told me the time"?

There are two common answers:

The code for this note uses sentinels: an empty string for text and 0 for the duration. The reason is practical. Spring AI's schema generator leaves a @Nullable field out of required, and strict mode wants every field in required. Sentinels give all three versions (Spring, Node.js, Python) one identical contract. In a Node-only or Python-only project, nullable fields are the nicer choice.

Whichever you pick, the first thing your code does is turn the sentinel back into a proper "missing". Section 16 shows why that matters.

11. The Schema Is Also a Prompt

Look at the description on each field in section 6. Those are not comments for developers. They are sent to the model inside the schema, and the model reads them while filling each field.

public record MeetingDetails(
        @JsonPropertyDescription("A short title of 2 to 6 words. Make one from the message if the user gave none.")
        String title,

        @JsonPropertyDescription("The person to meet, as the user wrote it. Empty string if no person is named.")
        String attendee,

        @JsonPropertyDescription("Meeting date as yyyy-MM-dd. Empty string if the user gave no date.")
        String date,

        @JsonPropertyDescription("Start time in 24 hour HH:mm, for example 15:00. Empty string if the user gave no time.")
        String time,

        @JsonPropertyDescription("Length in minutes. 0 if the user did not say how long.")
        int durationMinutes) {
}

export const MeetingDetails = z.object({
  title: z.string().describe("A short title of 2 to 6 words. Make one from the message if the user gave none."),
  attendee: z.string().describe("The person to meet, as the user wrote it. Empty string if no person is named."),
  date: z.string().describe("Meeting date as yyyy-MM-dd. Empty string if the user gave no date."),
  time: z.string().describe("Start time in 24 hour HH:mm, for example 15:00. Empty string if the user gave no time."),
  durationMinutes: z.number().int().describe("Length in minutes. 0 if the user did not say how long."),
});

class MeetingDetails(BaseModel):
    title: str = Field(
        description="A short title of 2 to 6 words. Make one from the message if the user gave none."
    )
    attendee: str = Field(
        description="The person to meet, as the user wrote it. Empty string if no person is named."
    )
    date: str = Field(description="Meeting date as yyyy-MM-dd. Empty string if the user gave no date.")
    time: str = Field(
        description="Start time in 24 hour HH:mm, for example 15:00. Empty string if the user gave no time."
    )
    # The alias keeps the JSON name the same as in Java and Node.
    duration_minutes: int = Field(
        alias="durationMinutes",
        description="Length in minutes. 0 if the user did not say how long.",
    )

Three languages, one schema. Each library reads the class, writes the JSON Schema from section 6, and copies the descriptions in. In Python, the alias lets the field duration_minutes appear as durationMinutes in the JSON.

A few habits that pay off:

• Put the format in the description: "yyyy-MM-dd", "24 hour HH:mm". The type alone only says "string".
• Say what "not given" looks like, field by field.
• Give one example where the format is easy to get wrong: "for example 15:00".
• Field names matter too. durationMinutes beats duration, because the name carries the unit.

12. Prompt Instruction vs Contract

It helps to put the two approaches side by side.

Look at the last row. It is the same on both sides, and it is the reason this note does not end here.

13. Shape Is Not Meaning

Strict mode promises that date is a string. It does not promise that the string is a date. All of these match the schema perfectly:

{ "title": "Project review", "attendee": "Ravi", "date": "tomorrow",   "time": "15:00", "durationMinutes": 45 }
{ "title": "Project review", "attendee": "Ravi", "date": "2024-10-10", "time": "15:00", "durationMinutes": 45 }
{ "title": "Project review", "attendee": "Ravi", "date": "2026-02-30", "time": "3 PM",  "durationMinutes": 45 }
{ "title": "Project review", "attendee": "Ravi", "date": "2026-10-10", "time": "25:00", "durationMinutes": 4500 }

The first one copied the user's word instead of turning it into a date. The second has the right day and month in the wrong year. This is the most common date bug in LLM features. The model has no clock, so it falls back to a year that felt current in its training data. The third has a day that does not exist, and a time in the wrong format. The fourth is 25 o'clock and a meeting of 4,500 minutes, over three days.

Every one of these would pass JSON.parse and the schema, reach your calendar code, and do damage. So structured output needs a second layer.

The schema checks the shape. Your code checks the meaning.

Some of these could be pushed into the schema. A regex pattern (a text-matching rule) on date would reject "tomorrow". An enum (a fixed list of allowed values) works for lists such as "priority": ["LOW", "MEDIUM", "HIGH"]. But "not in the past" depends on today, "a real calendar day" needs a calendar, and "no more than 8 hours" is a business rule that will change. Those belong in code, where they can be tested.

14. The Three Gates

Every answer goes through three gates, in order:

Strict mode moves most failures out of gates 1 and 2. Gate 3 stays exactly as busy as before. That is why the rules file is the biggest file in each version of the code.

The rules in our extractor:

The messages are written for the model as much as for logs. Section 17 explains why.

15. Today Is Your Job

The model does not know what day it is. It has no clock and no calendar. If you do not tell it, "tomorrow" gets resolved against a guess.

So the system prompt starts with the facts the model cannot know:

Today is 2026-10-09, a Friday. The time zone is Asia/Kolkata.

The weekday is there for phrases like "next Monday". Without it, the model has to work out the weekday of a date on its own, which is exactly the kind of arithmetic it gets wrong.

The time zone is the part people forget. Your server probably runs in UTC. Your users do not. Take a user in New York on a Friday evening:

From 8 PM to midnight in New York (7 PM in winter, when the offset is five hours), the UTC date is already tomorrow. If "today" comes from the server clock, every "tomorrow" typed in those hours lands a day late. Evenings are exactly when people plan the next day. India has the same problem the other way round: from midnight to 5:30 AM IST, the UTC date is still yesterday.

The fix is to compute today in the user's zone. Our code reads the zone from APP_TIME_ZONE. A real product reads it from the user's profile. Each version also turns the zone name into a zone object once, up front, so a typo such as "Asia/Kolkatta" fails with a clear error right away. Spring and Node fail at start-up. Python builds its extractor on the first request, so it fails there.

@Bean
Clock meetingClock(@Value("${meeting.time-zone}") String timeZone) {
    return Clock.system(ZoneId.of(timeZone));
}

/**
 * Today's date as yyyy-MM-dd in the given IANA time zone.
 * The en-CA locale happens to format dates exactly as yyyy-MM-dd.
 */
export function todayIn(timeZone, now) {
  return new Intl.DateTimeFormat("en-CA", { timeZone, year: "numeric", month: "2-digit", day: "2-digit" }).format(now);
}

moment = self._now().astimezone(self._zone)
today = moment.date()
instructions = system_prompt(today.isoformat(), moment.strftime("%A"), self._zone_name)

The same today goes to two places: into the prompt, and into the rule that rejects past dates. If those two disagreed, the rules would reject answers the model got right.

16. Missing Is Not Wrong

Look at this message:

Set up a call with Priya next Monday about the Q3 budget.

No time. No length. That is not the model's fault, and it is not an error. The user simply did not say.

There are two very different things that can be "not right" in an answer, and they need different handling:

This split is the most useful idea in the whole extractor. Mixing the two gives you one of two bad products. Either the model is pushed to fill every field, so it guesses "10:00" for a time nobody said. Or every missing field is an error, so a perfectly clear "call with Priya next Monday" fails.

The prompt backs this up with one line: "Never guess a date, a time or a person that is not in the message. Use an empty string instead." Then the rules turn each empty string into a name in missingFields.

Defaults are a third case. If the user gives no length, a 30 minute meeting is a fine guess. That is a business decision, so it lives in code:

static final int DEFAULT_DURATION_MINUTES = 30;

Not in the prompt. A default in the prompt is a default you cannot test, and one that two prompts can disagree on. A constant in code is the same on every request.

17. Retry With Feedback

When gate 3 fails, the cheapest fix is often to ask again. But a plain retry with the same input tends to get a similar answer. The better move is to show the model what it wrote and say exactly what was wrong.

The second request carries the whole short conversation: the original message, the model's own answer as an assistant turn (a message marked as coming from the model), and a new user turn that lists the problems:

Your last answer had these problems:
- date 2024-10-10 is before today (2026-10-09)
Return the corrected meeting. Keep every value that was already right.

"Keep every value that was already right" stops the model from changing the attendee while fixing the date.

The problem messages from section 14 were written for this moment. "got "3 PM"" tells the model what it wrote. "must be 24 hour HH:mm" tells it what to write instead. A message like "invalid input" would give it nothing to act on.

How many retries? Our code allows two attempts in total: the first try and one retry.

Why stop at two? Each attempt is a full paid model call. And in practice, if the model could not fix a problem after being told exactly what it was, a third try rarely helps. The problem is usually in the input: "lunch on 30th February" has no right answer. Better to stop and tell the caller.

18. What a Retry Costs

Some rough, illustrative numbers, so the trade-off is concrete. The prices and the retry rate below are example figures, not quotes. Check current prices before you rely on them.

One attempt for our extractor is about 450 input tokens (system prompt, schema with descriptions, the user's sentence) and around 50 tokens of JSON out. A reasoning model such as gpt-5-mini also spends hidden reasoning tokens (thinking you pay for but never see) before it answers, often a few hundred. Call it 400 output tokens in total.

Assume (illustrative prices) $0.25 per million input tokens and $2 per million output tokens:

input:   450 x 0.25 / 1,000,000 = $0.00011
output:  400 x 2.00 / 1,000,000 = $0.00080
one attempt                     = about $0.0009, under 10 paise

A retry costs a little more than the first try, because the second request also carries the first answer and the problem list. Even so, the retry line barely moves the bill. What it does move is latency (waiting time). A reasoning model takes roughly 1 to 4 seconds for this call (a rough figure), so a request that needs a retry takes twice as long. Track the share of requests with attempts: 2. If it climbs above a few percent, fix the prompt or the descriptions instead of paying for retries forever.

19. When Not to Retry

Not every failure deserves another model call.

The provider errors deserve a note. The OpenAI SDKs and Spring AI both retry connection errors and rate limits on their own, with growing waits between tries. If your loop also retried them, one bad minute at the provider would turn into many times the calls. Keep the two kinds of retry separate: the SDK handles "the provider did not answer", your loop handles "the answer was wrong".

One SDK retry can still double the bill: a client timeout. If you set the timeout below the model's real latency, the client gives up, the SDK sends the request again, and the provider bills both calls, because the first one finished on its side. Reasoning models make this easy to hit, since a slow call can take 10 seconds or more. Set the timeout above your slowest normal call, not above your average one.

Retrying is also only safe here because extraction has no side effects. Nothing gets booked until the caller decides. If the model call itself booked something, a retry could book it twice. Keep "read the request" and "do the action" apart, and the read side can be retried freely.

20. Structured Output vs Tool Calling

Both produce JSON that matches a schema, so people mix them up. They solve different problems.

Structured output is about the format of the model's final answer. The object is the result. Nothing runs.

Tool calling (note 5) is the model asking your application to run a function. The JSON is the input to that function. Something runs, and the result usually goes back to the model.

Under the hood, strict tool arguments use the same constrained decoding as strict structured output. Tool arguments are structured output aimed at a function. The difference is who is in control. With structured output, your code always gets the object and decides what happens next. With tools, the model decides whether to call anything at all.

For our meeting example:

• Structured output: "Project review with Ravi tomorrow at 3 PM" becomes a MeetingDetails. Nothing is scheduled.
• Tool calling: the model calls bookMeeting(...) and a calendar entry is created.

When in doubt, start with structured output. It is one call, it is easy to test, and your code stays in charge. Move to tools when the model needs to choose between several actions or needs live data mid-answer.

21. Where This Leads

After extraction, one piece is still missing. We have the meeting as data, but no calendar event exists.

The structured output tells the application what the user wants. Something else has to actually do it. That is where tools and external integrations come back. Notes 14 and 15 introduce MCP (Model Context Protocol), a standard way to plug tools such as a calendar into a model without writing a custom integration for each one.

Keep three ideas apart:

22. Building It: the Meeting Extractor

Time to put it together. The service has one endpoint:

Why 422 and not 502 when the rules keep failing? Because the provider did answer. The trouble is usually the request itself, such as an impossible date, and the caller can show the problem to the user. A 502 would suggest "try again later", which would not help.

A successful request moves through the layers like this:

The HTTP layer knows nothing about models. The extractor knows nothing about HTTP. The rules know nothing about either: they take a MeetingDetails and a date, and return a result. That last part is the one to test hardest, and it needs no API key at all.

23. The Code, Side by Side

The full projects are on the code page:

Browse the code for this note, or open src/FDE/code/13-meeting-extractor/ on GitHub. The README there has the run commands, the settings and sample requests.

23.1 Calling the model with a schema

This is the heart of each version: one model call, with the schema sent as the strict response format.

private MeetingDetails askModel(String systemPrompt, List<Message> conversation) {
    try {
        return chatClient.prompt()
                .system(systemPrompt)
                .messages(conversation)
                // Send the schema as the provider's strict response format, instead of
                // pasting it into the prompt and hoping. Spring AI builds the schema
                // from the MeetingDetails record passed to entity().
                .advisors(AdvisorParams.ENABLE_NATIVE_STRUCTURED_OUTPUT)
                .call()
                .entity(MeetingDetails.class);
    } catch (JacksonException e) {
        return null;
    }
}

// Built once: the schema never changes between requests.
const MEETING_FORMAT = zodTextFormat(MeetingDetails, "meeting_details");

async function askModel(instructions, input) {
  try {
    const response = await client.responses.parse({
      model,
      instructions,
      input,
      // Send the schema as the provider's strict response format, instead of
      // pasting it into the prompt and hoping.
      text: { format: MEETING_FORMAT },
    });
    // null when the model refused, or when there was no text to parse.
    return response.output_parsed ?? null;
  } catch (error) {
    // SyntaxError: cut-off JSON. ZodError: JSON that does not match the schema.
    if (error instanceof SyntaxError || error instanceof ZodError) return null;
    throw error;
  }
}

async def _ask_model(self, instructions: str, input_items: list[dict[str, str]]) -> MeetingDetails | None:
    try:
        response = await self._client.responses.parse(
            model=self._model,
            instructions=instructions,
            input=input_items,
            # Send the schema as the provider's strict response format,
            # instead of pasting it into the prompt and hoping.
            text_format=MeetingDetails,
        )
    except ValidationError:
        # Cut-off JSON, or JSON that does not match the schema.
        return None
    # None when the model refused, or when there was no text to parse.
    return response.output_parsed

The one line to notice in Spring is .advisors(AdvisorParams.ENABLE_NATIVE_STRUCTURED_OUTPUT). Without it, .entity() pastes the schema into the prompt (way 3 from section 7). With it, Spring AI sends the schema as the provider's strict response format (way 4). Node and Python get way 4 from responses.parse with a schema.

All three return "nothing usable" in the same two cases, a parse failure and a refusal, and let every other exception through. A network error is not a bad answer, and must not be retried as one.

23.2 The attempt loop

List<String> problems = List.of();
for (int attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) {
    MeetingDetails details = askModel(systemPrompt, conversation);

    if (details == null) {
        problems = List.of(MeetingPrompts.UNREADABLE_ANSWER);
    } else {
        MeetingRules.Check check = rules.check(details, today);
        if (check.problems().isEmpty()) {
            return MeetingResult.from(check, attempt);
        }
        problems = check.problems();
        // Show the model its own answer, so the retry is a correction, not a fresh guess.
        conversation.add(new AssistantMessage(jsonMapper.writeValueAsString(details)));
    }
    conversation.add(new UserMessage(MeetingPrompts.retry(problems)));
}
throw new ExtractionFailedException(problems, MAX_ATTEMPTS);

let problems = [];
for (let attempt = 1; attempt <= MAX_ATTEMPTS; attempt += 1) {
  const details = await askModel(instructions, input);

  if (details === null) {
    problems = [UNREADABLE_ANSWER];
  } else {
    const check = checkMeeting(details, today);
    if (check.problems.length === 0) {
      const status = check.missingFields.length === 0 ? "READY" : "NEEDS_INFO";
      return { status, meeting: check.meeting, missingFields: check.missingFields, attempts: attempt };
    }
    problems = check.problems;
    // Show the model its own answer, so the retry is a correction, not a fresh guess.
    input.push({ role: "assistant", content: JSON.stringify(details) });
  }
  input.push({ role: "user", content: retryMessage(problems) });
}
throw new ExtractionFailedError(problems, MAX_ATTEMPTS);

problems: list[str] = []
for attempt in range(1, MAX_ATTEMPTS + 1):
    details = await self._ask_model(instructions, input_items)

    if details is None:
        problems = [UNREADABLE_ANSWER]
    else:
        check = check_meeting(details, today)
        if not check.problems:
            status = "READY" if not check.missing_fields else "NEEDS_INFO"
            return {
                "status": status,
                "meeting": check.meeting.to_json(),
                "missingFields": check.missing_fields,
                "attempts": attempt,
            }
        problems = check.problems
        # Show the model its own answer, so the retry is a correction, not a fresh guess.
        input_items.append({"role": "assistant", "content": details.model_dump_json(by_alias=True)})
    input_items.append({"role": "user", "content": retry_message(problems)})

raise ExtractionFailedError(problems, MAX_ATTEMPTS)

The same twenty lines in three languages. Each round asks the model and checks the answer. Only a broken rule or an unreadable answer leads to another round. Missing fields return at once as NEEDS_INFO.

One Spring detail is easy to miss. The user's message goes in as a UserMessage object, not through .user(message). ChatClient treats text passed to .user() and .system() as a template, so a message such as "{urgent} sync with Ravi" would be read as a placeholder. A message object is sent as it is.

23.3 One rule in three languages

The date rule shows the care each stack needs:

LocalDate date;
try {
    // ISO_LOCAL_DATE is strict: "2026-02-30" and "10/10/2026" both fail.
    date = LocalDate.parse(value, DateTimeFormatter.ISO_LOCAL_DATE);
} catch (DateTimeParseException e) {
    problems.add("date must be a real calendar date in yyyy-MM-dd, got \"" + value + "\"");
    return null;
}
// The classic model mistake: the right day and month in its training year.
if (date.isBefore(today)) {
    problems.add("date " + date + " is before today (" + today + ")");
    return null;
}

function isRealDate(value) {
  if (!DATE_PATTERN.test(value)) return false;
  const [year, month, day] = value.split("-").map(Number);
  const date = new Date(Date.UTC(year, month - 1, day));
  // Date.UTC quietly turns 2026-02-30 into March 2, so a real date must survive the round trip.
  return date.getUTCFullYear() === year && date.getUTCMonth() === month - 1 && date.getUTCDate() === day;
}

if not DATE_PATTERN.fullmatch(value):
    problems.append(not_a_date)
    return None
try:
    parsed = dt.date.fromisoformat(value)
except ValueError:  # the right shape but not a real day, such as 2026-02-30
    problems.append(not_a_date)
    return None

Each language has its own trap. Java's strict ISO parser is the easy case. JavaScript's Date never fails, it rolls February 30 into March 2, so the code checks that the parts come back unchanged. Python's fromisoformat accepts more than we want since Python 3.11, including 20261010, so a pattern runs first.

23.4 How the three map

23.5 Where to look in the code

23.6 What this demo leaves out on purpose

24. What Breaks in Production

Strict mode removes the loud failures, the ones that crash a parser. These are the quiet ones that are left.

An evaluation set is a fixed list of real messages, each with the object you expect. Collect 200 to 500 messages (cleaned of personal data), write down the expected object for each, and run the extractor over them on every change to the prompt, the schema or the model. Count exact matches per field. "Date correct in 97.5% of cases" is a number you can defend. "It looked fine when I tried it" is not.

25. When Not to Use Structured Output

Structured output is a tool for when code reads the answer. It is the wrong tool in a few common cases.

• A person reads the answer. A support reply or a summary should be text. Forcing a chat reply into JSON only to print one field of it makes the answer worse and the code longer.
• A form would do. If your UI already has a date picker and a time picker, do not send the user's clicks through a model. An LLM earns its place when the input is free text in many shapes.
• Long creative writing. A 2,000 word report in a single "body" string gains nothing from a schema, and strict mode makes streaming it harder.
• The data is the whole document. Pulling 300 line items out of a 40 page invoice in one call is a document processing job. Split it into pages or sections, extract each, and join the results in code.

26. The Full Mental Model

We started with a polite reply that the calendar could not read. Here is the whole path from sentence to safe object:

Three layers keep the model honest. The schema fixes the shape. The rules check the meaning. The split between "missing" and "wrong" decides who gets asked again: the user or the model.

In one line:

schema (shape) + strict mode (enforced) + rules (meaning) + feedback retry = an object you can trust

Interview Questions

"What is structured output, and why is asking for JSON in the prompt not enough?"

Structured output means the model's answer follows a schema your code defined in advance, so code can read it field by field. Asking for JSON in the prompt is only a request: the model can add chatty text, a code fence, wrong key names or wrong types, and at scale a few percent of answers break the parser.

"How does strict mode guarantee that the output matches the schema?"

The provider turns the schema into a grammar and uses constrained decoding. At every step it removes tokens that would break the schema before the next token is picked, so the model chooses the values but cannot change the shape.

"If strict mode guarantees the schema, why do you still validate?"

The schema checks shape, not meaning. A string field can hold "tomorrow", a date from last year or 2026-02-30, and an integer can be 4500 minutes. Business rules such as "not in the past" depend on today and on company policy, so they live in code where they can be tested.

"Strict mode requires every field. How do you represent a value the user never gave?"

Either a nullable type such as string or null, or a sentinel such as an empty string or 0. Either way, the first thing the code does is turn that back into an explicit "missing", so it is never mistaken for a real value.

"What is the difference between a missing field and a wrong field, and why does it matter?"

A missing field is the user's gap: they did not say it, so you ask the user and never let the model guess. A wrong field is the model's mistake, such as 3 PM instead of 15:00, so you retry the model and tell it exactly what was wrong. Mixing them leads to either invented values or needless errors.

"How would you design retries for an LLM extraction endpoint?"

Retry only answers that failed to parse or broke a rule, send the model its own answer plus a precise list of problems, and cap it at one or two retries. Leave network errors and rate limits to the SDK's own backoff, and return a clear 422 with the problems when the cap is reached.

"When would you pick structured output over tool calling?"

Use structured output when you always want the same kind of object back and your code decides what to do with it, such as extraction or classification. Use tool calling when the model must choose between actions or fetch live data in the middle of answering.

"Why does the current date belong in the prompt, and what goes wrong with time zones?"

The model has no clock, so without today's date it resolves "tomorrow" against a guess, often its training year. Today must also be computed in the user's time zone: a server in UTC is already on the next day for a New York user every evening after 8 PM (7 PM in winter), so every "tomorrow" typed then lands a day late.

Key Takeaways

• When code reads the answer, a paragraph is useless. Ask for an object.
• Structured output is a contract: you define the shape, the model fills the values. JSON is only how it travels.
• A schema is a printed form. Fixed boxes, names, types, and nothing extra.
• "Reply in JSON" is a request. Strict structured output is enforced by constrained decoding, token by token.
• Strict mode wants every field in every answer, so decide how "not given" looks: nullable, or a sentinel such as "" and 0.
• Field descriptions are prompts. Put formats and "what if missing" in them.
• The schema checks shape. Your code checks meaning: real dates, not in the past, sane ranges.
• The model has no clock. Send today, the weekday and the user's time zone, and use the same today in your rules.
• Missing is not wrong. Missing means ask the user. Wrong means retry the model with feedback.
• Keep defaults such as a 30 minute meeting in code, not in the prompt.
• Retry once with the model's own answer and a precise problem list. Leave network retries to the SDK.
• Structured output reads the request. Tool calling and MCP do the action. Keeping them apart makes retries safe.