Structured Output: Typed JSON From an LLM

The main idea of this chapter is simple:

Don't just ask an LLM to give you an answer. Ask it to return data in a structure your code can understand, then validate that data before using it.

We'll use the chapter's meeting-scheduling example and connect it to practical backend development.

1. The problem: humans understand sentences, but code needs fields

Imagine a user sends this message to your AI assistant:

Set up a project review with Ravi tomorrow at 3 PM for 45 minutes.

The LLM replies:

Sure! I've noted a 45-minute project review with Ravi for tomorrow at 3 PM.

Sounds correct to a person. But has your calendar database received anything? No. The reply is only a sentence. Your application needs separate fields to work with.

Step 1: Human-readable response

“Project review with Ravi tomorrow at 3 PM for 45 minutes.”

Step 2: Machine-readable data

{
  "title": "Project review",
  "attendee": "Ravi",
  "date": "2026-10-24",
  "time": "15:00",
  "durationMinutes": 45
}

Step 3: Normal application code

Validate the date, check calendar availability, and only then create the meeting.

The date in this example assumes the user means tomorrow, October 24, 2026, based on the current date. A real application should calculate it in the user's time zone.

The chapter calls this structured output: defining the shape of the answer before the model fills in the values.

2. What is a schema?

Think of a schema as a form with fixed boxes.

At an office reception, you might have a form with these fields:

• Name
• Person to meet
• Date
• Time

The visitor fills in the values, but cannot randomly change the form's structure.

A JSON Schema does the same thing for an LLM. For our meeting extractor, the contract defines five fields:

The schema can also require all five fields and prohibit additional fields. That prevents mistakes such as returning meetingTime when the application expects time.

Memory trick: A schema is the form; the model fills the form.

3. Four ways to get JSON from an LLM

This is an important distinction for building production AI applications.

Why isn't “Reply only in JSON” enough?

You might ask the model to return:

{
  "attendee": "Ravi",
  "durationMinutes": 45
}

But the model could return:

{
  "attendee": "Ravi",
  "durationMinutes": "45 minutes"
}

This is valid JSON, but durationMinutes is a string instead of an integer.

Or it might return:

{
  "meetingTitle": "Project review",
  "meetingTime": "3 PM"
}

The keys don't match your application's expected schema.

Remember: Valid JSON does not necessarily mean correct application data.

How strict mode works

Strict structured output uses a technique called constrained decoding.

Normally, an LLM scores possible next tokens and chooses one. With constrained decoding, the provider prevents tokens that would violate the supported schema from being selected.

Simplified example

The schema expects durationMinutes to be an integer.

Strict mode controls the shape, not whether the chosen values are factually correct. It also depends on provider and model support.

4. The most important rule: shape is not meaning

This is probably the most important concept in the chapter.

Suppose the schema expects:

{
  "date": "string",
  "time": "string",
  "durationMinutes": 45
}

The schema might accept all these values because they have the correct data types:

A schema can check that date is a string. It cannot, by that fact alone, prove that the date exists, is in the future, or is permitted by your business rules.

Think of it like a form

Imagine a form where the Date field accepts any text.

Someone enters 31 February.

• The form is filled in.
• The value is still wrong.

Your backend must validate the actual value.

Memory trick: Schema = shape. Business rules = sense.

5. The three validation gates

Before your application uses the LLM's response, it should pass three checks.

1. Parsing: Can the response be read as JSON? Reject malformed or truncated JSON and handle refusals appropriately.
2. Schema validation: Are the required keys and data types correct? For example, duration must be an integer.
3. Business-rule validation: Do the values make sense for your application? Check real dates, permitted duration, permissions, and other relevant rules.

Only after the checks pass should the application treat the result as ready for the next step. Strict mode makes the first two checks much more reliable, but the third remains your application's responsibility.

Example business rules

For the meeting extractor in this chapter:

• Date must be a real date in yyyy-MM-dd format.
• Date must not be before today.
• Time must follow 24-hour HH:mm format.
• Duration must be between 1 and 480 minutes, or 0 if not supplied.
• Title and attendee must not exceed 100 characters.

These are rules from the example application, not universal rules for every calendar system.

6. What if the user leaves a field out?

Consider this message:

Set up a call with Priya next Monday about the Q3 budget.

The user has provided the person, date intent and subject, but hasn't given a time or duration.

Should the LLM invent 10 AM?

No. This is where we need to distinguish between missing and wrong.

How do you represent missing values?

The chapter describes two approaches.

Option A: Nullable fields

{
  "time": null
}

This clearly represents that no time was provided.

Option B: Sentinel values

{
  "time": "",
  "durationMinutes": 0
}

Here, an empty string and 0 have special meanings: the user didn't supply those values.

The application must convert these sentinels back into explicit missing values before doing anything with them. Otherwise, it might accidentally interpret 0 minutes as a real meeting duration.

The chapter uses sentinels to keep the Spring AI, Node.js and Python versions aligned. It notes that nullable fields can be cleaner in a Node-only or Python-only application.

Memory trick: Missing → ask the user. Wrong → correct the model.

7. Why the current date and time zone matter

LLMs don't have a reliable live clock just because they can talk about dates.

If a user says “tomorrow,” the application needs to resolve that relative phrase using the current date.

The chapter's approach is to supply context such as:

Today is 2026-10-23, Friday.
The user's time zone is Asia/Kolkata.

Then the application validates the date using the same notion of today.

Why use the user's time zone?

Imagine a user in India planning a meeting just after midnight. The user's local date can already be October 24, while the server's UTC date is still October 23.

If the server uses its own date without accounting for the user's time zone, it may resolve “tomorrow” incorrectly.

The safe flow is:

1. Read the user's configured time zone.
2. Calculate today's date in that time zone.
3. Include the date and weekday in the model's instructions.
4. Validate the result against that same date.

This is especially important for scheduling, reminders, bookings and other date-sensitive features.

8. How retry with feedback works

Suppose the model returns a date in the past.

Instead of simply sending the same message again, the application should tell the model exactly what went wrong.

The feedback should also tell the model to keep values that were already correct. Otherwise, while fixing the date, it might accidentally change the attendee or meeting title.

The example implementation permits two attempts total: the first attempt and one retry. If it still fails, the application returns a clear error.

Which failures should be retried?

One critical design decision: extract the user's intent first, perform side effects later. If a retry could book a meeting directly, you risk creating duplicate bookings.

9. Structured output vs. tool calling vs. MCP

These concepts work together, but they solve different problems.

Structured output

Question: What does the user want, represented as data?

Tool calling

Question: Which function should the application execute?

MCP

Question: How can an AI application connect to tools and other capabilities through a standard protocol?

The chapter recommends starting with structured output when you need a predictable object and your code should remain in control. Use tool calling when the model needs to choose among actions or retrieve live data during the task.

10. How this fits into a Spring Boot application

Since this chapter includes Spring AI, let's connect the concepts to a backend architecture.

Client
  |
  v
POST /api/meetings/extract
  |
  v
Controller
  |
  v
MeetingExtractor
  |
  +----> LLM + strict schema
  |
  v
MeetingRules
  |
  +----> Invalid model output? Retry with feedback
  |
  +----> Required details missing? Return NEEDS_INFO
  |
  +----> All checks passed? Return READY
                                   |
                                   v
                            Calendar integration
                                   |
                                   v
                             Meeting created

The chapter separates the responsibilities:

• Controller: Handles the HTTP request and response.
• Extractor: Calls the model, parses the result and manages retries.
• Schema/record: Defines the expected data structure.
• Rules: Checks whether the data is valid.
• Calendar integration: Performs the actual booking after the application decides the data is ready.

This separation makes the validation rules easier to test without calling an LLM.

Example API contract

The chapter uses POST /api/meetings/extract with a request such as:

{
  "message": "Set up a project review with Ravi tomorrow at 3 PM for 45 minutes."
}

Possible results:

Notice the difference between 422 and 502: in the chapter, 422 means the provider responded but the extraction couldn't be accepted; 502 means the provider failed.

11. When should you use structured output?

Use it when your application needs consistent, typed data from free-form language.

Good examples:

• Extracting meeting details from natural language.
• Classifying a customer support ticket into predefined categories.
• Extracting invoice fields.
• Turning a resume into a predictable object.
• Identifying a product name, quantity and delivery preference from a message.

It's usually unnecessary when the user simply wants a paragraph, when the UI already collects the values through form fields, or when the task is long creative writing. Large documents requiring many records to be extracted may need a document-processing pipeline rather than one enormous structured response.

12. Revision notes: remember the entire chapter

The complete mental model

The one-line formula from the chapter is:

Schema (shape) + strict mode (enforcement) + rules (meaning) + feedback retry = a trustworthy object.

13. Quick quiz

Test your understanding.

1. What does a schema guarantee?
   • That every value is factually correct
   • The expected structure and supported data types
   • That the calendar event has been created
2. The model returns a valid date from last year. What should happen?
   • Accept it because the JSON is valid
   • Skip business validation
   • Reject it and retry with precise feedback
3. The user says 'Call Priya next Monday' but gives no time. What should the application do?
   • Invent 10 AM
   • Ask the user for the missing time
   • Retry the model repeatedly
4. What does structured output do compared with tool calling?
   • Structured output returns data; tool calling requests an action
   • They always create the same result
   • Structured output directly executes the calendar booking
5. What is constrained decoding?
   • A database index
   • A way to prevent schema-breaking tokens from being selected
   • A retry after a network timeout

Interview questions to practise

1. What is structured output, and why isn't “reply in JSON” enough?
2. How does strict mode use constrained decoding?
3. Why do we need business-rule validation even with a strict schema?
4. What is the difference between missing and invalid information?
5. How would you design a safe retry mechanism for an LLM extractor?
6. How are structured output, tool calling and MCP different?

If you remember only three things from this chapter, remember these: the schema checks shape, application code checks meaning, and missing information should never be guessed.