JSON Mode and Structured Outputs

Large language models normally return natural-language text, but many software applications need machine-readable data. A backend may need an object containing a user's name and email, a frontend may need a predictable list of items, or an automation system may need a classification result with one of several predefined values. Asking an LLM to simply "return JSON" can help, but it does not necessarily guarantee that the response follows the exact structure an application expects.

JSON mode and structured outputs are two approaches designed to make LLM responses easier for software to consume. They are related, but they solve different levels of the problem. JSON mode generally focuses on producing valid JSON, while structured outputs can impose a more specific schema on that JSON.

Understanding the difference is important when building production AI applications. Choosing the right approach can make parsing easier, reduce formatting errors, simplify validation, and create a much more predictable interface between an LLM and traditional application code.

What Is JSON Mode?

JSON mode is an LLM API feature that instructs the model to return a response in valid JSON format. Instead of generating arbitrary prose, the model is constrained or guided to produce JSON that can be parsed by the application.

For example, an application might ask an LLM to analyze a support message and return JSON:

{
  "category": "billing",
  "priority": "high"
}

The important point is that JSON mode primarily addresses the serialization format. It helps prevent responses such as "The category is billing and the priority is high" when the application expects JSON.

What JSON Mode Does Not Guarantee

Valid JSON is not the same thing as valid application data. JSON mode may ensure that the response can be parsed as JSON, but the application may still receive the wrong properties, unexpected values, missing fields, or incorrect data types depending on the API and its capabilities.

{
  "category": "billing",
  "importance": "very-high",
  "details": "Customer is asking about a charge."
}

Suppose the application expects category and priority. The object above may be valid JSON, but it does not necessarily satisfy the application's expected contract. The property importance has a different name, and priority is missing.

What Are Structured Outputs?

Structured outputs take the idea further by defining the expected shape of the model response. Instead of merely requiring valid JSON, the application can specify properties, types, required fields, arrays, nested objects, and sometimes allowed values.

{
  "type": "object",
  "properties": {
    "category": {
      "type": "string",
      "enum": ["billing", "technical", "account"]
    },
    "priority": {
      "type": "string",
      "enum": ["low", "medium", "high"]
    }
  },
  "required": ["category", "priority"],
  "additionalProperties": false
}

The schema gives the model and API a much clearer contract. Instead of simply asking for JSON, the application describes the exact structure that should be returned.

JSON Mode vs Structured Outputs

The exact behavior depends on the LLM provider. Different APIs use different terminology and expose different levels of schema enforcement. Developers should therefore check the documentation for the specific model and API they are integrating.

Why Valid JSON Is Not Enough

JSON is intentionally flexible. The same information can be represented using different property names, different nesting, optional fields, and different data types. That flexibility is useful for general data exchange but can create problems when an LLM is connected directly to application logic.

{
  "name": "Alex",
  "age": 30
}

An application expecting this structure may fail if the model instead returns:

{
  "fullName": "Alex",
  "age": "30 years old"
}

Both objects can be represented as valid JSON, but the second one does not satisfy the same application contract. Structured outputs are designed to reduce this kind of mismatch.

JSON Schema

JSON Schema is a standard way to describe the structure and constraints of JSON data. It can define objects, strings, numbers, booleans, arrays, required properties, enumerations, nested structures, and other rules.

{
  "type": "object",
  "properties": {
    "name": {
      "type": "string"
    },
    "age": {
      "type": "integer",
      "minimum": 0
    },
    "role": {
      "type": "string",
      "enum": ["developer", "designer", "manager"]
    }
  },
  "required": ["name", "age", "role"],
  "additionalProperties": false
}

When an LLM API supports schema-based structured output, a schema like this can become part of the contract between the application and the model.

How JSON Mode Works

The exact implementation differs between providers, but the general workflow is straightforward. The application sends a prompt together with a request for JSON output. The API generates a response intended to be valid JSON, which the application then parses.

Application
    ↓
Prompt + JSON requirement
    ↓
LLM API
    ↓
JSON response
    ↓
JSON.parse()
    ↓
Application logic

The key limitation is that JSON parsing only answers one question: can the response be parsed as JSON? It does not automatically answer whether the object contains the correct fields or whether those fields contain acceptable values.

How Structured Outputs Work

Structured-output APIs add a schema to the request. Depending on the provider, the generation system may use constrained decoding or another mechanism to make the generated result conform more closely to the requested schema.

Application
    ↓
Prompt + output schema
    ↓
LLM API
    ↓
Schema-constrained response
    ↓
Parsing
    ↓
Runtime validation
    ↓
Application logic

This approach reduces the amount of formatting responsibility placed on the model. The application explicitly defines the shape it needs instead of relying only on natural-language instructions.

Prompting JSON Mode Correctly

Even when an API provides JSON mode, the prompt should clearly explain what the model needs to return. A good prompt describes the task, expected fields, allowed values, and important edge cases.

Analyze the customer message.

Return an object containing:
- category: "billing", "technical", or "account"
- priority: "low", "medium", or "high"
- summary: a short string

Return only the requested JSON object.

Customer message:
I was charged twice for the same order.

Clear instructions are useful even when a structured-output mechanism is available. The schema defines the shape, while the prompt explains what the fields should represent.

Structured Outputs and Runtime Validation

Schema-constrained generation improves reliability, but production applications should still consider runtime validation. The LLM is an external service, and application code should not blindly trust external data.

import { z } from "zod";

const ResultSchema = z.object({
  category: z.enum(["billing", "technical", "account"]),
  priority: z.enum(["low", "medium", "high"]),
  summary: z.string(),
});

const result = ResultSchema.safeParse(modelResponse);

if (!result.success) {
  throw new Error("Invalid AI response");
}

const data = result.data;

Validation is especially useful when the result will be stored in a database, passed to another service, displayed in a critical interface, or used as input for application logic.

TypeScript Types Are Not Enough

TypeScript interfaces and types describe what the compiler expects, but they do not validate data at runtime. This distinction is important when processing responses from an LLM API.

interface CustomerResult {
  category: "billing" | "technical" | "account";
  priority: "low" | "medium" | "high";
}

const result = modelResponse as CustomerResult;

The type assertion above does not inspect the actual object. It simply tells TypeScript to trust the value. Runtime schemas provide a real validation step.

When to Use JSON Mode

JSON mode is useful when the application needs valid JSON but does not require a highly strict schema or when the API does not provide a stronger structured-output mechanism.

• Simple extraction tasks
• Small internal tools
• Prototype applications
• Responses with flexible structures
• APIs that support JSON mode but not schema-constrained output
• Applications where the result is validated separately

For a prototype, JSON mode may be a practical compromise between free-form text and fully constrained structured output.

When to Use Structured Outputs

Structured outputs are preferable when the generated data must follow a predictable contract. They are particularly valuable when the response is consumed automatically rather than simply displayed to a human.

• Production API integrations
• Database record generation
• Data extraction pipelines
• Classification systems
• Automated workflows
• Structured UI generation
• Agent systems
• Document processing
• Applications with strict data contracts

Data Extraction Example

Imagine an application that receives customer emails and needs to extract structured information. A free-form model response might be difficult to process automatically.

With a schema, the application could request a structure such as:

{
  "customerName": "Maria",
  "orderId": "ORD-8421",
  "issueType": "delivery",
  "urgent": false
}

The backend can then validate the response and pass the individual fields to the appropriate application components.

Classification Example

Classification is another strong use case. Suppose an application needs to classify incoming messages into exactly three categories.

{
  "category": "technical"
}

A schema can restrict category to a known set of values. This is better than accepting arbitrary natural-language labels because downstream code can rely on a finite set of possible results.

Nested Structured Responses

Structured outputs can also describe nested objects and arrays. This is useful when a single response contains several related entities.

{
  "product": {
    "name": "Mechanical Keyboard",
    "price": 99.99
  },
  "features": [
    "Hot-swappable switches",
    "RGB lighting",
    "USB-C"
  ],
  "rating": 4.5
}

The deeper the schema becomes, the more important it is to keep the design understandable. Developers should avoid adding unnecessary nesting simply because the schema system allows it.

Optional vs Required Fields

One of the most important schema design decisions is whether a field should be required. A field should be required when the application cannot operate correctly without it. If information may legitimately be unavailable, the schema should represent that possibility.

const ArticleSchema = z.object({
  title: z.string(),
  summary: z.string(),
  author: z.string().optional(),
});

Making every field required can create unnecessary failures when the source text does not contain enough information. Making everything optional, on the other hand, can make the result too ambiguous for downstream code.

Enums and Controlled Values

Enums are especially useful for AI classification. If an application expects a fixed set of states, defining those states explicitly reduces the number of possible outputs.

const TicketSchema = z.object({
  status: z.enum([
    "open",
    "pending",
    "resolved",
  ]),
});

Without a controlled vocabulary, the model might return values such as "closed", "done", "finished", or "resolved successfully". An enum makes the application's expected states explicit.

Handling Null Values

Some information may be genuinely unavailable. In those cases, a schema can explicitly allow null values when the application needs to distinguish between a missing field and a known empty value.

const UserSchema = z.object({
  name: z.string(),
  phone: z.string().nullable(),
});

The exact representation should be chosen according to the application's data model. The important principle is to represent uncertainty explicitly rather than forcing the model to invent a value.

JSON Mode and Hallucinations

Neither JSON mode nor structured outputs prevents hallucinations. A model can generate a perfectly valid JSON object containing information that is simply incorrect.

{
  "company": "Example Corp",
  "revenue": 125000000
}

The object may satisfy every schema requirement while the revenue figure is completely fabricated. Structured generation solves the format problem, not the factual accuracy problem.

Security Considerations

Structured output should still be treated as untrusted input. A schema can verify that an amount is a number or that an action belongs to an allowed list, but it cannot determine whether the current user is authorized to perform that action.

This becomes especially important when structured responses control tools, database operations, financial actions, account changes, or external API calls. The LLM should suggest or generate data, while deterministic application code decides what the system is actually allowed to do.

Structured Outputs and Function Calling

Function calling is related to structured output but has a different purpose. Structured output is primarily about the shape of the model's response. Function calling allows the model to request execution of a predefined function or tool using structured arguments.

{
  "name": "getWeather",
  "arguments": {
    "city": "London"
  }
}

An application can use structured outputs for the final response and function calling for interactions with external tools. These techniques can therefore complement each other in agentic applications.

Streaming Structured Responses

Streaming introduces an additional challenge because the application receives the response incrementally. A partial JSON object may not be valid JSON until the model finishes generating it.

For this reason, applications should avoid treating every streamed fragment as a complete structured response. If structured streaming is required, use the mechanisms provided by the API and validate the complete result before performing important operations.

Error Handling

A reliable application should distinguish between different types of failures. A request may fail because of a network problem, API error, rate limit, timeout, invalid input, or provider-side restriction. Alternatively, the request may succeed while the resulting data fails application-level validation.

try {
  const response = await generateResponse(input);

  const result = ResultSchema.safeParse(response);

  if (!result.success) {
    throw new Error("Invalid structured response");
  }

  return result.data;
} catch (error) {
  console.error("AI request failed", error);
  throw error;
}

Separating infrastructure errors from validation errors makes debugging much easier. It also allows the application to retry temporary failures without repeatedly retrying responses that are invalid because of a prompt or schema problem.

Testing JSON and Structured Outputs

Structured-output systems should be tested with realistic input rather than only ideal examples. Users can provide incomplete, ambiguous, malformed, multilingual, or adversarial information.

• Test empty input.
• Test incomplete information.
• Test conflicting information.
• Test very long input.
• Test unexpected values.
• Test multiple languages if supported.
• Test malformed source documents.
• Test prompt-injection attempts inside user-provided content.
• Test provider errors and timeouts.
• Test validation failures.

Maintain representative test cases and run them after changing the prompt, schema, model, or application logic. This is especially important when an LLM is part of a larger automated workflow.

Performance and Cost Considerations

Structured output itself does not eliminate model latency or token usage. The prompt and schema become part of the request, and larger schemas can increase the amount of information sent to the model API.

For applications making large numbers of requests, keep schemas concise and avoid including unnecessary descriptions or deeply nested structures. At the same time, do not remove information that is necessary to make the contract unambiguous.

Choosing Between JSON Mode and Structured Outputs

Best Practices

• Use JSON mode when valid JSON is the main requirement.
• Prefer structured outputs when the application requires a specific schema.
• Define schemas around real application requirements.
• Use enums for finite sets of allowed values.
• Keep schemas as simple as practical.
• Clearly explain the semantic meaning of important fields in the prompt.
• Validate model responses at runtime.
• Treat model-generated data as untrusted input.
• Keep authorization and business rules outside the LLM.
• Handle provider, network, parsing, and validation errors separately.
• Test with incomplete, ambiguous, and adversarial inputs.
• Check the current documentation for the specific provider and model.

Frequently Asked Questions

Helpful AI Tools

Developer tools for JSON formatting, JSON validation, schema inspection, API testing, and prompt testing can be useful when building structured LLM integrations. They make it easier to inspect generated responses, identify malformed data, and verify that application schemas match the data being produced.

Conclusion

JSON mode and structured outputs solve an important problem in LLM application development: turning flexible model generation into data that software can consume. JSON mode is primarily concerned with producing valid JSON, while structured outputs provide a more explicit contract for fields, types, and allowed values.

For simple applications, JSON mode combined with runtime validation may be sufficient. For production systems that depend on predictable data, schema-based structured outputs are usually a stronger foundation. Regardless of the approach, developers should validate external model responses, keep security and business rules in deterministic code, and remember that valid structure does not guarantee factual correctness.