Dify LLM Node: Prompts, Parameters and Structured Output

Who this is for, and what you will be able to do

This guide is for builders who want a Dify workflow to call a language model with a prompt assembled from their own variables, and who need the output in a fixed format. After reading it, you will know how to configure the node, write prompts with variables and knowledge base context, get structured output, and handle reasoning output and errors.

Before you start

Configure at least one model provider under Integrations > Model providers first^[1]. The LLM node then lets you choose from any configured model^[1].

Set the model parameters

Temperature ranges from 0 (deterministic) to 1 (creative). Nucleus sampling limits word choice by probability. Frequency penalty reduces repetition, and presence penalty encourages new topics^[1]. The presets Precise, Balanced and Creative give you a starting point^[1].

Share one model setting across nodes

When several LLM nodes should use the same model, store the choice in an environment variable instead of configuring each node^[1]. Open the Environment variables panel, add a variable of type LLM, and choose the model and its parameters^[1]. In each LLM node, switch the Model field to variable mode and select that variable^[1]. The variable carries the model provider, the model and its parameters, while settings such as the prompt stay in each node^[1]. Editing the variable, for example to point it at another model, updates every node bound to it^[1].

Write the prompt

The prompt editor adapts to the model type. Chat models use message roles: System sets behaviour, User gives input and Assistant gives examples. Completion models use simple text continuation^[1].

Reference workflow variables with double curly braces. They are replaced with their actual values before the prompt reaches the model^[1]. A chat prompt has this shape^[1]:

System: You are a technical documentation expert.
User: {{user_input}}

Add knowledge base context to the prompt

Context variables inject external knowledge while keeping its source attribution. This is what makes retrieval-augmented generation apps possible, where the model answers from your own documents^[1]. Connect the output of a Knowledge retrieval or Agent knowledge retrieval node to the context input of the LLM node, then reference it in the prompt^[1]:

Answer using only this context:
{{knowledge_retrieval.result}}

Question: {{user_question}}

When a context variable comes from knowledge retrieval, Dify tracks the citations automatically, so users can see the sources^[1].

Get structured output

Structured output makes the model return a fixed data format, such as JSON, that a program can use directly^[1]. There are three ways to configure it^[1]:

• Visual editor: add named, typed fields, mark the required ones and set descriptions. The editor generates a JSON Schema automatically.
• JSON Schema: write the schema directly for nested objects, arrays and validation rules.
• AI generation: describe the need in natural language and let AI write the schema.

A schema for a sentiment field, which is required and accepts three values^[1]:

{
  "type": "object",
  "properties": {
    "sentiment": {
      "type": "string",
      "enum": ["positive", "negative", "neutral"]
    }
  },
  "required": ["sentiment"]
}

If Model not supported appears beside Structured output, the model does not support structured output natively. Dify then adds the JSON Schema to the prompt, but the reply may still not match the schema and the node run may fail. For stable structured output, choose a model without this notice^[1].

Keep memory in a Chatflow

Enable memory to keep context across several LLM calls in one Chatflow conversation. Earlier interactions are included in later prompts as formatted user and assistant output^[1]. Memory belongs to the node and does not carry over to other conversations. You can customize the user prompt by editing the USER template^[1].

Handle files and images

For file handling, add a file variable to the prompt of a multimodal model. GPT-4V processes images, Claude processes PDFs directly, and other models may need preprocessing^[1]. For vision, High detail gives better accuracy on complex images but uses more tokens, while Low is faster and cheaper for simple images^[1]. The default variable selector for vision is userinput.files, which takes the files automatically from the User Input node^[1].

Use Jinja2 for loops and conditions

In Jinja2 mode (edition_type: "jinja2"), prompts accept Jinja2 templates for loops, conditions and complex data transformation^[1]. Jinja2 variables are handled separately from regular variable substitution^[1]. A loop over search results looks like this^[1]:

{% for item in search_results %}
{{ loop.index }}. {{ item.title }}: {{ item.content }}
{% endfor %}

Streaming and reasoning output

The LLM node streams results by default. Each text chunk is emitted as a RunStreamChunkEvent for real-time display^[1]. Some reasoning models wrap their thinking in <think>...</think> tags, which by default stay in the text output. Turning on Enable reasoning tag separation moves the thinking into a separate reasoning_content output variable^[1]. In the API this switch maps to reasoning_format, and streaming clients receive the reasoning through a separate reasoning_chunk event^[1]. The setting applies only to models that wrap their reasoning in <think> tags^[1].

Handle errors and retries

Set the retry behaviour for failed LLM calls: the maximum number of retries, the retry interval and the backoff multiplier. If retries are not enough, define a fallback, such as a default value, error routing or an alternative model^[1].

Next steps

For the full list of LLM node settings, see the Dify documentation^[1].

Adapted from the Dify documentation page on the large language model node^[1], released under the CC BY 4.0 license.