Program-Aided Language Models (PAL): Let the Model Write Code to Compute

Who this is for

This guide is for readers who want LLMs to get date and arithmetic questions right more reliably. By the end you will know how to wire LangChain and the OpenAI client together, prompt the model to write Python code, and let an interpreter compute the answer.

What PAL is

Program-Aided Language models (PAL) were proposed by Gao et al. (2022)^[1]. The model reads a question written in natural language and writes the intermediate reasoning steps as a program. Chain-of-thought prompting reaches its answer through free-form text, while PAL hands the solving steps to a programming runtime similar to a Python interpreter^[1].

The example builds a small app with LangChain and OpenAI GPT-3. It reads a question, has the model generate code, and runs that code to get the answer. It targets questions that require date reasoning, and its few-shot examples come from the date understanding prompt file in the reasoning-machines/pal project on GitHub^[1].

What you need

The code imports openai, dateutil, the OpenAI wrapper from LangChain and dotenv, so your environment needs all of them^[1]:

import openai
from datetime import datetime
from dateutil.relativedelta import relativedelta
import os
from langchain.llms import OpenAI
from dotenv import load_dotenv

The API key is read from the OPENAI_API_KEY environment variable. The example sets it for both the openai client and LangChain^[1].

The key to the example is the set of examples inside the prompt. Each one writes the question as a comment and the solution as code, so the model continues in the same format. The way you write the examples therefore decides whether the generated code is usable^[1]. If you use your own questions, prepare several examples in the same format rather than giving a single question.

Step by step

First, configure the environment. Call load_dotenv(), then set the key the way the example does^[1]:

load_dotenv()

# API configuration
openai.api_key = os.getenv("OPENAI_API_KEY")

# for LangChain
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")

Second, create the model instance. The example sets temperature to 0 to keep the output as stable as possible^[1]:

llm = OpenAI(model_name='text-davinci-003', temperature=0)

Third, set the prompt and the question. The question is "Today is 27 February 2023. I was born exactly 25 years ago. What is the date I was born in MM/DD/YYYY?", and it is assigned to the question variable^[1]:

question = "Today is 27 February 2023. I was born exactly 25 years ago. What is the date I was born in MM/DD/YYYY?"

The prompt includes several worked examples. Each one writes the question as a comment, then computes the answer in Python. The first example looks like this^[1]:

# Q: 2015 is coming in 36 hours. What is the date one week from today in MM/DD/YYYY?
# If 2015 is coming in 36 hours, then today is 36 hours before.
today = datetime(2015, 1, 1) - relativedelta(hours=36)
# One week from today,
one_week_from_today = today + relativedelta(weeks=1)
# The answer formatted with %m/%d/%Y is
one_week_from_today.strftime('%m/%d/%Y')

Fourth, call the model and print the output^[1]:

llm_out = llm(DATE_UNDERSTANDING_PROMPT.format(question=question))
print(llm_out)

What the model returns

For this question, the model returns Python code rather than a plain answer. The sample output in the example is^[1]:

# If today is 27 February 2023 and I was born exactly 25 years ago, then I was born 25 years before.
today = datetime(2023, 2, 27)
# I was born 25 years before,
born = today - relativedelta(years=25)
# The answer formatted with %m/%d/%Y is
born.strftime('%m/%d/%Y')

Fifth, run the code with exec^[1]:

exec(llm_out)
print(born)

Running it prints 02/27/1998^[1].

Why it works this way

With plain chain-of-thought, the model does the arithmetic in its own text, and a wrong step is hard to spot. PAL moves the computation into the interpreter. The model only translates the question into code, and the date arithmetic is done by relativedelta^[1].

PAL works in three steps. The model turns the question into a program, the interpreter runs the program to get a result, and that result is what you read. The example follows these steps: the prompt asks for code, exec runs it, and print shows the date^[1]. The date arithmetic is done by the interpreter rather than by the model's text.

Caveats

• exec runs whatever code the model returns. Use the example for local practice, and do not run it directly in a production environment.
• The example uses text-davinci-003 and LangChain's OpenAI wrapper. If you switch to another model or interface, confirm that the parameters and call pattern still match.

Where to go next

For more detail and the example code, see the reasoning-machines/pal repository on GitHub^[1].

Adapted from the PAL page of DAIR.AI's Prompt Engineering Guide^[1], released under the MIT License.