DSPy

What it is

DSPy is a programming model for building applications with language models (LMs). Its central idea is to abstract an LM pipeline as a text transformation graph: an imperative computational graph in which LMs are invoked through declarative modules.^[1] In this view, a pipeline is not a long prompt string but a set of modules connected by data flow. Each module declares a transformation to be performed; the graph says how those transformations are composed. The language model is called inside the modules, while the surrounding program controls the order and combination of calls.^[1] This is a different starting point from treating a prompt as the main object of design.

Why it was proposed

The paper argues that existing LM pipelines are typically implemented using hard-coded prompt templates—lengthy strings discovered through trial and error.^[1] In such a pipeline, the exact wording of a prompt is written directly into the program. When the task or the model changes, the prompt text must be edited by hand, and the logic of the pipeline is entangled with the wording of individual prompts. DSPy's proposal is to make modules the unit of programming and to let the system handle how prompts are constructed and combined.^[1] The developer describes computational relations among modules rather than maintaining prompt text word by word.

How modules learn

DSPy modules are parameterized, meaning they can learn by creating and collecting demonstrations of how to apply compositions of prompting, finetuning, augmentation and reasoning techniques.^[1] A demonstration is an example of the behavior a module should produce; by collecting such examples, a module can adapt the way it uses the underlying techniques. The point is that a module is not a fixed string. It has parameters that can be set, and those parameters determine how prompting, finetuning, augmentation and reasoning are composed when the module runs.^[1] This makes the pipeline something that can be improved from examples rather than only by manual prompt editing.

The compiler

To make this practical, the authors designed a compiler that optimizes any DSPy pipeline to maximize a given metric, and they present two case studies.^[1] The metric is the objective the user wants the pipeline to satisfy; the compiler treats the pipeline as a program to be optimized against that objective. The paper describes the compiler as applying to any DSPy pipeline, rather than being tied to one hand-written prompt.^[1] The paper presents two case studies alongside the compiler.

Origin and project

The paper was first posted to arXiv on 5 October 2023 by Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam and other collaborators including Heather Miller, Matei Zaharia and Christopher Potts.^[1] The associated GitHub repository, stanfordnlp/dspy, was created on 9 January 2023; it is released under the MIT license and has roughly 39,000 stars.^[2] The repository therefore predates the arXiv posting, and MIT is a permissive open-source license.

Position in development

In the broader development of LM applications, DSPy represents an attempt to move from prompt strings toward programs with modules and optimization. The paper's framing is explicit: hard-coded prompt templates are the default, and DSPy replaces them with declarative modules inside computational graphs.^[1] The compiler is meant to improve a DSPy pipeline according to a metric rather than by trial-and-error editing of prompt text.^[1] The GitHub repository has roughly 39,000 stars, which gives one measure of attention around the project.^[2]