OpenAI DevDay 2023: GPT-4 Turbo and custom GPTs

Background

OpenAI's models reach most people through ChatGPT, but a large part of the company's business runs through an application programming interface (API): other companies call the models from their own software and pay for what they use. That audience cares about different things from chat users — how much text a model can take in at once, how recent its knowledge is, and what each request costs. On 6 November 2023 OpenAI held its first developer conference, DevDay, aimed at exactly that audience ^[1][2].

The scale described at the event set the context for everything announced there. OpenAI said ChatGPT had about 100 million weekly active users, that more than 92% of Fortune 500 companies used its platform, and that more than 2 million developers were building on its API ^[2].

What was announced

The centrepiece was GPT-4 Turbo ^[1]. Its headline figure was a 128K context window. A context window is the amount of text a model can hold in view at one time — the instructions, the conversation so far, and any documents pasted in — measured in tokens, which are roughly word fragments. Everything outside the window is invisible to the model when it answers. A window of 128K tokens therefore means that long reports, source code files or whole conversations can be handed to the model in one go instead of being cut into pieces first.

OpenAI also said that GPT-4 Turbo's knowledge ran up to April 2023 ^[1]. A model learns from a fixed body of text collected before training, and its knowledge stops at that point; a later cutoff means the model is less likely to be unaware of recent events. Alongside it, OpenAI announced an updated GPT-3.5 Turbo with a 16K context window by default ^[1].

The second major item was GPTs: customised versions of ChatGPT, built with a tool called GPT Builder ^[3]. Rather than calling the API directly, a user assembles a variant of ChatGPT for a particular job inside the builder.

Whisper large-v3 was released at the event as well ^[4]. Whisper is OpenAI's speech recognition system, which turns audio into text; the word large in the name refers to the biggest model size in that family, and large-v3 is a new version of it.

Key details

OpenAI described dozens of additions and improvements at the event, together with reduced pricing across many parts of its platform ^[5]. Price matters more here than it does for a chat subscription: a developer shipping a product pays for every request, so a lower price per token changes which applications are affordable to run. The company said it would begin rolling out the new features to customers at 1pm PT on the day of the event ^[5], rather than at some later date.

Why it matters

A developer conference is where a model provider stops addressing end users and addresses the people who build on top of it, and the mix announced at DevDay reflects that. Several of the changes were about removing practical limits: a longer context window saves the work of chopping documents into pieces, a newer knowledge cutoff narrows the gap between training and the present, and lower prices lower the cost of running a product. Custom GPTs pushed in a different direction, moving part of the work of building a narrow assistant out of the API and into a builder inside ChatGPT ^[3].

The event also worked as a public scoreboard. The user and developer numbers given on stage ^[2] were a way of describing how far the platform had spread, at a moment when other model providers were bringing out products of their own.