Step-1V

What it is

Step-1V is a multimodal large model released by StepFun, with about one hundred billion parameters, unveiled on 23 March 2024^[1]. "Multimodal" means that one model handles more than text: it also takes in images and connects what it sees to language. Reading a chart and writing an answer used to be separate jobs for separate systems; a multimodal model puts both in one place, so it can look at a picture and answer questions about it, or produce new content based on what the picture contains.

Origins

Chinese Wikipedia dates the model's completion to November 2023^[2], with the public launch coming some months later. Later models in the same line include Step-1.5V and Step-2^[3].

What it can do

According to the launch description, Step-1V reads the text, data and charts inside images and can create content, reason and analyse data on that basis; it also supports video understanding^[1]. The difference from plain text input is that the model has to work out what is in the picture before it can answer — where the numbers sit in a table, which bar in a chart is taller, what a diagram is claiming. Video understanding extends the input from a single static frame to material that changes over time.

Where it stood at launch

At launch, Step-1V ranked first on the OpenCompass multimodal leaderboard, and StepFun described its performance as on par with GPT-4V^[1]. A leaderboard works by running many models through the same set of tasks and comparing the scores; the result therefore describes a relative position under a particular evaluation setup rather than a verdict on every kind of use.

By November 2024, Chinese Wikipedia records it as ranking first among Chinese models in the vision category of the LMSYS Chatbot Arena^[2]. The Arena uses a different method: people compare the answers of two models without being told which is which, and the ranking is built from those preferences. A fixed task set and blind pairwise comparison measure somewhat different things, which is why a model can sit in different places on the two.

Successor

On 5 July 2024, at WAIC, StepFun released Step-1.5V as the follow-up^[3]. The new version improved image perception, added video understanding covering objects, people, scenes and moods, and used Step-2 to strengthen reasoning^[3]. That added reasoning shows up in concrete tasks: solving maths problems from images, writing code, and composing poetry from an image^[3]. The pattern is a common one for multimodal systems — perception, which is getting the content of the image right, and reasoning, which is what to do with that content, are improved separately and then combined.

Closed model and current status

Step-1V is a closed model, and Chinese Wikipedia lists it as deprecated^[2]. Closed here means the model is not published for others to download and run; it is reached through a service instead. Being listed as deprecated reflects the arrival of a successor that took over its role in the line.