SenseNova 5.0

What it is

SenseNova 5.0 is a large language model from the Chinese AI company SenseTime. It was released on 23 April 2024 at SenseTime Tech Day events held in Shanghai, Shenzhen and other cities^[1]. It is the fifth iteration of the company's large model line, which first appeared in April 2023^[4].

How it is built

The model uses a mixture-of-experts (MoE) architecture and was trained on more than 10TB of tokens, including a large amount of synthetic data^[1]. During inference it supports a context window of about 200K tokens^[1].

A mixture-of-experts model keeps many specialised sub-networks inside it and routes each piece of input to only a few of them, instead of sending everything through the whole network. That lets the parameter count grow while the computation for any single token stays comparatively small. The context window is the amount of text the model can keep in view at once; a larger window means longer documents or longer conversations can be handled together.

SenseTime CEO Xu Li described SenseNova 5.0 as built on a three-tier Knowledge, Reasoning and Execution (KRE) architecture guided by the Scaling Law^[4]. The scaling law is the observed pattern that model capability tends to improve as parameters, data and compute grow.

Benchmark results

SenseTime said the model matches or surpasses GPT-4 Turbo on mainstream objective benchmarks, with gains in knowledge, mathematics, reasoning and coding^[1]. The company reported the top aggregate score on MMBench and leading results on MathVista, AI2D, ChartQA, TextVQA, DocVQA and MMMU^[1]. These figures were published by the developer itself, as a way of comparing models on standardised test questions.

From cloud to edge

The launch covered a cloud-to-edge lineup rather than a single model. An edge-side model generates 18.3 words per second on mid-range platforms and 78.3 words per second on flagship platforms^[1]. SenseTime said its edge image diffusion produces output of 12 million pixels or more in under 1.5 seconds on a mainstream platform^[4]. Text-to-video was still in the pipeline and was not released at the 5.0 launch^[4].

Partners and market reaction

Partners at launch included Kingsoft Office (WPS 365), Haitong Securities and Xiaomi, whose SU7 car cabin uses SenseTime technology via Xiao Ai^[1]. SenseTime shares surged more than 30% after the launch before trading was suspended^[2]; Wikipedia likewise records a share surge of over 30% following the SenseNova 5.0 announcement in April 2024^[3].

Where it sits

SenseNova 5.0 is the fifth generation of a line that debuted in April 2023^[4]. It was presented with GPT-4 Turbo as the public reference point for performance^[1], and with an emphasis on running both in the cloud and on devices: the cloud carries the largest model, while edge versions run locally on hardware^[1][4]. The launch partners span office software, securities and cars^[1], and the jump in the share price showed how closely the market was watching the release^[2][3]. Text-to-video, still unfinished at the time, marked out the direction the company said it was heading next^[4].