Baidu upgrades ERNIE Bot to ERNIE 3.5

Background

When Baidu announced ERNIE 3.5, ERNIE Bot was still in invite-only testing, so access to the chatbot was limited ^[2]. An invite-only test means users need an invitation or approval to try the product, which is common when companies want to control load and gather feedback before a wider release. ERNIE 3.5 was built on Baidu's previous ERNIE 3.0 model, continuing the company's line of large models ^[2].

What happened

In late June 2023, Baidu announced ERNIE 3.5 ^[1]. The model had already been powering beta features of ERNIE Bot since version 2.1.0 on 21 June, so some users encountered it before the formal announcement ^[1]. Beta features are early versions of product functions that may still change. The timing meant the model was not entirely new to every user at the announcement; some had already used it through ERNIE Bot.

Key details

Baidu cited a China Science Daily study using AGIEval and C-Eval and said ERNIE 3.5 beat ChatGPT in comprehensive ability and GPT-4 in several Chinese-language capabilities ^[1]. Benchmarks such as these are standardized tests used to compare model performance; Baidu's statement presented the results as evidence of the model's relative strength. The study was cited by Baidu, so the comparison came from the company's announcement. The company also claimed 17x higher inference throughput and about 2x higher training throughput ^[1]. Inference throughput refers to how much output a model can produce in a given time, while training throughput refers to how quickly a model can be trained. Higher throughput can matter because large models are expensive to run and train; it can mean more outputs per unit of hardware time or faster experimentation. ERNIE 3.5 introduced a plugin architecture with a Baidu Search plugin and the ChatFile document plugin ^[1]. A plugin architecture lets a model call outside tools or data sources, extending what it can do beyond its built-in capabilities. Baidu planned to open the plugin architecture to third parties ^[1], which would let outside developers connect their own tools or services.

Impact

The announcement placed ERNIE 3.5 in direct comparison with ChatGPT and GPT-4, especially on Chinese-language tasks, according to Baidu's cited benchmarks ^[1]. Because ERNIE Bot remained invite-only at the time, public access was still limited ^[2]. The planned opening of the plugin architecture signaled that Baidu intended to let outside developers build on ERNIE 3.5's plugins ^[1]. For users, the combination of benchmark claims, throughput claims and plugins framed ERNIE 3.5 as both a model upgrade and a platform for connected features. The claims also gave Baidu a way to compare its model with ChatGPT and GPT-4 in public, while the invite-only status kept direct access limited.