Multimodal Concepts
| Multimodal models A multimodal model handles more than one kind of input or output, such as images together with text, inside a single model. Work on CLIP and Flamingo showed how such models are trained and how they can be steered toward new tasks. | |
| Vision-Language-Action model (VLA) A vision-language-action model (VLA) is a robot policy that takes in camera images and a language instruction and outputs actions directly, usually by adapting a pretrained vision-language model to control. | |
| Computer use Computer use is an Anthropic capability, released in public beta in October 2024, that lets an AI model operate a computer the way a person does — reading the screen, moving a cursor, clicking and typing — so it can drive ordinary software built for humans. |