Fine-tuning with ms-swift

Choosing a fine-tuning framework comes down to three questions: how wide is the model coverage, how low is the VRAM floor, and how complete is the path from training to deployment. ms-swift, the ModelScope community's answer, supports training, inference, evaluation, quantization, and deployment for 600+ text LLMs and 400+ multimodal LLMs with Day-0 support for popular releases ^[1]. This page follows the framework's classic example: 10 minutes on a single RTX 3090 to give Qwen3-4B-Instruct-2507 a new self-identity ^[1].

The capability map

First, what the framework actually covers ^[1]. Training: lightweight methods in a full row — LoRA, QLoRA, DoRA, LoRA+, LLaMAPro, LongLoRA, LoRA-GA, ReFT, RS-LoRA, Adapter, LISA; preference tasks — DPO, KTO, RM, CPO, SimPO, ORPO; plus Embedding, Reranker, and sequence classification. Reinforcement learning bundles a rich GRPO family — GRPO, DAPO, GSPO, SAPO, CISPO, CHORD, RLOO, Reinforce++ — with synchronous and asynchronous vLLM acceleration and plugins for reward functions, multi-turn schedulers, and environments ^[1].

Scaling takes two roads: conventional distribution — DDP, device_map simple model parallelism, DeepSpeed ZeRO2/ZeRO3, FSDP/FSDP2 — and Megatron parallelism with TP/PP/SP/CP/ETP/EP/VPP strategies that speed MoE training significantly across 300+ text and 100+ multimodal models in full-parameter and LoRA modes ^[1]. The VRAM toolbox carries GaLore, Q-GaLore, UnSloth, Liger-Kernel, Flash-Attention 2/3, and Ulysses plus Ring-Attention sequence parallelism for long-text work ^[1].

Hardware reach is equally broad: NVIDIA (CUDA), AMD (ROCm), Huawei Ascend NPU, MetaX GPU, Apple MPS, and plain CPU ^[1]. Quantized models train directly in BNB, AWQ, GPTQ, AQLM, HQQ, and EETQ formats — a 7B model trains with only 9GB ^[1]. On the multimodal side, packing lifts training speed by 100%+, text, image, video, and audio mix in one dataset, and the vit/aligner/llm components control independently ^[1].

Datasets come bundled: 150+ across pre-training, fine-tuning, alignment, and multimodal tasks; custom sets follow the documented format and pass through --dataset ^[1]. The rest of the pipeline is complete too: four inference engines (Transformers, vLLM, SGLang, LmDeploy) behind an OpenAI-compatible interface; evaluation through EvalScope with 100+ datasets; quantization export in AWQ, GPTQ, FP8, and BNB that plugs straight into vLLM/SGLang/LmDeploy ^[1]. A Web-UI wraps the whole chain for the command-line-averse ^[1].

Ten minutes to a new identity

Self-cognition fine-tuning is the most直观 way to verify training worked: teach the model who it is. The official example re-brands Qwen3-4B-Instruct-2507 on one 3090, about 13GB of VRAM, in roughly ten minutes ^[1]:

# 13GB
CUDA_VISIBLE_DEVICES=0 \
swift sft \
    --model Qwen/Qwen3-4B-Instruct-2507 \
    --tuner_type lora \
    --dataset 'AI-ModelScope/alpaca-gpt4-data-zh#500' \
              'AI-ModelScope/alpaca-gpt4-data-en#500' \
              'swift/self-cognition#500' \
    --torch_dtype bfloat16 \
    --num_train_epochs 1 \
    --per_device_train_batch_size 1 \
    --per_device_eval_batch_size 1 \
    --learning_rate 1e-4 \
    --lora_rank 8 \
    --lora_alpha 32 \
    --target_modules all-linear \

    --gradient_accumulation_steps 16 \
    --eval_steps 50 \
    --save_steps 50 \
    --save_total_limit 2 \
    --logging_steps 5 \
    --max_length 2048 \
    --output_dir output \
    --warmup_ratio 0.05 \
    --dataloader_num_workers 4 \
    --model_author swift \
    --model_name swift-robot

The dataset mix deserves attention: 500 Chinese and 500 English general-instruction samples form the base while 500 self-cognition samples inject the identity — identity-only data produces a model that does nothing but introduce itself, and general data underneath is the standard guard against catastrophic forgetting ^[1]. A few tips: --model_author and --model_name only take effect when the dataset includes swift/self-cognition; switching models is a single --model change; downloads default to ModelScope with --use_hf true switching to HuggingFace ^[1].

The LoRA baseline, parameter by parameter

This command doubles as a baseline template worth reading closely ^[1]. tuner_type lora with rank 8 and alpha 32, and target_modules all-linear pulls every linear layer into training — rank 8 is a small budget, and the 4x alpha-to-rank ratio gives adapters plenty of scaling room. The 1e-4 learning rate is conventional for LoRA, an order of magnitude above full-parameter fine-tuning. Batch size 1 with gradient accumulation 16 yields an effective batch of 16 — the standard play on small-VRAM single cards. One epoch suffices for identity-injection tasks; more invites overfitting. save_steps 50 with save_total_limit 2 keeps only the two newest checkpoints on disk, and max_length 2048 caps sequence length, directly setting memory use ^[1].

Inference, acceleration, publication

After training, swift infer verifies the result with --adapters pointing at the last checkpoint folder ^[1]:

# 使用交互式命令行进行推理
CUDA_VISIBLE_DEVICES=0 \
swift infer \
    --adapters output/vx-xxx/checkpoint-xxx \
    --stream true \
    --temperature 0 \
    --max_new_tokens 2048

A convenient design: the adapters folder carries the training arguments in args.json, so --model and --system are read automatically — disable with --load_args false ^[1]. Temperature 0 is right for verification, where deterministic output lets one question — who are you — reveal whether the fine-tune landed ^[1].

For production, merge the LoRA into the base weights and switch to the vLLM backend ^[1]:

# merge-lora并使用vLLM进行推理加速
CUDA_VISIBLE_DEVICES=0 \
swift infer \
    --adapters output/vx-xxx/checkpoint-xxx \
    --stream true \
    --merge_lora true \
    --infer_backend vllm \
    --vllm_max_model_len 8192 \
    --temperature 0 \
    --max_new_tokens 2048

Merging removes the adapter's runtime overhead while vLLM lifts throughput, and vllm_max_model_len 8192 leaves context headroom ^[1].

The final step publishes the result to ModelScope ^[1]:

CUDA_VISIBLE_DEVICES=0 \
swift export \
    --adapters output/vx-xxx/checkpoint-xxx \
    --push_to_hub true \
    --hub_model_id '<your-model-id>' \
    --hub_token '<your-sdk-token>' \
    --use_hf false

Train, verify, deploy, publish — four steps, one CLI ^[1]. Once this ten-minute example runs, swapping datasets, models, or training tasks is all parameter changes. That is the framework's value: turning fine-tuning from writing training scripts into adjusting command-line flags.

Adapted from the Quick-start section of the ModelScope ms-swift documentation^[1], released under the Apache-2.0 License.