NVIDIA NeMo — End-to-End Flow (verified Sep 29, 2026)

Selected layers only: adapters usually go on some weight matrices, such as the attention and MLP layers, not every one.
Low rank: B × A has the same shape as W but holds far fewer numbers. That's where the memory and storage savings come from.
Merge or keep separate: after training, the adapters can be merged into W (W + B × A) to give one set of weights, or kept separate so different adapters can be swapped onto the same base model.

Sources (github.com/NVIDIA-NeMo, Apache-2.0): Curator · Automodel · Megatron-Bridge · RL · Evaluator · Export-Deploy · Speech (formerly NeMo) · org overview table. NVIDIA NIM is a separate NVIDIA product (developer.nvidia.com/nim), not a NeMo Framework repo — shown dashed.
Kubernetes layer: vendor-neutral. The object types shown (RayCluster, PyTorchJob or JobSet, Job, Deployment, Helm, NIM Operator) are common ways to run each stage, not NeMo requirements, and NeMo also runs on Slurm.
Simplifications: stage order is illustrative. Real workflows skip or repeat stages, and evaluation often happens throughout. Parallelism is part of training; it's shown separately here for clarity.