Run Examples
After installing MS-AMP, you can run several simple examples using MS-AMP. Please note that before running these commands, you need to change work directory to examples .
#
#
python mnist.py --enable-msamp --opt-level=O2
#
torchrun --nproc_per_node=8 mnist_ddp.py --enable-msamp --opt-level=O2
#
python mnist_fsdp.py --msamp
#
#
deepspeed cifar10_deepspeed.py --deepspeed --deepspeed_config ds_config.json
#
deepspeed cifar10_deepspeed.py --deepspeed --deepspeed_config ds_config_msamp.json
#
deepspeed cifar10_deepspeed.py --deepspeed --deepspeed_config ds_config_zero_msamp.json
#
deepspeed cifar10_deepspeed_te.py --deepspeed --deepspeed_config ds_config_zero_te_msamp.json
Note
If you get "ModuleNotFoundError: No module named 'timm'" error when running this example, you need to install timm using
pip install timm
.
For more comprehensive examples, please go to MS-AMP-Examples .