SageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints
中文摘要
Amazon SageMaker现支持NVIDIA MIG技术,可将GPU硬件分区,实现模型并行运行或按需分配算力,从而显著提升部署效率与利用率。
English Summary
Amazon SageMaker now supports NVIDIA MIG, allowing GPU partitioning to run multiple models per chip, effectively increasing utilization and optimizing performance for model endpoints.
原文节选
Amazon SageMaker Inference now supports GPU partitioning with NVIDIA Multi-Instance GPU (MIG), which divides one GPU into hardware-isolated partitions so you can run a different model in each or take a right-sized fraction of a newer accelerator. This post shows how two optional API fields place one model per partition, and what we measured against a whole older GPU.