AMD MI355X Achieves GLM 5.2 Inference at Half the Cost of NVIDIA B200

According to Wafer, a startup specializing in inference optimization, AMD MI355X chips running the latest open-source model GLM 5.2 achieved inference costs at roughly half that of NVIDIA's flagship B200 GPU, while delivering approximately 80% of the performance. Testing showed single-node aggregate throughput of 2,626 tokens per second, with single-stream throughput reaching 213 tokens per second. The optimization leveraged software techniques including model quantization, inference engine selection, and ROCm compatibility fixes, rather than custom hardware.

A separate report from SemiAnalysis confirmed similar results, citing AMD MI355X inference costs of $0.22 per million tokens in interactive scenarios, below NVIDIA B200's $0.30. However, AMD's advantage remains primarily in single-node deployments; NVIDIA retains a significant lead in multi-node distributed systems and enterprise-scale applications.