AMD announced recently that Oracle Cloud Infrastructure (OCI) has selected AMD Instinct MI300X accelerators with ROCm open software to power its latest OCI Compute Supercluster instance, BM.GPU.MI300X.8.

This new instance is designed for AI models that can include hundreds of billions of parameters. The OCI Supercluster, utilizing AMD MI300X, supports up to 16,384 GPUs in a single cluster, using the same ultrafast network fabric technology as other accelerators on OCI. These OCI bare metal instances, designed for high-throughput AI workloads such as large language model (LLM) inference and training, have already been adopted by companies like Fireworks AI.

“AMD Instinct MI300X and ROCm open software continue to gain momentum as trusted solutions for powering the most critical OCI AI workloads,” said Andrew Dieckmann, corporate vice president and general manager, Data Center GPU Business, AMD. “As these solutions expand further into growing AI-intensive markets, the combination will benefit OCI customers with high performance, efficiency, and greater system design flexibility.”

“The inference capabilities of AMD Instinct MI300X accelerators add to OCI’s extensive selection of high-performance bare metal instances, removing the overhead of virtualized compute commonly used for AI infrastructure,” said Donald Lu, senior vice president, software development, Oracle Cloud Infrastructure. “We are excited to offer more choice for customers seeking to accelerate AI workloads at a competitive price point.”

The AMD Instinct MI300X underwent extensive testing, validated by OCI, which confirmed its AI inferencing and training capabilities, especially for latency-sensitive use cases, even with larger batch sizes, and the ability to handle the largest LLM models in a single node. These performance results have attracted attention from AI model developers. Fireworks AI, a platform designed to build and deploy generative AI, is utilizing the performance benefits of OCI powered by AMD Instinct MI300X.

“Fireworks AI helps enterprises build and deploy compound AI systems across a wide range of industries and use cases,” said Lin Qiao, CEO of Fireworks AI. “The memory capacity available on the AMD Instinct MI300X and ROCm open software allows us to scale services for our customers as models continue to grow.”