A100 GPU becomes critical bottleneck for frontier AI model deployment, infrastructure capex skyrockets

As of February 2023, Nvidia's A100 GPU (priced at roughly $10,000 per unit) has become the dominant workhorse for training and inference across frontier large language models and generative image models. New Street Research estimates that deploying ChatGPT at Bing scale requires $4 billion in A100-based infrastructure, and at Google scale, $80 billion. Nvidia held 95% of GPU market share for machine learning at that time, with H100 (the successor) beginning to ship at higher cost. This establishes the capital intensity and supplier concentration underlying AI capex cycles and reveals the magnitude of infrastructure spend required to commercialize frontier models.