One cannot dispute the fact that Nvidia (NVDA) presently stands on the throne as the top dog in the realm of artificial intelligence (AI) computing infrastructure, solidified after years of strategic investments in capital.
Nevertheless, the burning question remains: can Nvidia fiercely defend its title, or will its supremacy be encroached upon?
The Anatomy of an AI Supercomputer
Lately, signs have emerged of a few underdogs peering out of their hiding places, hinting at the possibility of challenging Nvidia’s unchallenged dominance. To comprehend whether these contenders pose a genuine threat to Nvidia’s hegemony, one must first grasp the fundamentals of how an AI supercomputer operates.
In basic terms, an AI supercomputer comprises clusters of graphic processing units (GPUs) churning through myriad calculations. However, the equation also involves a sprinkle of central processing units (CPUs) to conduct orchestration along with memory chips — encompassing both short-term Dynamic Random-Access Memory (DRAM) and long-term NAND chips.
A cost breakdown of an AI supercomputer typically unfolds such that approximately 50% to 60% of the total costs are attributed to GPUs. Following suit, 10% to 15% find solace in CPUs and DRAM chips, leaving a modest 5% to 10% share to be shouldered by NANDs.
The Critical Role of Infiniband and Networking
Of paramount importance is uniting these components together. This critical task is undertaken either via Nvidia’s InfiniBand or through the humble Ethernet cables, aptly termed “networking.” Notably, this latter element accounts for the final 10% to 15% of the hardware expenditure.
In a strategic move, Nvidia expanded its horizons by acquiring InfiniBand through the acquisition of Mellanox for a substantial $6.9 billion. Mellanox, a pioneer in high-performance interconnect technology, birthed the InfiniBand interconnect technology — now a staple in over half of the world’s speediest supercomputers and several top-tier hyperscale data centers.
By integrating this acquisition, Nvidia positioned itself as a one-stop hub for all hardware necessities required by any prospective buyer looking to construct an AI powerhouse.
Painting a picture of the future, envisage data centers metamorphosing into colossal computing engines housing tens of thousands of compute nodes, meticulously architected with interconnects designed to deliver peak performance. The ultimate objective is to establish connections boasting maximum possible bandwidth to facilitate swift data flow.
Amidst the hullabaloo around Nvidia’s GPU segment, the networking department quietly steers a significant portion of its success. During the recent earnings call, Nvidia proudly showcased robust sales figures in the networking realm, which have assumed an increasingly pivotal role as corporations embark on assembling clusters integrating tens of thousands of chips requiring seamless interconnection. Nvidia proudly revealed a $3.2 billion figure in networking revenue, predominantly emanating from its InfiniBand products, surging over threefold compared to the preceding year.
Clash of Titans: Infiniband vs. Ethernet
A longstanding belief thrived that InfiniBand outshined Ethernet in the realm of AI due to its capability to shuttle data directly between memory chips sans the CPU’s meddling, thereby trimming latency.
Within the sphere of AI workloads, bandwidth and latency reign supreme, with InfiniBand basking in low latency courtesy of an architecture that curbs packet loss. Conversely, Ethernet networks tend to grapple with an inherent issue of packet loss. However, Ethernet outshines InfiniBand in terms of achieving superior raw bandwidth ceilings.
In essence, the analogy of InfiniBand vs. Ethernet can be compared to traversing an interstate highway (Ethernet) flaunting high-speed limits yet plagued by probable traffic snarls, versus navigating local roads (InfiniBand) embodying slower speeds but steering clear of congestion.
One standout reason behind InfiniBand’s surge in popularity stems from its coupling with Nvidia’s GPUs, which were notoriously scarce. Customers willingly embraced InfiniBand, spurred by the promise of securing GPUs simultaneously. Yet, the dwindling backlog of GPUs injects a hint of uncertainty, potentially eroding the allure of InfiniBand among consumers.
In a recent earnings rendezvous, Ethernet networking behemoth Arista Networks (ANET) unabashedly lauded its prowess in overpowering InfiniBand in the arena of AI computing.
The Ascendancy of AI Networking Growth
Undeniably, Arista appears to be hitting all the right notes. In the initial quarter, Arista witnessed a 16% surge in revenue year-on-year, with earnings per share notching up a commendable 44%. This growth trajectory is expected to gain momentum as AI infrastructure investments soar. Around 40% of Arista’s business lifeline is nourished by giants like Microsoft (MSFT) and Meta (META), who are intensifying capital expenditure on AI ventures.
This isn’t to insinuate that Nvidia’s reign is on the cusp of faltering as the reigning champion in the AI infrastructure realm. The majority of developers are tethered to Nvidia’s CUDA program, effectively solidifying its status as the go-to lingua franca for AI engineers, constituting a formidable advantage for Nvidia.
While Nvidia still spearheads the race in AI, the operating margin exceeding 50% has conjured a cauldron of competitors eyeing a slice of the pie. Standing toe-to-toe in this arena is Arista, alongside Broadcom (AVGO), vying to carve a niche in the cutthroat networking domain.
The future holds a more favorable outlook for Arista, with anticipated revenue contributions trickling in from the AI domain this year, poised to catapult to $750 million in 2025 — a mere glimpse of the iceberg for the company.
Penning down his thoughts earlier this month, Morningstar analyst William Kerwin bolstered Arista’s potential, forecasting a surge in value hailing from generative artificial intelligence networking. He painted a promising picture of Arista reaping the fruits of accelerated AI spending enveloping its best-of-breed high-speed Ethernet switches, steering the transition towards Ethernet and away from Nvidia’s Infiniband technology in generative AI networks over the ensuing half-decade.
Unquestionably, the stock outlook for ANET beckons a buy signal at its prevailing price hovering in the low $300s.
