HodlX guest post Submit your message
In January 2024, leading private equity firm Blackstone announced it was building a $25 billion AI data empire.
A few months later, OpenAI and Microsoft followed suit with a proposal to build Stargate, a $100 billion AI supercomputer that will launch the company at the forefront of the AI revolution.
This is of course no surprise. With the rapid acceleration witnessed by the AI sector in recent years, industry giants around the world are rushing to grab a front-row seat.
Experts are already predicting that the global AI market will reach $827 billion in volume by 2030, with an annual growth rate of 29%.
The only problem? GPUs.
Von Neumann’s architecture, the design model on which most general-purpose computers operate composed of the CPU, memory, I/O devices, and the system bus It is inherently limited, even though it offers simplicity and cross-system compatibility.
The single ‘system bus’ of this architecture limits the speed at which data can be transferred between memory and the CPU making CPUs suboptimal for AI and machine learning purposes.
This is where the GPUs (graphics processing units) come in.
By incorporating parallelism as a processing technique, GPUs provide improved performance and independent instruction execution through their multi-cores.
However, with the dawn of AI technology, the demand for GPUs has skyrocketed, straining supply chains and creating a serious bottleneck for the efforts of many researchers and startups.
This is especially true because the world’s supply of GPUs comes from just one major manufacturer Nvidia.
While hyperscalers like AWS, Google Cloud Platform, and others may be able to easily gain access to Nvidia’s A100s and H100s, what are other viable alternatives that could help companies, researchers, and startups stay on the AI train instead of being stuck indefinitely to be on the Nvidia waiting list?
Field programmable gate arrays
FPGAs (field programmable gate arrays) are reprogrammable integrated circuits that can be configured to meet specific tasks and application needs.
They offer flexibility, can be adapted to different requirements and are cost-effective.
Because FPGAs are efficient at parallel processing, they are well suited for AI and machine learning uses and have remarkably low latency in real-world applications.
An interesting implementation of FPGAs can be seen in the Tesla D1 Dojo chip, which the company released in 2021 to train computer vision models for self-driving cars.
However, a few disadvantages of FPGAs include the high technical expertise required to design the hardware, which can translate into expensive initial purchase costs.
AMD GPUs
In 2023, companies like Meta, Oracle and Microsoft expressed their interest in AMD GPUs as a more cost-effective solution and a way to avoid a potential supplier lock-in with the dominant Nvidia.
For example, AMD’s Instinct MI300 series is considered a viable alternative for scientific computing and AI use.
Its GCN (graphics core next) architecture, which emphasizes modularity and support for open standards, plus its more affordable price make it a promising alternative to Nvidia GPUs.
Tensor processing units
TPUs (tensor processing units) are ASICs (application-specific integrated circuits) programmed to perform machine learning tasks.
A brainchild of Google, TPUs rely on a domain-specific architecture to perform neural networks such as tensor operations.
They also have the benefit of energy efficiency and optimized performance, making them an affordable alternative for scaling and controlling costs.
However, it should be noted that the TPU ecosystem is still nascent and current availability is limited to the Google Cloud Platform.
Decentralized marketplaces
Decentralized marketplaces are also trying to alleviate the limited GPU supply line in their own way.
By taking advantage of idle GPU resources from older data centers, academic institutions, and even individuals, these marketplaces provide researchers, startups, and other institutions with enough GPU resources to run their projects.
Many of these marketplaces offer consumer-grade GPUs that can adequately meet the needs of small to mid-sized AI/ML companies, reducing the pressure on high-end professional GPUs.
Some marketplaces also offer additional options for customers who also want industrial-grade GPUs.
CPUs
CPUs (central processing units) are often considered the underdogs for AI purposes due to their limited throughput and the Von Neumann bottleneck.
However, there are ongoing efforts to figure out how to run more AI-efficient algorithms on CPUs.
These include assigning specific workloads to the CPU, such as simple NLP models and algorithms that perform complex statistical calculations.
While this may not be a one-size-fits-all solution, it is perfect for algorithms that are difficult to run in parallel, such as recurrent neural networks or training and inference recommendation systems.
Rounding up
The scarcity of GPUs for AI purposes may not go away anytime soon, but there is good news.
The continued innovations in AI chip technology indicate an exciting future full of possibilities that will one day make the GPU problem fade into the background.
There is still a lot of potential yet to be unleashed in the AI sector, and we may be on the cusp of the most important technological revolution known to humanity.
Daniel Keller is the CEO of InFlux technologies and has over 25 years of IT experience in technology, healthcare and non-profit/charitable work. He successfully manages infrastructure, bridges operational gaps and deploys technology projects effectively. As an entrepreneur, investor and proponent of disruptive technology, Daniel has an ethos that resonates with many on the Flux Web 3.0 team – ‘for the people, by the people’ – and is deeply involved in projects that uplift humanity.
Featured image: Shutterstock/2Be Graphics/INelson
Credit : dailyhodl.com
