Meta is Building an AI Supercomputer
Reading Time: 2 minutes

Meta announces it will build a new AI Research SuperCluster, an AI supercomputer for AI research projects. After working on the supercomputer for almost two years it will help Meta develop more powerful AI software, perhaps will help identify hate speech in posts. 

According to Shubho Sengupta, a software engineer at Meta, “With RSC, we can more quickly train [AI] models that use multi-modal signals — for example, to understand the context of a post, including language, images, and tone.” 

What are Supercomputers?
In recent times supercomputers have become popular especially with AI research. Supercomputers feature many interconnected processors clustered together as nodes. Summit, a supercomputer with the US Department of Energy is the fastest supercomputer in the United States and is the second-fastest in the world. It helps carry out investigations into things like unfamiliar proteins. Large tech companies like Microsoft and Nvidia also have their own supercomputers. 

Sengupta from Meta in a blog post said, “AI supercomputers are built by combining multiple GPUs into compute nodes, which are then connected by a high-performance network fabric to allow fast communication between those GPUs. RSC today comprises a total of 760 NVIDIA DGX A100 systems as its compute nodes, for a total of 6,080 GPUs — with each A100 GPU being more powerful than the V100 used in our previous system. The GPUs communicate via an NVIDIA Quantum 200 Gb/s InfiniBand two-level Clos fabric that has no oversubscription. RSC’s storage tier has 175 petabytes of Pure Storage FlashArray, 46 petabytes of cache storage in Penguin Computing Altus systems, and 10 petabytes of Pure Storage FlashBlade.”

Early tests suggest supercomputers can train large language models three times faster than the system currently in use. This means an AI model can take 9 weeks to train on the existing system while it can be trained in 3 weeks using the supercomputer. 

According to Meta, it hopes the new supercomputer will eventually be able to train AI models with trillions of parameters. There are not many known existing AI models of that scale and also it is larger than GPT-3, a large language model from OpenAI. It can generate human-sounding text and is being used for applications like language-learning and tax software for freelancers.

Meta in the end said supercomputers will lead to technologies required to build the so-called metaverse. It is a wide-ranging interconnected virtual work where people can communicate via digital avatars, with others virtually. 

Related Articles:
Brave Browser Now Offers Brave Wallet to Manage your Crypto Portfolio
Exchange Reddit Karma Point for Crypto
Brand Rush in Metaverse – Companies Exploring the Virtual Universe