At the Intel Vision 2024 customer and partner conference, Intel introduced the Intel Gaudi 3 accelerator, designed to enhance enterprise generative AI (GenAI) with improved performance, openness, and choice.
Addressing the challenge of only 10% of enterprises successfully deploying GenAI projects into production, Intel unveiled a range of new scalable systems, products, and collaborations.
Gaudi 3 for AI Training and Inference
The Intel Gaudi 3 accelerator builds upon the success of its predecessor, the Gaudi 2, offering openness and choice to customers. With community-based software and standard Ethernet networking, businesses can scale AI systems more flexibly.
Custom Architecture for GenAI Performance
Manufactured on a 5nm process, the Intel Gaudi 3 accelerator features a custom architecture for efficient AI computation.
Its engines, including Matrix Multiplication Engine (MME), Tensor Processor Cores (TPCs), and Networking Interface Cards (NICs), enable fast and efficient deep learning computation.
Key Features
- AI-Dedicated Compute Engine: Each Intel Gaudi 3 accelerator boasts a heterogeneous compute engine with 64 AI-custom and programmable TPCs and eight MMEs. These engines deliver high computational efficiency, supporting complex matrix operations crucial for deep learning algorithms.
- Memory Boost: With 128GB of HBMe2 memory capacity, 3.7TB of memory bandwidth, and 96MB of on-board SRAM, the Gaudi 3 offers ample memory for processing large GenAI datasets, enhancing workload performance and data center cost efficiency.
- Efficient System Scaling: Integrated with twenty-four 200Gb Ethernet ports, the Gaudi 3 enables flexible and efficient scaling for large compute clusters, eliminating vendor lock-in from proprietary networking fabrics.
- Open Industry Software: Intel Gaudi software integrates with popular frameworks like PyTorch, offering optimized models for GenAI developers and facilitating ease of use and model porting across hardware types.
Gaudi 3 PCIe
The Gaudi 3 PCIe add-in card provides high efficiency with lower power consumption, making it ideal for inference, fine-tuning, and retrieval-augmented generation (RAG) workloads.
With a full-height form factor, 128GB memory capacity, and 3.7TB/s bandwidth, it offers significant performance benefits.
Performance Advantages
The Intel Gaudi 3 accelerator is projected to outperform competitors, delivering:
- 50% faster time-to-train across various models.
- 50% faster inference throughput and 40% greater inference power-efficiency.
- 30% faster inferencing compared to competitors.
Generating Value for Customers with Intel AI Solutions
Intel outlined its strategy to foster an open ecosystem for scalable AI systems, encompassing hardware, software, frameworks, and tools.
This approach enables a wide range of partners, including equipment manufacturers, database providers, and service providers, to offer tailored solutions for enterprise-specific GenAI needs.
Strategic Collaborations
Intel highlighted partnerships with various companies across industries, demonstrating the adoption of Intel Gaudi accelerator solutions for innovative AI applications.
Examples include NAVER for advanced language models, Bosch for smart manufacturing, and IBM for validating platforms with Intel Gaudi accelerators.
Intel partneres with companies in India for AI projects:
- Bharti Airtel: Airtel will use Intel’s tech to boost its AI capabilities, aiming to improve customer experiences and explore new revenue streams.
- Infosys: Infosys joined hands with Intel to integrate Intel’s processors and accelerators into Infosys Topaz, aiming to accelerate business value through AI.
- Ola/Krutrim: Ola and Krutrim are working together to develop India’s first AI model in multiple languages. They are also using Intel’s Gaudi 2 cluster for this project.
Expanded Collaborations
Intel announced collaborations with Google Cloud, Thales, and Cohesity to leverage Intel’s confidential computing capabilities in cloud instances.
This involves technologies like Intel Trust Domain Extensions (Intel TDX) and Intel Software Guard Extensions (Intel SGX) to ensure secure execution environments for AI models.
Ecosystem Rallies to Develop Open Platform for Enterprise AI
Intel, along with numerous partners, announced plans to develop an open platform for enterprise AI.
This initiative aims to create multivendor GenAI systems that offer ease of deployment and superior performance, enabled by retrieval-augmented generation (RAG) techniques.
Intel’s Expanded AI Roadmap and Open Ecosystem Approach
In addition to the Gaudi 3 accelerator, Intel provided updates on its next-generation products across various segments of enterprise AI.
This includes new Intel Xeon 6 processors, Intel Core Ultra processors for client devices, and advancements in edge computing and connectivity solutions.
Intel Tiber Portfolio of Business Solutions
Intel unveiled the Intel Tiber portfolio, aiming to streamline the deployment of enterprise software and services, including for GenAI applications.
This unified experience prioritizes security, compliance, and performance, providing customers with accessible solutions to unlock business value.
Availability
Original equipment manufacturers (OEMs) can access the Intel Gaudi 3 accelerator in the second quarter of 2024, with general availability expected in the third quarter.
The Intel Gaudi 3 PCIe add-in card will be available in the last quarter of 2024. Notable OEM adopters include Dell Technologies, HP Enterprise, Lenovo, and Supermicro.
Additionally, the Intel Gaudi 3 accelerator will power cost-effective cloud LLM infrastructures, including those of NAVER, providing organizations with price-performance advantages.
Commenting on the launch, Intel CEO Pat Gelsinger said:
Innovation is accelerating at an unparalleled rate, driven by silicon – and every organization is rapidly transitioning into an AI-centric entity. Intel is democratizing AI adoption across all levels of enterprise, spanning from the PC to the data center to the edge. Our newest Gaudi, Xeon, and Core Ultra platforms offer a unified suite of adaptable solutions, meticulously designed to accommodate the evolving demands of our valued customers and partners, and seize the abundant opportunities on the horizon.