Red Hat, the IBM-owned open source software firm, is acquiring Neural Magic, a startup that optimizes AI models to run faster on commodity processors and GPUs.
The terms of the deal weren’t disclosed.
MIT research scientist Alex Matveev and professor Nir Shavit founded Somerville, Massachusetts-based Neural Magic in 2018, inspired by their work in high-performance execution engines for AI.
Neural Magic’s software aims to process AI workloads on processors and GPUs at speeds equivalent to specialized AI chips (e.g. TPUs). By running models on off-the-shelf processors, which usually have more available memory, the company’s software can realize these performance gains.
Big tech companies like AMD and a host of other startups, including NeuReality, Deci, CoCoPie, OctoML, and DeepCube, offer some sort of AI optimization software. But Neural Magic is one of the few with a free platform and a collection of open source tools to complement it.
Neural Magic had so far managed to raise $50 million in venture capital from backers like Andreessen Horowitz, New Enterprise Associations, Amdocs, Comcast Ventures, Pillar VC, and Ridgeline Ventures.
Red Hat CEO Matt Hicks said Neural Magic’s work on vLLM, an open source project for model serving, was of particular interest to Red Hat. With Neural Magic, the open source company is benefiting from a vLLM-based “enterprise-grade” stack that allows customers to optimize and deploy models across cloud environments, Hicks said, with full control over infrastructure and security.StrictlyVC San Francisco
Mix and mingle with other investors and founders, and hear insights from top-tier VCsSan Francisco | December 4REGISTER NOW
Red Hat is already involved in the vLLM project, which it leverages to run models in products like Red Hat Enterprise Linux AI and Red Hat OpenShift AI. Hick said through the combined work of Red Hat and Neural Magic, its infrastructure partners will be able to better scale AI across platforms, and its integrated service provider partners will gain more robust inference and performance to integrate with their offerings.
“AI workloads need to run wherever customer data lives across the hybrid cloud; this makes flexible, standardized and open platforms and tools a necessity, as they enable organizations to select the environments, resources and architectures that best align with their unique operational and data needs,” Hicks said in a statement. “We’re thrilled to complement our hybrid cloud-focused AI portfolio with Neural Magic’s groundbreaking AI innovation, furthering our drive to not only be the ‘Red Hat’ of open source, but the ‘Red Hat’ of AI as well.”
Red Hat’s acquisition of Neural Magic comes as the company makes a number of AI-related announcements at KubeCon, the annual computing conference, in Salt Lake City this week. Red Hat also unveiled Climatik, a tool developed in collaboration with Intel, Bloomberg and IBM to optimize data center energy efficiency, and new releases of its OpenShift AI and Device Edge development platforms.
As the power demands and costs of creating and deploying AI increase, big tech providers have moved to snatch up firms with products that could help optimize AI algorithms. In April, Nvidia acquired Run:ai, a company that makes it easier for devs to manage and optimize their AI infrastructure. And in October 2020, Intel bought SigOpt, a specialist in modeling optimization.