To stop the secret sauce from leaking, don’t bring the data to the GenAI model, bring the GenAI model to the data, Expedient’s SVP for artificial intelligence Bradley Reynolds told CRN.
“We want to take the fear of adopting AI because of data leakage and data governance, compliance, we want to take that out of the equation,” said Reynolds (pictured above). “The most secure way is, ‘Here’s your model XYZ company. Here’s your data connectors, and it all runs inside a walled garden at Expedient.’ That’s the most secure. But that’s just one gradient.”
The Pittsburgh-based co-location and cloud services provider said its secure AI gateway has a few ways to address security concerns of customers who are working at AI and machine learning implementation, but fear compromising their data for security and compliance reasons. Expedient can host a “walled garden” for them with the model and data all local. It can offer hybrid access to models, or access to public models with a “security wrapper.”
[RELATED: 5 Cool AI And HPC Servers With AMD Instinct MI300 Chips]
“We have security wrappers that say when you use our thing and we log it, it can go out to OpenAI, or something similar, you can use one of those tools as well,” Reynolds said.ADVERTISEMENT
Data security is the top concern that stops customers from iterating on their data with large language models (LLMs), according to 451 Research’s “Voice of the Enterprise: AI & Machine Learning, Infrastructure” study.
And with good reason, Reynolds told CRN, the more the data moves from the servers hosting it, the higher the risk of the organization running afoul of data regulations around governance and compliance while also increasing the chance of leakage.
“It gets further out of the client’s enterprise control and they have to determine on the continuum of risk tolerance, where they want to be, with one side being no AI, and the other being, ‘use any tool you want,” Reynolds told CRN. “If they want to be super secure, sure. Here’s a dedicated Llama-3 instance, and a bunch of data connectors inside of Expedient. Let’s go. Let’s rock and roll.”
Expedient, which has 472,000 square feet of data center space in 15 locations nationwide, said its AI gateway also offers customizable configurations that can preemptively halt the leakage of sensitive information or personally identifiable data into public AI models. Users also gain secure entry to sanctioned AI models through single-sign-on authentication, integrated with organizational user directories for oversight by administrators.
Role-based access control, inside the product gives organizations gates around data and AI model accessibility, tailoring permissions according to users’ roles. The product includes observability tools for point of usage metrics while also pinpointing potential security vulnerabilities.
Reynolds said the Expedient’s Secure AI Gateway can accommodate all of the leading AI platforms, so customers can inference their own data, against OpenAI’s GPT-4, Meta’s LLAMA, Anthropic’s Claude, Mistral’s Mixtral, and Cohere’s Command.
“That we download. We run it on top of our orchestration layer, so that our client, what they get is an API endpoint,” Reynolds said. “But underneath, we’ve abstracted all the complications of how do you host thessece servers, and how do you scale it up, and how do you scale it down. And how do you make it performant.”
He said while anyone can buy an endpoint from OpenAI, it’s not private and it’s not controlled. He said with Expedient, users can take advantage of multiple AI resources within a single framework. The API access gives developers the chance to craft custom applications tailored to their businesses.
Expedient guarantees reliable, secure, and low-latency connectivity inside of its 15 US data centers offering colocation. Reynolds said combining that coverage with a locally hosted AI model also gives it a significant speed boost.
“The notion of giving them an easy, AI enabled platform in their data center, where their data is already located, enables them to spider and connect to all of that data that’s already with us, really fast. I mean you are talking racks away. It allows you to spin up applications a lot more efficiently than if it’s out in the cloud.”
The Secure AI Gateway is Expedient’s first offering in its AI CTRL family of AI tools to unlock the benefits of AI.
“We have more to come throughout 2024,” Expedient CEO Bryan Smith said in a statement.
The AI CTRL category of products that are designed to help customers seize the promise of GenAI while relieving common pain points, he said.
Among the AI CTRL features expected this year are real-time data accessibility for diverse data sources including databases, SaaS platforms, and spreadsheets. Additionally, the integration of vector databases is expected to improve the efficacy of AI models, elevating their performance, the company said. Expedient said the adoption of Retrieval-Augmented Generation, gives users more accurate responses by mitigating false positives which dramatically reduces hallucinations.
Expedient is a technically sophisticated VMware partner and one of the just 12 in the US to be awarded the top Pinnacle status as a VMware Cloud Service Provider in the Broadcom Advantage Partner Program. In 2021, Expedient introduced its Expedient Cost CTRL Suite, a technology aimed at helping businesses understand their multi-cloud costs and how to better optimize their cloud spend.