Inference—the process of applying trained AI models to real-world tasks—requires low latency and high processing efficiency
Akamai (NASDAQ: AKAM), a cybersecurity and cloud computing company, has launched Akamai Cloud Inference, a new solution designed to enhance AI efficiency by enabling faster and more cost-effective inference processes. The platform, which runs on Akamai Cloud, aims to address the growing challenges of centralised cloud models by bringing AI inference closer to users and devices.
Addressing AI Inference Challenges
Inference—the process of applying trained AI models to real-world tasks—requires low latency and high processing efficiency. However, traditional centralised cloud infrastructure struggles to keep up with these demands. Akamai’s solution leverages its highly distributed cloud network to run inference at the edge, improving speed, scalability, and cost-effectiveness.
Adam Karon, Chief Operating Officer and General Manager, Cloud Technology Group at Akamai, highlighted the importance of this shift:
“Getting AI data closer to users and devices is hard and it’s where legacy clouds struggle. While the heavy lifting of training LLMs will continue to happen in big hyperscale datacentres, the actionable work of inferencing will take place at the edge where the network Akamai has built over the past two and a half decades becomes vital for the future of AI and sets us apart from every other cloud provider in the market.”
Akamai Cloud Inference: Key Features
Akamai Cloud Inference offers a comprehensive AI infrastructure that includes:
Compute Power: Akamai Cloud provides versatile compute options, including CPUs for fine-tuned inference, GPUs optimised through Nvidia’s AI Enterprise ecosystem, and ASIC VPUs for high-efficiency AI processing.
Advanced Data Management: Partnering with VAST Data, Akamai ensures real-time data access for faster inference. The platform supports highly scalable object storage and integrates with leading vector databases, such as Aiven and Milvus, enabling retrieval-augmented generation (RAG) for AI applications.
Containerisation and Kubernetes Support: AI workloads are containerised for flexibility, efficiency, and scalability. Akamai leverages Kubernetes and its Linode Kubernetes Engine – Enterprise to optimise AI performance at scale. The Akamai App Platform also supports open-source Kubernetes projects, streamlining AI model deployment.
Edge Computing with WebAssembly (WASM): Akamai integrates WASM capabilities in partnership with Fermyon, enabling low-latency AI inferencing directly from serverless applications. This approach enhances real-time AI performance for latency-sensitive use cases.
These capabilities allow businesses to deploy AI models at a lower cost, with up to 86 per cent savings compared to traditional hyperscaler cloud infrastructure. The platform also delivers 3x better throughput and reduces latency by up to 2.5x, ensuring a faster and more efficient AI experience.
AI Inference: Next Frontier
As AI adoption evolves, businesses are shifting focus from training large AI models to deploying lighter, industry-specific AI solutions that deliver immediate and actionable results. While large language models (LLMs) are powerful for general-purpose tasks like summarisation and translation, they require significant computational power and investment. In contrast, smaller AI models can be optimised for specific industries, providing a better return on investment and faster decision-making.
Akamai Cloud Inference aligns with this industry shift, enabling real-time, AI-powered decision-making at the edge. Gartner predicts that by 2025, 75 per cent of data will be generated outside of traditional data centres, increasing demand for AI solutions that process data closer to its source.
Real-World Applications & Industry Adoption
Enterprises are already exploring AI inference for a variety of real-world applications, including:
In-car voice assistance
AI-powered crop management
Image optimisation for e-commerce
Virtual garment visualisation for online shopping
Automated product description generation
Customer feedback sentiment analysis
By leveraging Akamai’s distributed cloud network, businesses can ensure faster, more secure, and cost-effective AI operations, no matter where their users are located.
Summing up the importance of inference in AI’s future, Karon explained:
“Training a LLM is like creating a map – requiring you to gather data, analyse terrain, and plot routes. It’s slow and resource-intensive, but once built, it’s highly useful. AI inference is like using a GPS, instantly applying that knowledge, recalculating in real time, and adapting to changes to get you where you need to go. Inference is the next frontier for AI.”
As businesses move beyond AI hype to practical implementation, Akamai Cloud Inference positions itself as a leading solution for organisations aiming to scale AI efficiently and cost-effectively.