By admin 
In a rapidly evolving landscape of artificial intelligence, enterprises are exhibiting a striking disconnect between their aggressive investments in AI infrastructure and their ability to comprehend, manage, and optimize its economic implications. A comprehensive new report from VentureBeat Pulse Research, surveying 107 enterprises with over 100 employees, reveals a significant "compute gap" where substantial spending is outpacing the necessary visibility into costs and returns. This situation is particularly acute as organizations are increasingly looking to adopt specialized AI compute solutions, even as their current GPU resources remain underutilized and their cost-tracking mechanisms lag behind their investment pace.
The research highlights that while most organizations currently rely on familiar hyperscale cloud providers and established model-provider APIs for their AI operations, the trajectory of future spending is sharply aimed at specialized compute infrastructure – a domain that a majority of these same enterprises are not currently leveraging. This indicates a significant potential shift in the AI infrastructure market, with a clear intent among a majority of respondents to either switch or add new providers within the next twelve months, and a substantial portion planning such changes within the next quarter. Crucially, these purchasing decisions are being driven not by headline token prices, but by factors such as integration with existing technology stacks and a comprehensive total cost of ownership (TCO). This focus on TCO is somewhat paradoxical, given that most enterprises struggle to clearly define their unit economics for AI compute. The data reveals a concerning reality: GPUs are operating at half utilization or less, and fewer than half of organizations are rigorously tracking the actual costs associated with their AI compute resources. The overarching consequence is a widening "compute gap" – a scenario where heavy, fast-paced investment in AI infrastructure is occurring without the requisite visibility and control mechanisms to steer its economic outcomes effectively.
The VentureBeat Pulse Research initiative, in its latest wave focused on enterprise AI infrastructure and compute economics, delves into several critical areas. It examines the current deployment stage of AI within organizations, their existing infrastructure choices, levels of satisfaction, the drivers for potential vendor switching, and, most revealingly, their capacity to measure and control the economics of the underlying compute. The central finding, the aforementioned compute gap, underscores the distance between the aggressive pace of enterprise investment in AI infrastructure and the limited visibility into its economic realities.
Adding further detail to this finding, the report indicates that only about one in five enterprises (21%) are currently running AI in production at scale. Despite this relatively early stage of maturity for many, their spending intentions are already outpacing their current deployment levels. The single largest area that enterprises plan to evaluate over the next year is AI-specialized clouds (45%), a category that almost none of these organizations are utilizing today. This points to a proactive, perhaps even speculative, investment strategy. Compounding this, the compute infrastructure already in place is running inefficiently. A staggering 83% of respondents report that their GPU utilization is 50% or less, and a concerningly low 44% can rigorously track the actual costs of their AI compute. This suggests enterprises are acquiring more infrastructure at a faster rate than they can accurately account for what they already own.
The landscape of AI infrastructure providers is also far from settled. A clear majority of enterprises (64%) plan to switch or add an infrastructure provider within the next twelve months, with a significant 38% indicating such changes within the next quarter. This level of intended churn is unusually high for a category as foundational as compute infrastructure, suggesting a market in flux. When these enterprises make their provider selections, the primary considerations are integration with their existing technology stack (41%) and total cost of ownership (35%), rather than headline pricing. The cost per million tokens, a metric often emphasized by vendors, is a deciding factor for a mere 8% of respondents. Furthermore, a critical frontier constraint that is poised to shape future AI compute decisions – the shift from GPU compute to memory bandwidth as inference workloads scale – is barely registering on the radar for many organizations, with approximately one in five enterprises either unaware of this impending bottleneck or yet to address it.
Methodology Insights
The survey, conducted by VentureBeat as part of its ongoing Pulse Research series, focused specifically on enterprise AI infrastructure, compute, and inference economics. The findings are based on responses from 107 organizations with more than 100 employees, excluding the smallest size band (1-100 employees). The data was collected in a single wave during Q2 2026 (June), providing a cross-sectional snapshot of the market rather than inferring month-over-month trends. Some questions allowed for multiple selections, hence percentage shares may sum to more than 100%.
The sample distribution by organization size shows a concentration in the mid-market: 36% fall within the 101-250 employee range, and 27% are between 251-1,000 employees. Larger enterprises are also represented, with 22% in the 1,001-5,000 range, 8% in the 5,001-10,000 range, and 7% with over 10,000 employees. By role, the respondent base includes managers (38%), individual contributors (28%), VPs and directors (19%), and C-suite executives (13%). Regarding purchasing authority, the sample is highly credible for AI solutions, with 45% identified as final decision-makers and an additional 30% acting as recommenders or influencers. The largest industry represented is Technology/Software (26%), followed by Healthcare/Life Sciences (15%), Financial Services (13%), and Retail/E-commerce (12%).
With 107 respondents, the sample size is sufficient for directional insights but should be interpreted as a signal rather than a precise measurement. The self-selected nature of the survey and its skew towards the mid-market and earlier-stage AI adopters mean it best reflects the perspective of organizations actively building out AI infrastructure, rather than that of the largest hyperscale operators.
Finding 1: Ambition Outpaces Production
The current state of AI deployment within enterprises reveals a maturity curve that is heavily front-loaded. Three-quarters of organizations (76%) are either in the experimentation phase, running proofs of concept, or have some workloads in production but not yet across the entire organization. Only a minority, 21%, report running AI in production at scale. This data point is critical, as it suggests that the infrastructure decisions being made today are largely by organizations still in the nascent stages of their AI journey. Their compute footprints and associated costs are poised for significant growth. The strong intentions to evaluate and switch providers, as detailed in subsequent findings, represent the leading edge of this build-out, rather than the settled preferences of mature AI operators.
Finding 2: Enterprises Run on Hyperscalers and Model APIs – For Now
The current AI infrastructure landscape for these enterprises is characterized by a reliance on established hyperscale cloud providers and prominent model APIs. Google Cloud emerges as the most-used platform, with 48% of respondents utilizing it, followed by Microsoft Azure (29%), AWS (22%), and Oracle Cloud (22%). In terms of model providers, Google’s Gemini models are used by 41%, with OpenAI closely behind at 40%, and Anthropic at 12%. In stark contrast, the specialized AI cloud providers that have garnered significant attention in the industry, such as CoreWeave, Lambda, Crusoe, Nebius, and Together, register at or near zero usage among these surveyed enterprises. Only a small fraction (6%) are running their own on-premises or co-located GPU clusters, and 4% are utilizing a custom open-source self-managed stack. This indicates that, for the moment, enterprises are leveraging the providers they are already familiar with for their AI initiatives.
A note on interpreting these shares: As per the methodology, this sample is self-selected and skews mid-market. The question counted every provider used, with respondents averaging 2.1 selections each. Therefore, these figures represent presence within an organization’s AI stack rather than primary status or spending weight. Google’s strength here is consistent with its historical position among smaller enterprises adopting AI. These shares offer a portrait of what this AI-active cohort is running today.
Finding 3: The Next Dollar Targets Unused Infrastructure
The most striking tension in the report emerges when examining future evaluation plans. The single most cited area for planned evaluation over the next 12 months is AI-specialized clouds (45%), a category that, as previously noted, is almost entirely absent from their current usage. This suggests a significant potential re-platforming effort is on the horizon. Furthermore, nearly a third of respondents (32%) intend to evaluate non-NVIDIA accelerators, and 28% are looking into next-generation NVIDIA silicon. Even emerging areas like decentralized or distributed compute networks (16%) and sovereign or region-specific compute solutions (11%) are drawing meaningful interest. When viewed against current usage, this indicates not just incremental adjustments, but a fundamental re-evaluation of AI compute strategies. The direction-of-travel question reinforces this, showing that specialized AI clouds have the highest net momentum (+24), slightly edging out even the hyperscalers (+22). Enterprises are actively preparing to shift a substantial portion of their AI compute away from general-purpose cloud environments. This trend aligns with previous survey waves, where specialized AI clouds were identified as the most sought-after emerging compute option for evaluation.
Finding 4: A Wave of Provider Switching is Building Momentum
The AI infrastructure market is characterized by a notable degree of intended movement among providers. A clear majority (64%) of enterprises plan to switch or add an infrastructure provider within the next twelve months, with a substantial 38% intending to do so within the next quarter alone. This level of planned disruption is significant for such a foundational technology category. Only 36% of respondents have no plans to change their current provider. While a significant portion of this near-term movement (33% each) involves switching between incumbent providers like Microsoft Azure and Google Cloud, or consolidating with model providers like OpenAI (30%) and Gemini (22%), it underscores a dynamic market seeking optimal solutions. The interest in specialized AI clouds, as highlighted in Finding 3, represents a longer-term evaluation thesis, while the immediate switching activity appears to be more focused on optimizing within the existing major provider ecosystem.
Finding 5: Price is Not the Primary Driver; Integration and TCO Reign Supreme
Enterprises are not making AI infrastructure purchasing decisions based on headline pricing, a key battleground for many vendors. The most critical factor influencing provider selection is integration with the existing cloud and data stack (41%), followed closely by total cost of ownership (35%). Performance metrics like latency and throughput are cited by 24%, while security/compliance, autoscaling for spiky workloads, and GPU access/availability are each mentioned by 19%. The least influential factor, cited by only 8% of respondents, is the cost per million tokens. This pattern is consistent: buyers are prioritizing how a provider fits into their existing infrastructure and the true operational cost, rather than the advertised unit rate. This emphasis on TCO, however, is somewhat at odds with the finding that a majority of enterprises struggle to rigorously measure it, creating a potential disconnect between stated priorities and actual measurement capabilities.
Finding 6: Expensive GPUs Sit Idle Most of the Time
The report reveals a significant inefficiency in current AI compute utilization. A substantial 83% of enterprises operating GPUs report utilization levels of 50% or less. Delving deeper, 37% are operating at 26-50% utilization, 34% are at 10-25% utilization, and 15% are running at under 10% utilization. Only a small minority (12%) report utilization exceeding 50%. An additional 8% do not measure their utilization at all, and 7% consume compute solely via APIs, meaning they do not operate GPUs directly. This widespread underutilization of expensive GPU hardware represents a significant cost inefficiency. It underscores the compute gap by highlighting that enterprises are planning to acquire more advanced compute resources while the capacity they already own remains substantially idle and largely unmeasured. The potential for efficiency gains within the existing GPU fleet is substantial but largely untapped.
Finding 7: Rapid Spending Outpaces Slow Measurement
The ability of enterprises to quantify the cost and return on their AI infrastructure investments is lagging behind their spending pace. Fewer than half of organizations (44%) rigorously track the cost and return of their AI compute. A significant 39% track it only partially, 20% cannot quantify it yet, and 6% indicate it is not a priority. This measurement gap is particularly consequential given that total cost of ownership was identified as the second-highest buying criterion. Enterprises are selecting providers based on economic factors that they are largely unable to measure comprehensively. While overall satisfaction with current AI infrastructure is moderately positive (averaging 4.0 on a five-point scale), specific aspects like ease of implementation (3.8) and value for money (3.9) trail slightly, with cost being the weakest point. This suggests that enterprises are spending rapidly on AI, but their accounting and measurement practices are not keeping pace.
Finding 8: The Next Bottleneck is Largely Unseen
As AI inference scales, the binding constraint is shifting from raw GPU compute to memory bandwidth, particularly KV-cache capacity. However, this emerging bottleneck is not yet a widespread concern or priority for most enterprises. When asked about their approach to this shift, respondents’ answers are fragmented. Dell is cited by 31% as a potential solution provider, followed by NVIDIA at 16%. A significant 18% are not aware of this constraint or have not yet addressed inference-memory limits. Other solutions are scattered across storage vendors like Hammerspace (10%) and DDN (9%), as well as open-source KV-cache tooling, model-level efficiency techniques, VAST Data, and WEKA. The fact that nearly one in five enterprises are unaware of or have not addressed this critical shift indicates an early and unsettled market for inference memory solutions. This mirrors the broader measurement gap identified in Finding 7, suggesting that many enterprises are still grappling with existing compute economics before fully confronting the next wave of infrastructural challenges.
The Bottom Line: A Compute Gap Widened by Unseen Spending
Organizations with over 100 employees are demonstrating a strong appetite for AI infrastructure investment, outpacing their ability to accurately measure and manage its economic implications. A majority of these enterprises are still in the early stages of AI deployment, yet their spending intentions are clearly directed towards specialized cloud solutions and alternative accelerators that they are not currently utilizing. This proactive approach is coupled with a significant intent to alter their provider landscape within the next year, with many planning changes in the immediate quarter. Their purchasing decisions are pragmatic, prioritizing integration and total cost of ownership over mere headline price. However, the inherent challenge lies in the fact that most organizations lack the clear economic visibility required to truly assess and optimize these TCO considerations.
The scope of this visibility gap is starkly illustrated by the underutilization of existing GPU capacity, with the vast majority of enterprises reporting less than 50% utilization. Furthermore, fewer than half possess the rigorous tracking mechanisms needed to ascertain the true costs and returns of their compute expenditure. While overall satisfaction with current AI infrastructure is generally positive, it is tempered by a softness on value for money – precisely the dimension most impacted by a lack of measurement. The forthcoming constraint in large-scale inference, the transition from compute to memory, is also largely unrecognized by a significant portion of the enterprise landscape. While this report, based on a directional read from 107 respondents skewed towards the mid-market and earlier adopters, indicates a clear trend, the fundamental message is consistent: the drive to spend on AI is far outpacing the instrumental capabilities to spend wisely. The compute gap is not merely a matter of insufficient hardware; it is primarily a deficit in visibility into the true cost of the hardware already in place. The critical question for future research waves will be whether enterprises can bridge this visibility gap before the next wave of infrastructure re-platforming arrives, or if they will continue to acquire new layers of technology with the same economic opacity that characterizes their current investments.
