[Market Insights] The Token Factory Era: 6 Shifts in Enterprise AI Infrastructure

Key takeaways

What PwC, McKinsey, Bain, BCG, and KPMG reveal about turning GPU investment into measurable AI output

Enterprises are pouring money into GPU clusters and AI data centers. Yet for many, AI output hasn't kept pace with spending. The culprit is rarely a lack of hardware. It's idle GPUs, fragmented capacity, queued workloads, mixed hardware that can't be managed as one pool, and inference costs no one can trace.

The latest research from five leading consulting firms points to the same conclusion: AI infrastructure is no longer judged by how many GPUs you own, but by how efficiently you turn power and compute into tokens, inference results, and completed tasks. In other words, by how well you run your Token Factory.

• Global data center capex will total US$31.6 trillion between 2026 and 2050, and most of it goes into GPUs and servers that are replaced every 4–6 years (PwC).

• Inference is becoming the operating cost of enterprise AI, and 93% of companies report exceeding their AI budgets (McKinsey).

• Cost per token is falling, but usage is rising faster, so cost per task stays high (Bain).

• Leaders manage token spending toward an explicit return and build on a shared control plane (BCG).

• The fastest way to add capacity is to stop wasting the capacity you already have.

1. The Money Is Flowing into Hardware That Depreciates Fast

AI data center spending has reached the trillions, and the bulk of it goes into equipment with a short shelf life. That makes the utilization of every GPU a direct driver of ROI.

Bain warns that those who wait for the market to sort itself out will get locked out. For most enterprises, the fastest and most controllable "new" capacity is hiding in the idle hours of GPUs they already own.

2. Agentic AI Multiplies Both Value and Cost

Inference is overtaking training as the main consumer of AI compute. McKinsey expects it to account for 30–40% of all data center demand by 2030, and agentic AI is accelerating the shift.

• Value is real but concentrated. Almost 50% of companies now generate measurable value from AI. Those that pair strategic clarity with mature AI capabilities unlock five times as much value as those lacking both (BCG).

• Spending is climbing. AI spending has grown from about 1% to 3.3% of revenue in a year (BCG), and 93% of companies report exceeding their AI budgets (McKinsey).

• Tokens are now a finance issue. Token spend has gone from a rounding error to a material line item, and companies increasingly govern it like any other financial resource (McKinsey).

• Cheaper tokens, same bill. Cost per token is falling, but usage is growing faster; a new agentic workflow can cost 10x more than it should before optimization (Bain).

• Leaders manage, laggards don't. 75% of "future-built" companies manage token spending toward an explicit return, while half of laggards don't track or cap it (BCG).

Bain's advice: measure cost by workflow, not by user. The real question is what it costs to close a proposal or resolve a support ticket. That is where AI FinOps begins.

3. Manufacturing Is Scaling Up on Mixed Hardware

Enterprises are moving from pilots to production just as hardware is becoming more diverse, which makes unified scheduling a new priority for IT.

• Manufacturers are ready to scale. In KPMG's survey of 258 manufacturing tech leaders, 49% already see business value from AI, 68% expect to deploy at scale within 12 months, and 89% say managing AI agents will be a critical skill within five years (KPMG).

• Heterogeneous compute is the norm. Data centers now mix GPUs, CPUs, ASICs, and specialized accelerators, each suited to different workloads, and custom silicon is the fastest-growing chip category (McKinsey, Bain).

• Agents leave accelerators idle. Agents sit idle while waiting on tool calls, which calls for finer-grained scheduling (Bain).

• Data is still the weak link. 76% of manufacturing leaders cite unreliable data as a top AI risk (KPMG).

Without a unified software layer, multiple GPU generations, NPUs, and competing business units quickly fragment into silos.

4. The Metric Is Shifting from GPU Count to Token Output

The KPIs for AI infrastructure are moving from asset scale to output per watt and per dollar. A Token Factory treats the data center like a plant: power and compute go in; tokens, inference results, and completed tasks come out.

As McKinsey puts it, "inference is becoming the operating cost of enterprise AI." Whoever delivers reliably at the lowest cost per token will set the pace for AI at scale.

5. Five Bottlenecks Are Draining GPU ROI

Most enterprises don't have a compute shortage. They have a utilization problem.

The root cause is the same in every case: hardware is allocated by the card, but AI work happens by the task. The fix isn't more cards. It's a software layer that makes compute divisible, poolable, schedulable, and measurable.

6. Training, Inference and Agents Need Different Scheduling

Each AI workload stresses compute differently, so a one-size-fits-all allocation guarantees waste. Bain notes that training, inference, and agentic workloads each favor different hardware and scheduling.

Model strategy adds another layer of complexity:

• Portfolios, not single models. The advantage is shifting from accessing the biggest model to orchestrating a portfolio that balances performance, cost, speed, and governance. SLMs, compression, and open-weight models make on-prem and edge deployment practical (McKinsey).

• Right-size every task. Route each task to the smallest model that meets the requirement, and measure cost per successful outcome rather than cost per token (BCG).

• Physical AI drives training demand. World models and GPU-accelerated simulation let teams train and validate models virtually, dramatically compressing training timelines (McKinsey).

Running all of this on one cluster requires software that partitions, pools, and schedules compute dynamically, based on what each workload actually needs.

How INFINITIX Builds the Token Factory

INFINITIX delivers two platforms that work as one stack: AI-Stack keeps every GPU busy, and ixCSP turns that compute into services people can consume and pay for.

Token Factory architecture · compute plane, control plane, service layer

Because the hardware layer is abstracted, you can refresh or mix GPU vendors while scheduling, governance, and billing stay the same. That is the promise of software-defined infrastructure.

AI-Stack: One Pool for Every GPU

AI-Stack is INFINITIX's GPU resource management platform. It unifies heterogeneous GPUs and NPUs into a single pool and allocates capacity dynamically by workload.

Across customer deployments, AI-Stack lifts GPU utilization above 90% and cuts development environment setup from two weeks to one minute. Recent deployments include:

• Kaohsiung Medical University Chung-Ho Memorial Hospital (KMUH): Taiwan's first hospital to deploy NVIDIA B300 GPUs, using AI-Stack to schedule and govern LLM and vision-language model workloads across clinical care and research.

• Changhua Christian Hospital (CCH): Runs NVIDIA B200 GPUs on AI-Stack, letting multiple medical teams develop, train, and validate models in parallel.

• ELSA Lab, The University of Electro-Communications (Japan): Uses GPU pooling and smart scheduling to give researchers on-demand compute for physical AI.

• Smart Manufacturing Innovation Center (SMIC), Korea: A national testbed built by KETI, running a shared, multi-tenant AI compute platform for manufacturing research, including Manufacturing Foundation Models.

ixCSP: From Compute to Billable Services

ixCSP is INFINITIX's AI cloud operations platform. It forms the service layer of the Token Factory, letting compute be consumed, metered, and billed.

Enterprises use ixCSP to run a private AI cloud with internal chargeback. GPU cloud providers, telcos and sovereign AI centers use it to launch billable AI services and turn capex into recurring revenue.

A Six-Step Adoption Path

1. Baseline: Inventory GPUs and NPUs; measure utilization, queue times, and usage by team.

2. Pool: Unify hardware with AI-Stack and reclaim idle and fragmented capacity.

3. Separate training and inference: Aggregate GPUs across nodes for training; partition GPUs so inference services share capacity by model size and latency target.

4. Govern: Set tenant quotas and roll out AI FinOps dashboards.

5. Productize: Launch GPU, model, and token services on ixCSP.

6. Manage to tokens: Track tokens per watt, cost per token, and cost per workflow to guide future hardware spend.

The Bottom Line: Compete on Tokens per GPU

With power and supply chains limiting new capacity and agentic AI multiplying demand, the highest-return investment is often not the next GPU order. It's getting full, well-matched, and fully accounted-for use out of the GPUs you already own. Hardware depreciates; software-defined management compounds.

As workloads shift from training alone to a mix of training, inference, and agents, and as companies move from one large model to a portfolio of models, the metric is shifting too: from cost per token to cost per successful outcome. The organizations that can dynamically match compute to each workload, and tie every GPU-hour to measurable output, will be the ones that scale.

Founded in 2003 and headquartered in Taipei City, Taiwan, INFINITIX serves more than 250 enterprises and organizations across Asia. Talk to our team to see AI-Stack and ixCSP in action and map out your path to a Token Factory.

References

• PwC, "Where $31.6 trillion of capex flows in the era-defining AI build-out," Global Data Centre Outlook 2026–50, September 2026

• McKinsey & Company, Technology Trends Outlook 2026

• Bain & Company, Technology Report 2026

• BCG, The Formula for Agentic AI Value: The Applied AI Index 2026, September 2026

• KPMG, Global Tech Report 2026: Interoperability, AI, and the new rules of manufacturing excellence