courtesy: Digital India
At technology conferences, it helps to ask the people who build and run the systems what holds everything else back. Infrastructure teams have a particular vantage point. They see what breaks, what gets stuck, and the distance between what was designed on a whiteboard and what survives production.
At the India AI Impact Summit 2026, a session on scaling AI infrastructure and security did exactly this. Rahul Bhattacharya of EY Global Delivery Services, moderating a panel with Lakshmi Das of Prophaze and Clay Hoy and Mrinal Mathur of Arista Networks, asked the audience to choose the biggest barrier to scaling AI: compute, networking, data pipelines, security, or organisation and operating model.
The infrastructure specialists chose the operating model.
That is worth taking seriously. The people who build AI networks for a living, who manage GPU clusters, secure API endpoints and troubleshoot packet loss across data centres, believe the main obstacle is organisational. Who does what. Who owns the risk. How decisions are made about platforms, governance, data and measurement. In their view, the technology is solvable. The organisation around it is where deployments stall.
The technology is moving faster than the organisations using it
The session gave a technically grounded picture of where AI infrastructure stands. Hoy described the physical reality of large deployments: a 10,000-GPU cluster needs roughly 30,000 cables, and if even 5% of them fail, effective bandwidth can halve. Data centres are spreading across sites because of power and cooling limits, not design preference. Ethernet is displacing proprietary interconnects, which matters because open standards let teams reuse equipment as needs change.
Mathur, covering India and the wider region, noted that adoption is slower than global benchmarks, partly because organisations are repeating mistakes other markets have already worked through. Network engineers need new skills for AI workloads, and many existing data centres lack the power density AI requires.
Das brought the security view, describing how behavioural and anomaly-based defences held critical infrastructure through sustained attacks in recent years. Across all three perspectives, the technology is advancing on known paths. The unsolved problems sit in the organisations that deploy it.
Systems mirror the organisations that build them
In 1968, the computer scientist Melvin Conway observed that organisations design systems that copy their own communication structures. Separate teams produce separate components that meet awkwardly at the boundaries. The observation, now known as Conway's law, explains much of what the panel described.
Hoy gave a precise example. Machine learning observability and network observability are usually handled by different teams with different tools. When a training job runs slowly, nobody can see the full path from one GPU to another and everything in between, so troubleshooting happens in the dark. The technical fix is to combine the views. The organisational fix, getting two teams with different reporting lines to work as one, is harder.
Skelton and Pais's work on team design offers a way through. It distinguishes teams aligned to a flow of work, which own an outcome end to end, from platform teams that provide shared capabilities. AI deployments stall when nobody owns the end-to-end outcome. A technology team builds the system, IT provides infrastructure, legal reviews compliance, a business unit defines the use case, and leadership approves the budget. Each does its part. Nobody owns the whole. The pilot works because a small team can hold the picture in their heads. Production fails because the organisation cannot.
The questions that decide success are organisational
Bhattacharya, who works with organisations scaling AI, named the operating model as the persistent bottleneck. The questions that determine success are not technical. What do we want AI to do, and why does it matter? How will we measure value? Who is responsible for what? Which platforms do we need? What governance is required? Who owns the data?
These are questions of strategy, accountability, measurement and decision rights. Most AI deployments defer them, or spread them across too many people for anyone to answer them coherently.
Security cannot be a gate at the end
Das made a point that applies far beyond security. Organisations move fast to build and pilot AI, then add security at the end, as a gate before production. It fails, because security added to a system designed without it is fragile. Her image was a ship: you cannot add watertight compartments after the hull is built.
Security engineering has argued this for decades. Saltzer and Schroeder's 1975 design principles, including least privilege and fail-safe defaults, assume security is part of the architecture. The US Cybersecurity and Infrastructure Security Agency's Secure by Design initiative restates it for modern software. For AI, the attack surface grows quickly: application interfaces multiply, forgotten ones linger, and prompt injection or quiet data leakage can go unnoticed without behavioural monitoring.
The same principle applies to governance, compliance, data residency, auditability and human oversight. Each degrades when treated as a later phase. Adding it afterwards is more expensive, more disruptive and less effective than designing it in.
The panel's discussion of sovereignty made the point concrete. India's DPDP Act, SEBI and RBI requirements and telecom security rules demand architectural answers. Mathur described sovereignty as transparent architecture, trusted supply chains, auditable network layers and standards-based design that avoids dependence on any one supplier. Governance and sovereignty are design decisions, made or missed at the start.
If you cannot see it, you cannot run it
The panel agreed on one investment priority: observability. Hoy put it simply: if you cannot see it, you cannot troubleshoot it, and you are troubleshooting all the time.
Google's site reliability engineering practice turned this into a discipline. Teams define service level objectives, the reliability a system must deliver, and monitor against them continuously, with clear rules for what happens when a system falls short. The same discipline fits AI. Define what good looks like for each workflow, watch quality, cost and behaviour in production, and agree in advance who acts when something drifts.
Without observability, there is no governance. Without governance, there is no scale. And without scale, the returns that make AI worth the investment never arrive.
Questions to settle before scaling
Who owns the end-to-end outcome of each AI workflow, across teams?
Which decisions does the system make, which does it propose, and who approves them?
How are security, data residency and audit designed in, not added later?
What does good look like in production, and how will it be monitored?
When something drifts, who is told, and who decides what to do?
How Mitochondria's agentic AI systems resolve the operating model
The approach. Every engagement starts with the operating model, not the architecture. Stimuli, the first deployment stage, maps how the workflow runs today: who is involved, who makes which decisions, what moves between which systems, and where things break. Security, governance and data residency are part of the design from the first conversation: systems work with client systems through APIs, process data transiently and keep every action on record.
What it enables. The operating model is settled progressively rather than all at once. Each unit starts with a limited scope and full human oversight. As it proves reliable, autonomy widens, and roles, measures, and governance become clear and habitual. Full interaction records and monitoring make every decision visible from day one.
The impact. Deployments that scale because the organisation around them is ready, with one clear owner for each outcome and a system the organisation can see into and trust.
Read more about Mitochondria's approach.
Sources
Session on scaling AI infrastructure and security, India AI Impact Summit 2026, as presented
Conway, M. E. (1968). How do committees invent? Datamation, 14(4).
Skelton, M. and Pais, M. (2019). Team Topologies.
Saltzer, J. H. and Schroeder, M. D. (1975). The protection of information in computer systems. Proceedings of the IEEE, 63(9).
Beyer, B., Jones, C., Petoff, J. and Murphy, N. R. (2016). Site Reliability Engineering: How Google Runs Production Systems.
—
Mitochondria builds agentic AI for evidence-led core operations. Research-led and model-agnostic. ISO/IEC 27001:2022 certified, with controls aligned to the GDPR, UK GDPR, India's DPDP Act and Australia's Privacy Act. Amsterdam and Pune. Request a working session →