The Invisible Architecture: Why 173 Microsoft 365 Tenants Fell and What the Backup Industry Refuses to Confess
CryptoZoe
There is a specific kind of silence that follows a breach like this one. It is not the silence of a company that does not yet know the scope of what happened; it is the silence of an industry that understands perfectly well what happened and recognizes itself in the mirror. On the fifteenth of October, in the year two thousand twenty-six, one hundred seventy-three Microsoft 365 tenants were walked through by an application they had never directly installed, using credentials that no customer had ever been asked to approve, over a window lasting two hours and twenty-one minutes. The application was not an attacker's tool. It was ConnectWise Cloud Backup, a product that does exactly what it was purchased to do: read everything. The data hides what the eyes refuse to see, and what the eyes have refused to see for a decade is that the backup category has been quietly accumulating the most concentrated data-access privileges in the entire SaaS economy, and calling this accumulation a feature.
I have spent the better part of the past eighteen months modeling liquidity and access concentration in infrastructure businesses. The intellectual habit such modeling leaves behind is a specific kind of coldness — an unwillingness to accept a product's self-description at face value, an insistence on tracing the actual flows. When you trace the flows in the enterprise backup business, you discover that the category has inverted itself without announcing the inversion. What was once a passive recovery mechanism is now a persistent, always-on, cross-tenant read surface into every mailbox, every document library, every chat thread, and every directory object in every customer organization. The product did not change its name when it did this. It simply changed the verb: from “restore” to “read.”
Let me establish the essential context, because the technical detail here matters more than most coverage has allowed. The credential that was abused was not a password. It was not a user account. It was a client assertion — a private_key_jwt, a signed statement of application identity in the OAuth 2.0 client credentials flow. This distinction is not academic; it is the entire structure of the failure. In this model, an application authenticates to Microsoft Entra ID by signing a JWT with a private key stored somewhere on the vendor's side. No MFA challenge occurs because no human is involved. No conditional access policy fires because the signer presents itself as a workload, not a person. No customer receives a consent prompt because the consent was granted once, at the MSP level, in a configuration workflow that nobody remembers completing. The application simply asserts, and the assertion is accepted, and the tenant opens like a door with a worn latch.
The attacker did not break into Entra. Understanding this is essential to understanding the economics of the incident, and the economics are the reason this will happen again. To violate Entra you need resources approximating a nation-state — years of research, zero-day depth, an operational budget measured in tens of millions. To steal a client assertion you need to compromise one of the places a vendor stores a private key: a continuous integration variable, a key management service entry, a configuration file, a developer's laptop, the third-party vault of a subcontractor nobody audits. The cost differential between these two attack paths is not a matter of degree; it is a matter of orders of magnitude. This is the fundamental economic law of the software supply chain, and it has never been more brutally illustrated: attack the weakest link, harvest every downstream tenant. The supply chain is not a metaphor for weakness. It is the mechanism by which weakness is multiplied.
The behavioral fingerprint of the intrusion tells us more than the vendor's disclosures have. According to the technical record, the actor traversed tenants in GUID-sorted order, at intervals of several seconds, within a bounded window. There are three facts embedded in that sentence, and each of them is worse than the last. First, GUID-sorted traversal is not what online enumeration looks like. Enumeration produces noise — retries, misses, wild guesses against malformed identifiers. Ordered traversal at a constant interval is what batch processing against a pre-compiled target list looks like. The actor had a list.
Second, that list existed before the intrusion began. The tenant inventory was not discovered during the operation; it was an input to the operation. Which means the compromise extended beyond the private key into whatever system held the roster of protected customers — a customer database, a provisioning system, or possibly the application's own directory read permissions used to enumerate its own footprint. Third, and most consequential: the two-hour and twenty-one-minute window across one hundred seventy-three tenants is not long enough, at any realistic bandwidth, to exfiltrate the mailboxes, document libraries, and chat archives of one hundred seventy-three organizations. The math does not permit it. What this event actually represents is a permission verification and a persistence implant — the reconnaissance and the staging of a campaign whose data harvest either has not yet happened or has not yet been detected. Coverage that treats this as a data breach with a known scope has misread the shape of the thing entirely.
The persistence mechanism deserves its own paragraph, because it is where the operation stops resembling opportunism and starts resembling a plan. The application, apparently possessing the ability to grant itself the Global Administrator role, did so. It then executed a script — Python, per the reporting — that added new client secrets to two other applications: one managed service provider administration application, and Huntress Security Platform. The secrets were configured to expire in 2028.
Consider each element. An application that can promote itself to Global Administrator inside a customer tenant is not exhibiting a configuration error; it is exhibiting a structural defect in the multi-tenant authorization model, which apparently permits certain high-privilege applications to extend their own role assignments without independent customer-side authorization. This should not be architecturally possible. It is possible because the consent model collapses the distinction between “what I was approved for” and “what I can approve myself for,” and nobody had a reason to close the gap until someone walked through it. Then consider the target selection: planting long-lived credentials inside a security platform — a detection vendor, a gatekeeper — is not smash-and-grab behavior. It is the deliberate decapitation of the observer. You do not blind the camera unless you intend to operate in the room. Finally, consider the two-year expiry horizon. Ransomware affiliates operate on windows of days to weeks. An actor provisioning credentials valid into 2028 is operating on an annual planning cycle, which means the adversary is either a sophisticated initial-access broker positioning for resale, or a state-aligned collection program with patience built into its doctrine. Either way, the industry's reflexive framing of such events as criminal opportunity is doing the adversary's concealment work for it.
Now the structural insight, the one I have not seen made elsewhere: this is not a ConnectWise problem. It is the architecture of an entire product category, and the category will resist fixing it because the fix undermines the sales pitch. Any zero-configuration, fully automatic backup product for the MSP market must necessarily request full-domain read access — that is what “full backup” means. The permission breadth is not a bug on the roadmap; it is the core value proposition. The better the product performs its marketed function, the larger the blast radius it carries. Which means the correct remediation is not “request fewer permissions,” because a backup product requesting fewer permissions is a backup product that fails to back things up. The correct remediation is architectural: segmentable permission grants, customer-side independent revocation, and short-lived credentials renewed through a mechanism the customer can observe and veto. As of this writing, I am aware of no major vendor in this category that has implemented all three.
Then there is the measurement failure, and this one should alarm the governance layer more than the technical layer. By the vendor's own account, this event has no CVE, no protocol vulnerability, no misconfiguration to attribute. It cannot be captured by CVSS scoring, cannot be enumerated in a software bill of materials, cannot be tracked by patch SLAs, cannot be flagged by the vulnerability scanners that procurement departments use to populate their questionnaires. The entire apparatus of enterprise security metrics was blind to this event by construction — not because the apparatus was applied poorly, but because the risk class lives entirely outside its ontology. A purchasing committee that asks how many critical vulnerabilities a vendor has addressed in the past quarter is asking a question that would never have surfaced this breach. This is what I mean when I say the data hides what the eyes refuse to see: the measurement system itself does the hiding. The blind spot is institutionalized.
And here the picture widens into something the vendor disclosure will never describe, because it falls outside the frame of a single company's incident response. The backup vendor sits at the center of a distribution pyramid — B2B2B2C, in the industry's own shorthand. ConnectWise sells to managed service providers, who resell to small and mid-sized businesses who have no idea whose software is reading their mailboxes. The trust model is transitive and it is invisible to the end customer. The small business never chose ConnectWise. It chose an MSP, the MSP chose a stack, and the stack chose to hold a key that opens everything. Every layer of the pyramid adds convenience and subtracts observability, until the bottom tier is entirely dependent on actors it has never met, using software it has never seen, under a security model it cannot audit. That is the actual product being sold at scale: not backup, but a delegation of trust so complete that the delegator ceases to be able to revoke it.
I want to be careful about what I claim and what I infer, because the discipline matters more than the drama. The facts, as reported, are these: full-domain read scope across mail, files, chat, and directory; a stolen client assertion rather than a user credential; ordered traversal across one hundred seventy-three tenants within a bounded window; self-elevation to Global Administrator; secret injection into two secondary applications with a two-thousand-twenty-eight expiry; no CVE, no protocol flaw, no misconfiguration. The inference, clearly labeled as such, is this: the tenant roster was compromised independently and before the traversal; the operation's first phase was verification and staging, not harvest; the actor's time horizon is measured in years, not days; and the event is not concluded, whatever the disclosure language implies.
Which brings me to the contrarian point, the one I expect to be unwelcome in both the security press and the vendor's own post-mortem, and the one I have become increasingly convinced of as I have watched this category consolidate.
The prevailing narrative treats this incident as a cloud security failure — a story about identity providers, conditional access gaps, and the need for stronger workload identity hygiene. This framing is comfortable because it locates the failure in a technical abstraction that has a roadmap. But I would argue the failure is not in the cloud at all. It is in the governance of the backup category specifically, and it is a failure that predates cloud computing by two decades. On-premises backup software has always demanded domain administrator rights. It has always stored them centrally. It has always been trusted absolutely, and it has always been a favored target of sophisticated intrusion sets for exactly this reason. What the cloud did was not create this vulnerability; it pooled it. One hundred seventy-three tenants from a single key is simply the on-premises trust model, scaled by the multitenancy of the modern platform. The problem is not that we moved to the cloud. The problem is that we carried the backup category's absolutist permission model with us, unchanged, into an environment that magnifies its consequences by three orders of magnitude.
The second contrarian observation is that the industry's metrics are not merely blind to this risk class — they actively reward the behavior that produces it. A backup product that segments its permissions, exposes revocation controls to customers, and forces short-lived credentials would score worse on every procurement rubric in existence: more setup friction, more permission prompts, lower “ease of deployment” marks, more support tickets. The market is voting, consistently, for the architecture that produced this breach, and it will keep voting that way until the cost of the breach is internalized by the buyer rather than socialized across the hundred seventy-three. This is not a security problem in the traditional sense. It is a market design problem wearing a security costume, and market design problems are solved by pricing, liability, and disclosure regimes — not by patch management.
On that question of liability and regulatory architecture, the European picture is instructive, and it is where I expect the second-order consequences to concentrate. Under the regulatory regimes now maturing across the bloc, an incident of this shape creates obligations that the vendor's own disclosure does not discharge. The affected tenants are data controllers; the vendor is a processor; and the question of whether the processor notified controllers within the required window, with sufficient specificity for controllers to meet their own downstream obligations to supervisory authorities, is a legal question that no amount of technical remediation resolves. I would predict three things. First, that the contractual architecture between backup vendors and MSPs — which today typically caps liability at some multiple of annual subscription — will come under sustained pressure from mid-market customers who suddenly understand what they delegated. Second, that cyber insurance underwriters will reprice multi-tenant application credentials as a distinct risk category, which will do more to change vendor behavior in eighteen months than any security framework has done in eighteen years. Third, that procurement questionnaires will begin asking the question that would have caught this: not “how many CVEs,” but “show us the full lifecycle of your client assertions, show us every storage location of your private keys, and show us the customer-side revocation control.”
I recognize that prediction three requires the buyer side to develop a competence it currently lacks, and this is the part of the story that genuinely concerns me. The small and mid-sized businesses at the bottom of the pyramid are structurally incapable of performing this diligence. They will continue to rely on their MSPs, who will continue to rely on the vendors, who will continue to design for ease of deployment. The asymmetry is not going to self-correct. Waiting for the market to reveal its true cost is a strategy that only works if the market is capable of seeing the cost, and at the bottom of the distribution pyramid the market is blind by design.
So let me be precise about what I think actually changes. The vendors that survive this decade will be the ones that make the trust model legible — not the ones that make it invisible. A backup platform that shows the customer a live inventory of every application identity holding a key to their tenant, allows independent revocation of any one of them without breaking the service, and rotates those credentials on a schedule the customer can observe, will lose every head-to-head usability comparison and win every enterprise security review. That inversion is coming, but it will be driven by insurance pricing and enterprise procurement rather than by the mid-market, because the mid-market will not have the leverage to demand it. The consolidation pressure will fall hardest on the smaller MSPs, who cannot absorb the compliance overhead of the new diligence regime, and the largest platforms, which can, will inherit the channel. I have watched this exact dynamic play out in the European stablecoin settlement market as regulatory clarity sorted the viable from the merely enthusiastic, and the pattern is not specific to crypto — it is what regulatory clarity does to any fragmented distribution layer. It concentrates.
What remains unresolved, and what I will be watching closely through the end of this year, is the question of scope. The two-year secret expiry implies a campaign with a planning horizon. The injection into a detection vendor implies an intent to operate unobserved. The ordered traversal implies a pre-existing roster. Taken together, these are not the signatures of an incident that is over; they are the signatures of an incident that has been staged. Every disclosure I have read describes the verification phase with precision and the harvest phase with silence, and I have learned to pay attention to which silences are chosen. The data hides what the eyes refuse to see, and in this case the eyes are pointed firmly at the question of how the key was stolen — a question that matters, certainly, but a question whose answer will be delivered long before the answer to the more consequential one: what did the staging prepare for, and who is still holding credentials that expire in two thousand twenty-eight?
The industry will spend the next quarter debating permission scopes. It should be spending it asking how many other applications in the multi-tenant ecosystem can grant themselves Global Administrator, and how many tenants would ever know if they did. The answer to that question is the one that determines whether this event was an anomaly or a rehearsal. And until someone publishes that inventory, we are all of us — vendors, MSPs, and the small businesses at the bottom of a trust pyramid they never see — waiting for the market to reveal its true cost.
The door had a worn latch. It is still worn. The only thing that has changed is that we now know someone walked through it, and we still do not know how many times they intend to come back.