How Velvet Ant Hid in Your 'Air Gap' for a Decade
You operate an isolated network, believing it secure. Velvet Ant just demonstrated that assumption is flawed, maintaining presence for ten years. Sygnia researchers, in their 'Operation Highland' report, have attributed this cyberespionage campaign to the Chinese state-sponsored group Velvet Ant, revealing a core finding: This wasn't a rapid intrusion, but a decade-long residency in environments often considered impenetrable.
The public discourse highlights "Chinese hackers" and "decade-long persistence" in an "air-gapped" network. Sygnia and Cisco both issued warnings in 2024 regarding Velvet Ant's activities, including exploits against F5 BIG-IP devices and a Cisco NX-OS zero-day. However, the methodology is the critical detail, representing a masterclass in bypassing network segmentation and embedding so deeply that conventional defenses failed to detect it.
The Incident: A Decade of Undetected Espionage
Velvet Ant initiated this campaign in 2016. For ten years, they maintained full visibility into administrative activity on a target organization's isolated network. Sygnia researchers documented the full scope in 2024, revealing a level of persistence that demands serious analysis. Their objective wasn't direct air gap penetration, but an indirect bypass followed by deep entrenchment.
The Mechanism: Bypassing Isolation
Initial access involved targeting vulnerable internet-facing systems (MITRE ATT&CK T1190, T1133). The critical pivot involved moving from these exposed systems into the supposedly isolated network, which lacked direct external connectivity.
Velvet Ant deployed a modified GS-Netcat, disguised as a legitimate system component, to establish a reverse shell (MITRE ATT&CK T1059.004, T1071.001). This provided encrypted remote access. For persistence, they modified systemd services or startup scripts (MITRE ATT&CK T1543.002, T1547.001).
Internal tunneling followed. They installed a custom SOCKS5 proxy (MITRE ATT&CK T1090.002), running as a daemon masquerading as smbd -D. This transformed compromised internet-facing servers into internal pivot points. Varying filenames and ports across hosts complicated detection.
The jump to the isolated network was particularly ingenious. They altered the configuration of a compromised internet-facing Nginx server (MITRE ATT&CK T1071.001). This Nginx instance was configured to proxy specially crafted requests to a compromised backend server. That backend server's Nginx configuration was then modified to forward these requests to a FastCGI process, fcgiwrap, listening on a separate port.
The fcgiwrap acted as an execution bridge. It launched a custom binary named uptime. This uptime tool, contrary to its legitimate function, established SSH connections to systems within the isolated network (MITRE ATT&CK T1021.004), using parameters supplied in the HTTP POST requests (MITRE ATT&CK T1059.004). This effectively used web requests on an internet-facing server to trigger SSH connections into an isolated network.
Inside the network, Velvet Ant focused on deep persistence and credential theft. They achieved this through several sophisticated techniques:
- Linux PAM Hijack: Legitimate
pam_unix.somodules were replaced with backdoored versions (MITRE ATT&CK T1556.001, T1546.004). These malicious PAM modules both accepted hardcoded passwords for Velvet Ant's access and, more critically, harvested user credentials (MITRE ATT&CK T1552.004). Sygnia identified nine distinct variants of these backdoored modules. Every login and password on compromised systems was vulnerable. - OpenSSH Trojanization:
ssh,sshd, andscpbinaries were replaced with trojanized versions (MITRE ATT&CK T1556.002, T1546.003). These versions captured credentials and logged every command entered during SSH sessions (MITRE ATT&CK T1059.004). All collected data was stored locally, awaiting exfiltration.
At its core, this infiltration aimed to control the authentication process itself.
The Impact: Control Over Authentication
Velvet Ant achieved full visibility into administrative activity. Every login, every command executed across compromised hosts, was accessible to them. By embedding directly into the authentication process, they bypassed standard authentication flows. This meant their persistence remained even if passwords changed or sessions terminated.
This also made conventional containment measures largely ineffective. Changing a password or terminating a session is insufficient when the authentication mechanism itself is compromised. Sygnia noted that remediation was "particularly complicated" because replacing numerous critical system components risked operational outages. Recovery required rebuilding core infrastructure elements, not just cleaning up.
The Response: Re-evaluating Isolated Networks
Analysis of this incident, particularly the initial compromise of authentication infrastructure, the decade-long persistence, and the deep backdooring of critical Linux login components like PAM and OpenSSH, indicates a need to fundamentally reassess isolated network security. This operation has led many security professionals to reassess assumptions about network segmentation.
Sygnia's recommendations, while aligning with established security principles, gain new urgency from this incident. The Velvet Ant campaign underscores that authentication components—such as Linux PAM, OpenSSH, and even Windows LSASS—are not merely utilities, but critical security assets, serving as the gateway to system control. Protecting these components demands advanced controls, including robust EDR solutions, vigilant file integrity monitoring (FIM), hardened privileged access management, and mandatory multi-factor authentication (MFA). Furthermore, continuous monitoring for unauthorized modifications is essential; any unexpected change to critical binaries like pam_unix.so or sshd should trigger immediate, automated alerting and investigation. Finally, the complexity of remediation highlights the necessity of planning for offline recovery, which includes strict backups with immutable copies and adequate snapshot schedules. These backups, along with recovery hosts and scripts running validated operating systems, must be regularly tested to ensure operational resilience.
The incident clarifies several points: The concept of an "air gap" offering absolute security is a misnomer if the indirect bridges to it are not secured. Once an adversary establishes a foothold, they will seek the deepest, most persistent access available. Control over the authentication flow grants control over all subsequent activity. A compromised login process fundamentally impairs your visibility into system operations.
