The .fullgc Problem: What Actually Happened
Starting around February 7, 2026, users of Qinglong, a widely used task scheduling platform with over 19,000 GitHub stars, began reporting strange activity. Their servers were maxing out CPU, and a rogue process named .fullgc kept popping up. This wasn't a Java garbage collector; it was a cryptominer, a direct result of Qinglong RCE cryptomining attacks.
The attacks targeted publicly exposed Qinglong panels, specifically versions 2.20.1 and older. Attackers were able to gain unauthenticated remote code execution (RCE), then use that access to inject shell commands into the config.sh file. These commands downloaded and executed the .fullgc cryptominer from a remote resource, file.551911.xyz. The miner was designed to persist across restarts and came in variants for Linux x86_64, ARM64, and even macOS, showing the attackers' broad targeting.
The maintainers acknowledged the issue on March 1, 2026, urging users to update. But the path to a real fix wasn't straightforward, which is a common problem in open-source security.
How a Middleware Mismatch Led to Qinglong RCE Flaws
The core of this problem wasn't a single, glaring vulnerability, but a chain of two authentication bypasses (CVE-2026-3965 and CVE-2026-4047) that stemmed from a classic web application security anti-pattern: a disagreement between the security middleware and the routing layer.
Here's the chain:
- CVE-2026-3965: The Rewrite Rule Problem. Qinglong used Express.js, and a misconfigured rewrite rule mapped requests to
/open/*to/api/*. The/api/*path was supposed to be protected, but this rewrite effectively created an unauthenticated backdoor to those admin endpoints. This meant an attacker could hit/open/some/admin/endpointand bypass the initial authentication check. - CVE-2026-4047: The Case-Sensitivity Trap. Even if the rewrite rule wasn't there, there was another way in. The authentication middleware checked paths using case-sensitive string matching, like
req.path.startsWith('/api/'). The Express.js router, however, matched routes case-insensitively. So, an attacker could request/aPi/system/command-run, and the authentication middleware would miss it because of the capitalization, while the router would still correctly route it to the command execution endpoint.
This combination meant attackers could hit protected admin endpoints, like those for command execution or even reinitializing admin credentials, without ever needing a username or password. It's a subtle but critical difference in how two parts of the application interpreted the same URL. (I've seen similar issues with trailing slashes or URL encoding differences between proxies and application servers; it's a common blind spot.)
The root of this problem lies in the fundamental interaction between Express.js's routing engine and custom middleware. Express.js, by default, can be configured to handle routes case-insensitively, especially when using regular expressions or specific routing options. However, a custom authentication middleware often relies on simpler string comparisons, such as req.path.startsWith(). If the middleware performs a case-sensitive check while the router processes requests case-insensitively, a bypass becomes trivial. Attackers can simply alter the casing of a URL segment to evade the middleware's check, yet still have the request correctly routed by Express. This discrepancy is a classic example of how seemingly minor implementation details can lead to severe Qinglong RCE cryptomining vulnerabilities.
The Impact: Beyond Just CPU Cycles
The immediate impact was clear: developers' servers were turned into cryptomining rigs, chewing up CPU cycles and cloud bills. The .fullgc process, named to mimic a Java "Full GC" (Garbage Collection) process, was a clever evasion tactic, designed to blend in with legitimate system activity and delay detection.
But the impact went deeper. At least one user reported that their Nezha monitoring panel, which provides visibility into hundreds of machines, was also compromised. This shows how an initial RCE on a seemingly isolated task scheduler can become a beachhead for broader network compromise and lateral movement. It's not just about the cryptominer; it's about what else an attacker could have done with that level of access, leveraging the initial Qinglong RCE.
The Patching Pitfalls and What We Learned
The timeline of the fix is also telling. User reports started in early February. An initial pull request (PR #2924) was submitted by a Copilot SWE Agent, attempting to block shell injection patterns in config files. This PR was never merged and, as Snyk later pointed out, wouldn't have fixed the underlying authentication bypasses anyway. It was an attempt to treat a symptom, not the root cause.
The effective fix (PR #2941) came later, directly addressing the authentication bypass in the middleware. This highlights a common challenge in open-source security: quick fixes often target the most visible exploit, while the deeper architectural flaws take longer to identify and patch correctly.
The incident also underscores the complexities of maintaining security in widely adopted open-source projects. Volunteer maintainers often face immense pressure to deliver fixes quickly, sometimes leading to patches that address symptoms rather than root causes. This can create a cycle of reactive security, where new bypasses are discovered shortly after initial fixes. A more proactive approach, involving regular security audits, threat modeling, and a deeper understanding of framework-specific security nuances, is crucial to prevent future Qinglong RCE cryptomining scenarios. The community's role in reporting, testing, and contributing to robust solutions is invaluable, but it also highlights the need for dedicated security resources within such projects.
For anyone running self-hosted applications, especially open-source tools like Qinglong, this incident offers some clear lessons:
- Audit Your Middleware Chains: You need to understand how your security middleware interacts with your routing layer. Look for mismatches in how they handle URL normalization (case sensitivity, trailing slashes, URL encoding, rewrite rules). This is a non-negotiable check for any internet-exposed web application.
- Solid Authentication is Essential: Make sure every internet-facing application has strong, multi-factor authentication, and consider placing these tools behind a VPN or SSH tunnel.
- Monitor Resource Usage: Keep an eye on unexpected CPU saturation or network connections. Tools like Nezha (ironically, also a target here) or cloud provider monitoring can flag these anomalies early.
- Keep Software Updated: This sounds obvious, but it's the first line of defense. Update your Docker images and npm packages regularly.
- Know How to Detect Compromise: Look for unexpected processes (like
.fullgc), modified configuration files (config.sh), or new cron tasks. If you find a compromise, delete mapped Docker volumes, remove malicious code, recreate containers from a clean image, and update to the latest version.
The Qinglong RCE isn't just another cryptomining story. It's a stark reminder that subtle code interactions, especially between different layers of a web application, can create wide-open doors for attackers. We need to move beyond just patching known exploits and start proactively auditing our application architectures for these kinds of fundamental mismatches.
