Okta's Red Team published findings on July 16, 2026, detailing 'HollowByte' — an unauthenticated **HollowByte DDoS flaw** in OpenSSL servers. This critical issue stems from how OpenSSL handles TLS handshakes.
Understanding the HollowByte DDoS Flaw and Its 11-Byte Payload
The attack chain for the **HollowByte DDoS flaw** unfolds as follows:
During a TLS handshake, an incoming message includes a 4-byte header that specifies the size of the message body. In vulnerable OpenSSL versions, prior to the June 9, 2026 fixes, memory was allocated based on this declared length, occurring before the actual payload arrived and before any handshake validation could run. For a ClientHello message, this ceiling is 131 KB.
An attacker exploits this by sending a tiny 11-byte input, but crafts the header to claim a much larger message body. The server then allocates that large, declared memory block, and its worker thread blocks indefinitely, waiting for the rest of the data that never arrives. By repeating this across multiple connections, the attacker causes the server to allocate significant amounts of memory, all with minimal actual data transmitted.
While OpenSSL does free these buffers when a connection drops, systems using the GNU C Library (glibc) retain small-to-medium allocations for reuse rather than immediately returning them to the kernel.
This persistent memory retention by glibc is a critical factor in the **HollowByte DDoS flaw's** effectiveness. While OpenSSL might argue that buffers are eventually freed, the reality on many Linux systems means these resources are not immediately returned to the OS. This creates a window of vulnerability where an attacker can continuously fragment memory, making the system progressively slower and more unstable without ever triggering standard DoS alerts.
Attackers exploit this by randomizing claimed sizes across connections. This prevents glibc from effectively reusing freed memory, leading to severe heap fragmentation. The server's Resident Set Size (RSS) continues to climb, remaining persistently elevated even after attacker disconnects. The only way to fully reclaim that allocated space and prevent systems from slowing to a standstill is by restarting the process.
The Broader Impact of HollowByte
OpenSSL serves as the secure communication backbone for critical software such as NGINX, Apache, Node.js, Python, Ruby, PHP, MySQL, PostgreSQL, and most Linux distributions.
Okta's tests revealed significant real-world impact: a 1 GB NGINX server was OOM-killed with 547 MB of memory frozen in fragments, and a 16 GB server experienced 25% of its system memory consumed. Critically, this occurs without exceeding typical connection limits. Standard DoS defenses may not flag it, as attack bandwidth remains below alerting thresholds.
This flaw directly causes operational disruptions and reputational damage. Critical web services can slowly become unresponsive, or crash outright, due to a trickle of 11-byte requests.
The Fix, and the Fight Over Classification
OpenSSL released fixes on June 9, 2026, for versions 4.0.1, 3.6.3, 3.5.7, 3.4.6, and 3.0.21. The patch, authored by Matt Caswell, changes how the buffer grows: it only expands when data actually arrives, ignoring the header's declared size. This approach effectively addresses the vulnerability.
The classification of the **HollowByte DDoS flaw**, however, is contentious. OpenSSL labeled it a "bug or hardening" fix, assigning no CVE and issuing no advisory. Their rationale: 131 KB per connection is small, memory is allocated per connection regardless, and a bounded allocation does not constitute a vulnerability.
Okta's counter-argument is straightforward: the allocated memory is not reclaimed on glibc systems, which constitutes the core issue. This core issue elevates the problem beyond a simple bug, firmly establishing it as a significant **HollowByte DDoS flaw**.
This decision has drawn criticism within security communities. Critics highlight the inconsistency: OpenSSL has assigned CVEs to other memory-exhaustion DoS issues, such as CVE-2025-66199, a low severity TLS 1.3 certificate-compression bug, and CVE-2026-34183, a moderate severity unbounded memory growth in the QUIC PATH_CHALLENGE handler. The community identifies clear DoS potential here, and the absence of a CVE is seen as downplaying a serious operational risk.
The lack of a CVE for the **HollowByte DDoS flaw** is particularly concerning for automated security tools and vulnerability management systems. These systems rely heavily on CVE identifiers for tracking, prioritization, and remediation. Without one, organizations are left to manually track patch notes, a process that is both time-consuming and prone to error, especially in large-scale deployments. This oversight significantly elevates the operational risk associated with this specific OpenSSL vulnerability.
The complexity and maintenance challenges inherent in OpenSSL's codebase are widely recognized. When a flaw with such clear impact is quietly patched without a CVE, it complicates downstream security efforts. Distributions backporting patches might still report older, vulnerable version numbers. Without a CVE, standard advisories or OVAL feed updates are absent. Organizations must then scrutinize package changelogs or directly query maintainers regarding the June 9, 2026 fixes. For those building OpenSSL from source, upgrading to the listed fixed releases and restarting services is necessary.
Mitigating the HollowByte DDoS Flaw
Organizations running OpenSSL should prioritize upgrading their distribution's packages immediately, verifying versions against 4.0.1, 3.6.3, 3.5.7, 3.4.6, and 3.0.21. For older branches, ensuring the June 9, 2026 patches are applied is critical, as a process restart is necessary for clearing fragmented memory.
The classification of a persistent memory exhaustion DoS as a 'hardening fix' rather than a security vulnerability creates gaps in security visibility. This impedes accurate risk evaluation, effective patch deployment, and streamlined vulnerability tracking. Such distinctions complicate the identification and remediation of genuine threats, thereby increasing overall risk for users.
HollowByte functions as a critical **HollowByte DDoS flaw**. OpenSSL's classification does not diminish its operational impact; it remains a severe vulnerability capable of disrupting services.
