Netty `netty-tcnative` `2.0.84.Final` patches OpenSSL BIO ring-buffer heap overflow, session-ticket key rotation UAF, and modified UTF-8 SNI spoofing

CVE

CVE-2026-116539, CVE-2026-116540, CVE-2026-116541, CVE-2026-116542

CWE

CWE-122, CWE-787, CWE-416, CWE-170, CWE-295

Affected Surface

• Maven / Java `io.netty:netty-tcnative`, `io.netty:netty-tcnative-boringssl-static`, and `io.netty:netty-tcnative-classes` versions `< 2.0.84.Final` (`>= 2.0.66.Final, < 2.0.84.Final` when built against OpenSSL 3.x for `CVE-2026-116540`)
• Java, Kotlin, and Scala TLS servers and clients (including Netty `SslHandler` with `SslProvider.OPENSSL` / `OPENSSL_REFCNT`, gRPC-Java, Reactor Netty, Spring Cloud Gateway, Vert.x, Armeria, and Finagle) that use `netty-tcnative` for native OpenSSL/BoringSSL acceleration

TL;DR

On 26 October 2026, four vulnerabilities in netty-tcnative (io.netty:netty-tcnative* before 2.0.84.Final), the JNI native OpenSSL/BoringSSL transport used by Netty, gRPC-Java, Reactor Netty, Vert.x, and Spring Cloud Gateway to accelerate TLS, were published to NVD:

1. CVE-2026-116539 (GHSA-gf9q-9mg7-g5qh, CVSS 8.2 High): In openssl-dynamic/src/main/c/ssl.c, the custom OpenSSL BIO_write callback tcn_write_to_bytebuffer() calculated the ring-buffer write index as startIndex = bioUserData->nonApplicationBufferOffset + bioUserData->nonApplicationBufferLength without a modulo (% bioUserData->nonApplicationBufferSize). When the ring buffer wrapped around, writeChunk = nonApplicationBufferSize - startIndex went negative, underflowed to a huge size_t value near SIZE_MAX when passed to memcpy(), and caused a heap buffer overflow crashing the JVM.
2. CVE-2026-116540 (GHSA-mmc8-vg2v-fvvw, CVSS 8.2 High): On builds compiled against OpenSSL 3.x (>= 2.0.66.Final, < 2.0.84.Final), setSessionTicketKeys0() stored an OSSL_PARAM mac_params[3] array inside each tcn_ssl_ticket_key_t struct whose OSSL_MAC_PARAM_KEY pointer referenced the shared c->ticket_keys[i].hmac_key heap buffer. Although find_session_key() copied the tcn_ssl_ticket_key_t struct by value under a read lock, the copied key.mac_params[0].data pointer still pointed into the old c->ticket_keys allocation after the lock was released. Rotating session ticket keys (SSLContext.setSessionTicketKeys) while TLS handshakes were in flight freed c->ticket_keys and caused EVP_MAC_CTX_set_params(mac_ctx, key.mac_params) to read freed memory.
3. CVE-2026-116541 (GHSA-9xr3-jwv4-r489, CVSS 8.2 High): When tcn_write_to_bytebuffer() flushed queued non-application bytes before writing the caller’s in buffer, it returned writeAmount + writeChunk (the flushed bytes plus the newly consumed bytes) instead of writeChunk. Because OpenSSL expects BIO_write() to return at most inl bytes consumed from in, TLS 1.3 post-handshake flights (NewSessionTicket, KeyUpdate) miscalculated buffer offsets, truncating flights or aborting the JVM.
4. CVE-2026-116542 (GHSA-2784-hxmm-4hgf, CVSS 6.9 Medium): ssl_servername_cb() and SSL.getSniHostname() converted the raw C servername string from the TLS ClientHello SNI extension into a Java String using JNI (*e)->NewStringUTF(e, servername). Because JNI NewStringUTF decodes Modified UTF-8 (where the 2-byte sequence 0xC0 0x80 decodes to U+0000 instead of terminating the C string), an unauthenticated client could send overlong UTF-8 sequences in server_name to inject embedded NUL characters or alter the hostname seen by Java SNI matchers, virtual-host routers, and audit logs.

What to do now: Upgrade io.netty:netty-tcnative, io.netty:netty-tcnative-classes, and io.netty:netty-tcnative-boringssl-static to 2.0.84.Final (or upgrade your Netty BOM to a version that pulls 2.0.84.Final).

Affected packages and projects

Ecosystem	Maven coordinates	CVE / Advisory	Severity	Affected versions	Fixed version	Vulnerable component
Maven	io.netty:netty-tcnative*	CVE-2026-116539 (GHSA-gf9q-9mg7-g5qh)	CVSS 8.2 High	< 2.0.84.Final	2.0.84.Final	openssl-dynamic/src/main/c/ssl.c (tcn_write_to_bytebuffer ring-buffer wrap)
Maven	io.netty:netty-tcnative*	CVE-2026-116540 (GHSA-mmc8-vg2v-fvvw)	CVSS 8.2 High	>= 2.0.66.Final, < 2.0.84.Final (OpenSSL 3.x)	2.0.84.Final	openssl-dynamic/src/main/c/sslcontext.c (ssl_tlsext_ticket_key_cb / setSessionTicketKeys0)
Maven	io.netty:netty-tcnative*	CVE-2026-116541 (GHSA-9xr3-jwv4-r489)	CVSS 8.2 High	< 2.0.84.Final	2.0.84.Final	openssl-dynamic/src/main/c/ssl.c (tcn_write_to_bytebuffer return value)
Maven	io.netty:netty-tcnative*	CVE-2026-116542 (GHSA-2784-hxmm-4hgf)	CVSS 6.9 Medium	< 2.0.84.Final	2.0.84.Final	openssl-dynamic/src/main/c/sslcontext.c (ssl_servername_cb) and ssl.c (getSniHostname)

Technical analysis

1. CVE-2026-116539 and CVE-2026-116541: ring-buffer integer underflow and BIO_write over-reporting in tcn_write_to_bytebuffer

When Netty’s ReferenceCountedOpenSslEngine wraps or unwraps TLS frames, netty-tcnative bridges OpenSSL’s BIO abstraction to a direct Java ByteBuffer via tcn_write_to_bytebuffer(BIO* bio, const char* in, int inl) in openssl-dynamic/src/main/c/ssl.c.

If the target Java ByteBuffer (bioUserData->buffer) has no room left while OpenSSL is emitting non-application records (such as handshake messages, alerts, or TLS 1.3 NewSessionTicket / KeyUpdate records), tcn_write_to_bytebuffer() buffers the excess bytes inside an internal circular buffer (bioUserData->nonApplicationBuffer of size nonApplicationBufferSize).

Before commit 92dd66fd12, startIndex was computed by adding nonApplicationBufferOffset + nonApplicationBufferLength without wrapping modulo nonApplicationBufferSize:

--- a/openssl-dynamic/src/main/c/ssl.c
+++ b/openssl-dynamic/src/main/c/ssl.c
@@ -196,7 +196,9 @@ static jint tcn_write_to_bytebuffer(BIO* bio, const char* in, int inl) {
         writeAmount = TCN_MIN(nonApplicationBufferFreeSpace, (jint) inl) * sizeof(char);
-        startIndex = bioUserData->nonApplicationBufferOffset + bioUserData->nonApplicationBufferLength;
+        // use modulo to account for wrap around the ring buffer that buffers data.
+        startIndex = (bioUserData->nonApplicationBufferOffset + bioUserData->nonApplicationBufferLength)
+                                          % bioUserData->nonApplicationBufferSize;
         writeChunk = bioUserData->nonApplicationBufferSize - startIndex;

Once nonApplicationBufferOffset > 0 (meaning part of an earlier buffered record was drained) and new handshake or post-handshake data was written so that nonApplicationBufferOffset + nonApplicationBufferLength > nonApplicationBufferSize, startIndex exceeded nonApplicationBufferSize. That made the signed jint writeChunk = nonApplicationBufferSize - startIndex negative. Passing that negative writeChunk into memcpy converted it to an unsigned size_t close to SIZE_MAX, overflowing the heap buffer and crashing the JVM process.

In the exact same function, commit e31df4a588 fixes CVE-2026-116541. When bioUserData->nonApplicationBufferLength != 0 at the start of a subsequent BIO_write(bio, in, inl) call, tcn_write_to_bytebuffer() first flushed the previously queued ring-buffer bytes into the Java ByteBuffer (writeAmount = tcn_flush_sslbuffer_to_bytebuffer(bioUserData)), then copied writeChunk bytes from the caller’s new in buffer, and returned writeAmount + writeChunk:

--- a/openssl-dynamic/src/main/c/ssl.c
+++ b/openssl-dynamic/src/main/c/ssl.c
@@ -225,7 +225,14 @@ static jint tcn_write_to_bytebuffer(BIO* bio, const char* in, int inl) {
     // First check if we need to drain data queued in the internal SSL buffer.
     if (bioUserData->nonApplicationBufferLength != 0) {
-        writeAmount = tcn_flush_sslbuffer_to_bytebuffer(bioUserData);
+        tcn_flush_sslbuffer_to_bytebuffer(bioUserData);
+
+        if (bioUserData->bufferLength == 0) {
+            BIO_set_retry_write(bio); // no space left.
+            return -1;
+        }
     }
...
-    return writeAmount + writeChunk;
+    return writeChunk;
 }

Because writeAmount + writeChunk could exceed inl (the size of in), OpenSSL’s internal record layer advanced its input pointer past the end of in or failed an assertion when sending queued TLS 1.3 post-handshake messages.

2. CVE-2026-116540: shallow OSSL_PARAM copy in ssl_tlsext_ticket_key_cb on OpenSSL 3.x

In openssl-dynamic/src/main/c/sslcontext.c, find_session_key() acquires a read lock on the SSL context, copies the matching tcn_ssl_ticket_key_t entry into a stack-allocated *key struct (memcpy(key, &c->ticket_keys[i], sizeof(tcn_ssl_ticket_key_t))), and releases the lock so ssl_tlsext_ticket_key_cb() can initialize the cipher and HMAC contexts without holding the lock.

When OpenSSL 3.x support was added in 2.0.66.Final, tcn_ssl_ticket_key_t embedded an OSSL_PARAM mac_params[3] array that was initialized once inside setSessionTicketKeys0():

// openssl-dynamic/src/main/c/sslcontext.c (vulnerable, 2.0.66.Final to 2.0.83.Final)
ticket_keys[i].mac_params[0] = OSSL_PARAM_construct_octet_string(
    OSSL_MAC_PARAM_KEY, ticket_keys[i].hmac_key, 16);

Because OSSL_PARAM_construct_octet_string stores a raw pointer (data = &c->ticket_keys[i].hmac_key), copying the tcn_ssl_ticket_key_t struct onto the stack in find_session_key() copied mac_params[0].data as a dangling pointer back into the heap-allocated c->ticket_keys array. If another thread called SSLContext.setSessionTicketKeys() to rotate ticket keys, setSessionTicketKeys0() freed c->ticket_keys while ssl_tlsext_ticket_key_cb() was calling EVP_MAC_CTX_set_params(mac_ctx, key.mac_params). Commit 59662a8b07 removes mac_params from tcn_ssl_ticket_key_t and constructs OSSL_PARAM local_mac_params[3] on the stack inside ssl_tlsext_ticket_key_cb() pointing at the stack copy key.hmac_key.

3. CVE-2026-116542: JNI Modified UTF-8 NewStringUTF decoding in SNI callbacks

When a TLS client sends a server_name (SNI) extension in its ClientHello, ssl_servername_cb() in sslcontext.c and SSL.getSniHostname() in ssl.c pass the C string returned by SSL_get_servername() into Java. Previously, both functions used JNI NewStringUTF:

--- a/openssl-dynamic/src/main/c/sslcontext.c
+++ b/openssl-dynamic/src/main/c/sslcontext.c
@@ -2616,7 +2616,7 @@ static int ssl_servername_cb(SSL *ssl, int *ad, void *arg)
-        if ((servername_str = (*e)->NewStringUTF(e, servername)) == NULL) {
+        if ((servername_str = tcn_new_stringn(e, servername, strlen(servername))) == NULL) {
             return SSL_TLSEXT_ERR_ALERT_FATAL;
         }

JNI’s NewStringUTF expects Modified UTF-8, not standard UTF-8 or ASCII. In Modified UTF-8, 0xC0 0x80 is decoded as a Java \u0000 (NUL) character rather than rejected as an overlong sequence, and supplementary characters or high bytes are decoded differently from standard UTF-8. An attacker who placed 0xC0 0x80 or non-ASCII bytes in the ClientHello server_name caused OpenSSL (which sees a non-NUL C byte sequence) and Java’s SNIMatcher / routing layer (which receives a Java String with embedded \0 or altered characters) to disagree on the requested SNI hostname. Commit a879c85b97 replaces NewStringUTF with tcn_new_stringn(e, servername, strlen(servername)).

Detection and remediation

# 1. Search Maven and Gradle build files and lockfiles for netty-tcnative versions < 2.0.84.Final
rg -n 'netty-tcnative' pom.xml build.gradle* gradle/libs.versions.toml gradle.lockfile 2>/dev/null

# 2. Check resolved Maven dependency trees
mvn dependency:tree -Dincludes=io.netty:*tcnative* 2>/dev/null || true
./gradlew dependencies --configuration runtimeClasspath 2>/dev/null | grep -i tcnative || true

Upgrade io.netty:netty-tcnative, io.netty:netty-tcnative-classes, and io.netty:netty-tcnative-boringssl-static to 2.0.84.Final or later. If a service cannot be redeployed immediately and does not require native OpenSSL JNI acceleration, switching SslContextBuilder from SslProvider.OPENSSL / SslProvider.OPENSSL_REFCNT to SslProvider.JDK avoids all four netty-tcnative code paths.

Timeline

• 2026-08-06 to 2026-08-10: Netty maintainers merge commits 59662a8b07 (CVE-2026-116540), a879c85b97 (CVE-2026-116542), 92dd66fd12 (CVE-2026-116539), and e31df4a588 (CVE-2026-116541), releasing netty-tcnative 2.0.84.Final.
• 2026-10-26: CVE-2026-116539, CVE-2026-116540, CVE-2026-116541, and CVE-2026-116542 (GHSA-gf9q-9mg7-g5qh, GHSA-mmc8-vg2v-fvvw, GHSA-9xr3-jwv4-r489, and GHSA-2784-hxmm-4hgf) are published to NVD.

From research to remediation

Check whether this pattern exists in your codebase

Turn this research into a remediation workflow. Scan dependencies and package manifests for similar supply-chain risk, then prioritize fixes with reachability context.

Explore dependency scanning Get Demo

References

• netty/netty-tcnative commit 92dd66fd12: fix ring-buffer wraparound heap overflow in tcn_write_to_bytebuffer (CVE-2026-116539)
• netty/netty-tcnative commit 59662a8b07: fix use-after-free in ssl_tlsext_ticket_key_cb on OpenSSL 3.x (CVE-2026-116540)
• netty/netty-tcnative commit e31df4a588: stop over-reporting flushed bytes in tcn_write_to_bytebuffer (CVE-2026-116541)
• netty/netty-tcnative commit a879c85b97: replace JNI NewStringUTF with tcn_new_stringn in SNI callbacks (CVE-2026-116542)
• NVD: CVE-2026-116539, CVE-2026-116540, CVE-2026-116541, and CVE-2026-116542