Re: Alias internal format leaked to application code?
Hi, On Mon, Jun 1, 2020 at 5:24 PM Xuelei Fan wrote: > > Good catch, Simone. It is not expected to parser the alias in application > code. Would you like file a bug? https://bugs.openjdk.java.net/browse/JDK-8246262. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Alias internal format leaked to application code?
Hi, when using "PKIX" as KeyManagerFactoryAlgorithm, the alias is manipulated from what's in the keystore (for example, "jetty") to an internal format such as "N.0.jetty", where N is an increasing number (in sun.security.ssl.X509KeyManagerImpl.makeAlias()). The problem is that (especially in case of SNI) the KeyManager could be wrapped by a user-written KeyManager that may delegate to the JDK one. When the user-written KeyManager delegates to the JDK instance by calling keyManager.getServerAliases(keyType, issuers), an array of aliases is returned, but the aliases are of the internal format described above. This makes the user-written code fail any logic that is based on the aliases, as comparing these internal formats with the ones present in the KeyStore will fail. Can you please clarify if this is expected behavior and whether user-written code should "unwrap" this internal alias format (is it defined somewhere?), or if this internal format is wrongly leaked to user-written code? Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: Reading from a closed socket: different behavior between Linux and other operating systems
Hi, On Mon, Jan 6, 2020 at 5:39 PM Chris Hegarty wrote: > // force RST > clientChannel.setOption(StandardSocketOptions.SO_LINGER, 0); Just want to point out that when the channel/socket is set in non-blocking mode, SO_LINGER is either not supported or gives undefined behavior or throws (on Windows). -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: Aftermath of TLS 1.3 in Java 11 with wrapped IOExceptions
Hi, On Mon, Dec 2, 2019 at 7:36 PM Xuelei Fan wrote: > > Hi, > > Could someone please help to verify if the attached patch (JDK 14) works > HttpComponents? I'm not sure I see the difference between a generic IOException and SSLHandshakeException. If a client is connecting to a busy proxy that sends a TLS close_notify to the client, that would be a SSLHandshakeException even though it may be actually possible to retry and get a successful connection and TLS handshake. On the other hand, if a busy proxy sends a FIN to the client, that would be an IOException that may or not succeed when retried. Or the server is running out of file descriptors, very likely will be reported as an IOException, but retrying won't help. Or the client is running out of file descriptors. Or the client cannot connect to the server (again, a retry may or may not be successful). Bottom line is that for both cases (IOException and SSLHandshakeException) a retry may be successful and as such client code will have a hard time to detect when or not to retry in either case. SSLException (and subclasses) do not report the TLS AlertDescription code, so it won't be possible to know the exact reason for the SSLHandshakeException, and from it decide whether it's retryable or not. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: Change in closeOutbound() behavior for unused SSLEngine
Hi, On Wed, Oct 16, 2019 at 7:42 PM Xuelei Fan wrote: > > Hi Simone, > > The compatibility impact makes sense to me. Would you mind file a new bug? https://bugs.openjdk.java.net/browse/JDK-8232432 Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: Change in closeOutbound() behavior for unused SSLEngine
Hi, On Wed, Oct 16, 2019 at 5:42 PM Xuelei Fan wrote: > > The TLS protocol was changed to use half-close policy since TLS 1.3. As > means that sslEngine.closeOutbound() will close the outbound and keep > the inbound open. "NEED_UNWRAP" is used to indicate that the engine can > still be used to decode input message. Maybe it's a matter of interpretation, but the SSLEngine state was not used to indicate what may _possibly_ be done in the future. It was an exact indication of what an application should do _now_. In the normal case (after the handshake is completed), closeOutbound() turns the state into NEED_WRAP because wrapping will generate the close_notify. In this case, NEED_UNWRAP is strange because I just closed the outbound, and it's not clear what I need to read right now. In fact, most of the times the read will yield 0 bytes because the other peer is not aware that we have closed the outbound (as nothing was produced to be sent to the other peer). > For the specific case bellow, it is reasonable to expect > "NOT_HANDSHAKING" as the handshaking has not been started. On the other > side, as only the inbound open, it is also reasonable to me to use > "NEED_UNWRAP" although there is nothing to unwrap. I think, using > ""NOT_HANDSHAKING" may lead to confusing about what the next operation, > wrap() or unwrap(), could be in practice. CLOSED is not an option to me > as the inbound is still open. CLOSED is the _result_ (not the state) after a wrap(). Seems to me that you are saying that NOT_HANDSHAKING is wrong because it does not tell what to do now, and NEED_UNWRAP is also wrong because, well, there is nothing to unwrap. What remains is NEED_WRAP, which will correctly tell what an application needs to do. This is exactly what happens after the TLS handshake is completed: the application is in NOT_HANDSHAKING and calling closeOutbound() changes the state into NEED_WRAP. So I think there is an asymmetry between the behavior of closeOutbound() before and after the handshake that does not seem justified? > I understand there might be some compatibility issues for the use of > half-close policy. I may close both inbound and outbound of an engine > in the application code if the connection is not used. I think that if an application calls closeOutbound(), the implementation should only do that. > Is there a known compatibility impact on you applications? Yes. It's not possible to use the same code JDK 8 and JDK 11, unless there is some "if (isJDK11Behavior())" throughout the code. Looking just at JDK 11 behavior, what happens is: * closeOutbound() * state is now NEED_UNWRAP * read from network, 0 bytes * call unwrap(), produces a BUFFER_UNDERFLOW result, basically says "wait for bytes to read" that will never come * 0 bytes have been produced so nothing gets sent to the other peer * the application does not know _when_ to send a FIN to the other peer (it's waiting for the state to move away from NEED_UNWRAP and/or for a CLOSED result) Contrast that with: * closeOutbound() * state is now NEED_WRAP * call wrap(), produces CLOSED result, basically says "we are done with the output" * 0 bytes have been produced, so no close_notify is sent to the other peer, but we know from the CLOSED result that we can now send the FIN to the other peer. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Change in closeOutbound() behavior for unused SSLEngine
Hi,
SSLContext sslContext = SSLContext.getDefault();
SSLEngine sslEngine = sslContext.createSSLEngine();
sslEngine.closeOutbound();
SSLEngineResult.HandshakeStatus hsStatus = sslEngine.getHandshakeStatus();
System.err.println("hsStatus = " + hsStatus);
This prints "NOT_HANDSHAKING" in Java 8 and "NEED_UNWRAP" in JDK 11+.
In both cases, trying to wrap() consumes and produces 0 bytes (so the
close_notify is not generated, which I think is fine given that the
SSLEngine was never used) and produces a CLOSED result.
This case is common for connections that are established but never
used (not even a TLS byte was exchanged).
Is this change in behavior expected?
I find strange that calling closeOutbound() results in a NEED_UNWRAP
(as there is nothing to read).
Thanks!
--
Simone Bordet
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless. Victoria Livschitz
Re: JDK 13 SSLSession.getValue() throws NPE
Hi, On Fri, Sep 6, 2019 at 6:48 AM Jaikiran Pai wrote: > > Hi Simone, > > I remember hitting a similar issue back when I was testing a EA version > of JDK 13. There was bug for it here > https://bugs.openjdk.java.net/browse/JDK-8226649 and that one says it's > fixed in "13" but doesn't have a build number associated with it. Which > build of JDK 13 are you on? I was using 13+29. The issue is fixed in 13+33. Sorry for the noise. -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
JDK 13 SSLSession.getValue() throws NPE
Hi, we have a number of TLS tests failing in Jetty when running on JDK 13, while they work on JDK 11 and 12. The simplest way to reproduce this is to open 2 TLS sockets to Jetty, one after the other. On the server, the first socket creates a SSLEngine which has a SSLSession. Jetty calls SSLSession.getValue() to check whether a key (that we may have put on the session during SNI processing) is there. For the first socket this works. The second socket, however, is using resumption. On the server, a new SSLEngine is created, but its SSLSession has field "boundValues" == null so trying to call SSLSession.getValue() throws a NPE. Is this a regression? Thanks! -- Simone Bordet http://cometd.org http://webtide.com Developer advice, training, services and support from the Jetty & CometD experts.
Re: Use of OpenSSL as JCE security provider if available on system
Hi, On Thu, Mar 21, 2019 at 3:43 PM Sean Mullan wrote: > But, if we want to explore this further, I think it first makes sense to > take a step back and focus more on what benefits an OpenSSL provider or > "native bridge" would provide. Benchmarked 3x-10x performance improvements. https://nbsoftsolutions.com/blog/dropwizard-1-3-upcoming-tls-improvements I guess the memory allocation/footprint has similar improvements, with the JDK insisting at requiring ~17 KiB buffers to read HTTP requests in the order of <1 KiB. -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: Align SSLSocket and SSLEngine Javadocs
Xuelei,
On Mon, Aug 27, 2018 at 4:00 PM Xuelei Fan wrote:
>
> H Simone,
>
> There is no change for the SSLSocket.startHandshake() and
> SSLEngine.beginHandshake() specification. They are can be used for new
> handshake and key update.
>
> We want the specification independent from TLS versions as much as
> possible. An application developer only need to know the
> functionalities, but not necessarily to understand the TLS protocol details.
>
> For TLS 1.2 and prior versions, the key update is performed with
> renegotiation; for TLS 1.3, it is the KeyUpdate post-handshake.
Perhaps I was not clear. I'm not talking about the specification (what
the method does), just about the Javadoc.
A developer needs to know if calling a method causes a renegotiation or not :)
Would be great if your paragraph above ("For TLS 1.2 and prior ...")
would be included in the Javadoc.
In particular for SSLEngine, the current Javadoc says:
"Initiates handshaking (initial or renegotiation) on this SSLEngine."
It does not mention TLS 1.3 and does not mention KeyUpdate, so would
be great if it does.
And would be great if beginHandshake() and startHandshake() have a
very similar Javadoc.
Thanks!
--
Simone Bordet
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless. Victoria Livschitz
Align SSLSocket and SSLEngine Javadocs
Hi, SSLSocket.startHandshake() and SSLEngine.beginHandshake() are similar in that they start the TLS handshake, but they can also be used after the TLS handshake. SSLSocket.startHandshake() Javadoc seems to be more generic, describing that the method may not only start a new handshake but also be used to update encryption keys etc. Especially in light of TLS 1.3 where renegotiation is forbidden, I would like the Javadoc of these method to align and describe exactly when they do with respect to the TLS protocol version. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: SSLSocket session resumption not working with TLS 1.3 and 11+27
Hi, On Fri, Aug 24, 2018 at 8:54 PM Adam Petcher wrote: > On 8/23/2018 3:28 PM, Simone Bordet wrote: > > Hi, > > > > The code below reproduces a bug where session resumption is tested. > > Turns out that session resumption does happen on the client and on the > > server, but on the client the session id is not replaced with the > > resumed session id (while it is on the server). > > > > The code prints: > > > > session1 = Session(1535051727187|TLS_AES_128_GCM_SHA256) - [29, -38, ...] > > session2 = Session(1535051727187|TLS_AES_128_GCM_SHA256) - [11, -20, ...] > > > > where it can be seen that the second session has the same creationTime > > as the first, but different session id (the array). > > Just to be clear: are you saying that a short handshake is happening in > all cases, I have not tested all cases, but a short handshake happens (on the second connection) with the code I posted. > or are there some cases where a full handshake (including > Certificate and CertificateVerify) is used where you would expect a > short handshake? I have not seen this, but as I said, I have not tested all cases, such as client authentication, etc. The fact that the session ID is different is expected, > and it is a result of the fact that sessions are managed differently in > the TLS 1.3 implementation. However, on the server-side the session ID is the same, so I am confused by the fact that on the client the session has been resumed, but it has a different ID. > You can no longer use session IDs to > determine whether a full handshake was used. Why not? I could not find a section in the TLS 1.3 RFC that mandates that resumed session IDs must be different. We have had in the past Jetty users that relied on this feature heavily to confirm that they were actually resuming sessions - for performance reasons. Comparing SSLSession.creationTime seems more brittle. > Also sessions are managed > differently in the client and server code, so you should expect > different behavior in client/server with respect to session IDs. Indeed. Is this an implementation detail, or it's mandated by the RFC? > > About the code below, note how it is necessary to attempt a read after > > the first handshake to read the NewSessionTicket post-handshake > > message that is otherwise not consumed by the client. This was not > > necessary before. > > Scenarios where clients don't read from the server (or where many > > clients are created before one starts reading) will have a harder time > > to use session resumption. > > This is also expected behavior in the current TLS 1.3 implementation. I > created a ticket[1] to document your concerns and capture a few > potential ways to improve things. > > [1] https://bugs.openjdk.java.net/browse/JDK-8209953 Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
SSLSocket session resumption not working with TLS 1.3 and 11+27
Hi,
The code below reproduces a bug where session resumption is tested.
Turns out that session resumption does happen on the client and on the
server, but on the client the session id is not replaced with the
resumed session id (while it is on the server).
The code prints:
session1 = Session(1535051727187|TLS_AES_128_GCM_SHA256) - [29, -38, ...]
session2 = Session(1535051727187|TLS_AES_128_GCM_SHA256) - [11, -20, ...]
where it can be seen that the second session has the same creationTime
as the first, but different session id (the array).
About the code below, note how it is necessary to attempt a read after
the first handshake to read the NewSessionTicket post-handshake
message that is otherwise not consumed by the client. This was not
necessary before.
Scenarios where clients don't read from the server (or where many
clients are created before one starts reading) will have a harder time
to use session resumption.
Thanks!
public static void main(String[] args) throws Exception
{
String host = "localhost";
int port = 8443;
SSLContext serverSSLContext = ...;
SSLServerSocket sslServer =
(SSLServerSocket)serverSSLContext.getServerSocketFactory().createServerSocket(port);
Runnable serverReader = () ->
{
try
{
SSLSocket sslSocket = (SSLSocket)sslServer.accept();
InputStream input = sslSocket.getInputStream();
while (true)
{
int read = input.read();
if (read < 0)
break;
}
}
catch (Throwable x)
{
x.printStackTrace();
}
};
new Thread(serverReader).start();
SSLContext clientSSLContext = ...;
Socket client1 = new Socket(host, port);
SSLSocket sslClient1 =
(SSLSocket)clientSSLContext.getSocketFactory().createSocket(client1,
host, port, true);
AtomicReference session1 = new AtomicReference<>();
CountDownLatch handshakeLatch1 = new CountDownLatch(1);
sslClient1.addHandshakeCompletedListener(event ->
{
SSLSession session = event.getSession();
session1.set(session.toString() + " - " +
Arrays.toString(session.getId()));
handshakeLatch1.countDown();
});
sslClient1.startHandshake();
handshakeLatch1.await(1, TimeUnit.SECONDS);
try
{
sslClient1.setSoTimeout(1000);
sslClient1.getInputStream().read();
}
catch (SocketTimeoutException expected)
{
}
sslClient1.close();
new Thread(serverReader).start();
Socket client2 = new Socket(host, port);
SSLSocket sslClient2 =
(SSLSocket)clientSSLContext.getSocketFactory().createSocket(client2,
host, port, true);
AtomicReference session2 = new AtomicReference<>();
CountDownLatch handshakeLatch2 = new CountDownLatch(1);
sslClient2.addHandshakeCompletedListener(event ->
{
SSLSession session = event.getSession();
session2.set(session.toString() + " - " +
Arrays.toString(session.getId()));
handshakeLatch2.countDown();
});
sslClient2.startHandshake();
handshakeLatch2.await(1, TimeUnit.SECONDS);
System.err.println("session1 = " + session1);
System.err.println("session2 = " + session2);
sslClient2.close();
}
--
Simone Bordet
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless. Victoria Livschitz
SSLSocket weird behavior in JDK 11+27
Hi,
SSLSocket is behaving weird in 11+27.
In particular:
* Setup a SSLServerSocket.
* Connect with a normal Socket (rawClient).
* Wrap rawClient into a SSLSocket (sslClient).
* sslClient.startHandshake()
Now a few cases:
A) immediate rawClient.close()
If the server is reading via InputStream.read(), then it reads -1.
But if the server reads via InputStream.read(byte[]), then
SSLProtocolException is thrown.
I believe the second behavior is correct, as the client does not send
the close_notify, so the server should throw?
B) sslClient writes data + rawClient.close()
The server reads correctly the data, then reads -1, both with read()
and read(byte[]).
I believe this is wrong as close_notify is not sent by the client.
Running the example with JDK 10 always produces no exceptions and
always reads -1.
Below you can find a reproducible case.
Thanks!
public static void main(String[] args) throws Exception
{
SSLContext sslContext = __sslCtxFactory.getSslContext();
int port = 8443;
try (SSLServerSocket sslServer =
(SSLServerSocket)sslContext.getServerSocketFactory().createServerSocket(port))
{
Socket rawClient = new Socket("localhost", port);
SSLSocket sslClient =
(SSLSocket)sslContext.getSocketFactory().createSocket(rawClient,
"localhost", port, false);
SSLSocket socket = (SSLSocket)sslServer.accept();
CountDownLatch latch = new CountDownLatch(1);
new Thread(() ->
{
try
{
while (true)
{
//byte[] buffer = new byte[1024];
//int read = socket.getInputStream().read(buffer);
int read = socket.getInputStream().read();
System.err.println("read = " + read);
if (read < 0)
break;
}
}
catch (IOException x)
{
x.printStackTrace();
}
finally
{
latch.countDown();
}
}).start();
sslClient.startHandshake();
//OutputStream output = sslClient.getOutputStream();
//output.write(0);
//output.flush();
// Raw close.
rawClient.close();
latch.await(10, TimeUnit.SECONDS);
}
}
--
Simone Bordet
http://cometd.org
http://webtide.com
Developer advice, training, services and support
from the Jetty & CometD experts.
Re: SSLEngine weird behavior in 11+21?
Hi, > > On 7/12/2018 1:17 PM, Simone Bordet wrote: > > Currently: > > - Server wraps 160, 6 and 907 in 3 wraps. > > - Server *sends* the 1073 bytes in 1 TCP write > > - Client unwraps 160, then goes into NEED_WRAP > > - Client wraps 6, then goes again into NEED_UNWRAP to finish with the > > 6 and 907 received by the server. > > > > What I'm suggesting: > > - Server wraps 160, 6 and 907 in 3 wraps. > > - Server *sends* the 1073 (160+6+907) bytes in 1 TCP write > > - Client unwraps 160, stays in NEED_UNWRAP, unwraps the 6, unwraps the > > 907, then goes into NEED_WRAP > > - Client wraps 6 and 58 > > - Client *sends* the 64 (6+58) bytes in 1 TCP write. > > > > The 6 bytes of the dummy change_cipher_spec are *sent* just after > > receiving the ServerHello in both cases, they are just *generated* at > > different times. > > > > By having all the unwraps() consecutive and all the wraps() > > consecutive you can enable a number of optimizations I described > > earlier. > > > > Imagine a client that would perform a TCP *send* every time the state > > moves away from NEED_WRAP. > > Currently it would: > > 1. TCP send for ClientHello > > 2. Reads 1073 from server > > 3. Unwrap 160, then NEED_WRAP > > 4. Generate 6, then NEED_UNWRAP > > 5. TCP send the 6 bytes > > 6. Unwrap 6 and 907, then NEED_WRAP > > 7. Generate 58 then FINISHED > > 8 TCP send the 58 bytes. > > > > You can see that the client issues 3 TCP sends. > > > > With what I am suggesting, the client would only issue 2 TCP sends, > > which seems more in line with the efforts in TLS 1.3 to be more > > efficient. > > > > It would just be a matter of *generating* those 6 bytes a bit later, > > *after* having fully unwrapped the ServerHello. > > > On Fri, Jul 13, 2018 at 3:45 PM Xuelei Fan wrote: > It's a good idea! I just built: $ hg id -i 76072a077ee1 and I can still see the (very) inefficient behavior above. Are there plans to fix it? Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: SSLEngine weird behavior in 11+21?
Hi, On Tue, Jul 31, 2018 at 9:56 PM Xuelei Fan wrote: > Hm, I see your point. Because of the half-close policy, I think the > server could keep sending messages other than the close_notify alert. Exactly. > My concern of the CLOSED+NOT_HANDSHAKING is that it does not indicate > the following operations. Sure it does. CLOSED while unwrapping tells the application that the other side has closed. Now the application has a choice: it can wrap() more data, or call closeOutbound(). >From there, it follows SSLEngine instructions via the states. Having to take into consideration not only the Result, not only the HandshakeStatus, but also bytes produced seems weird to me. I'd prefer to just rely only on Result and HandshakeStatus. > Yes, but it is the desired behavior to me. For the TLS 1.3 half close > policy, it is not expected to generate close_notify reply unless the > server close its outbound. So the server can keep sending the > application data after the read close, as it write side is open. Exactly. > The TLS 1.3 spec says: >"Each party MUST send a "close_notify" alert before closing its write > side of the connection, unless it has already sent some error alert. > This does not have any effect on its read side of the connection." > > And the SSLEngine.closeOutbound() spec says: > "Signals that no more outbound application data will be sent on > this SSLEngine." > > I think it is fine to keep the write side open while the read side closed. > > In this update, for TLS 1.2 connection, the duplex-close behavior is > reserved: as the server received the close_notify, it moving to > NEED_WRAP; and the a close_notify will be delivered, and then duplex > close the connection. Exactly, and without the need for the receiver of the close_notify to call closeOutbound(). That will be a difference between TLS 1.2 and TLS 1.3 that applications should take into consideration. > However, for TLS 1.3 connection, if there is no call to > server.closeOutbound(), the server write side keeps open even the read > side has been closed. > > In this update, for TLS 1.3, I tried to support half-close for: > 1. SSLEngine.closeIn/Outbound() > 2. SSLSocket.shutdownIn/Output() > > and duplex-close for: > 3. SSLSocket.close() > > Unfortunately, it does introduce a compatibility issue that an existing > application may not close both inbound and outbound if the connection is > desired to be duplex-closed. In order to mitigate the impact, a new > System Property in introduced, "jdk.tls.acknowledgeCloseNotify". This > compatibility issues could be mitigated by closing both inbound and > outbound explicitly, or set the property to "true". If the property > value is "true", a corresponding close_notify alert will be sent when > receiving a close_notify alert, and therefore the connection will be > duplex closed. > > Does it make sense to you? Yes. Although I think application will need to make adjustments for TLS 1.3 anyway. That system property will smooth things, but won't guarantee that an application written for SSLEngine TLS 1.2 will work for SSLEngine TLS 1.3. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: SSLEngine weird behavior in 11+21?
Hi, On Tue, Jul 31, 2018 at 5:10 PM Xuelei Fan wrote: > > Hi, > > On 7/31/2018 6:43 AM, Xuelei Fan wrote: > > Current jdk11 tip with your patch: > > 1. client.closeOutbound() then goes into NEED_WRAP. > > 2. Client wraps 24 bytes, result is CLOSED, then goes into NEED_UNWRAP. > > 3. Server unwraps 24 bytes, result is CLOSED, then goes into NEED_WRAP. > > 4. Server wraps 0 bytes and stays in NEED_WRAP (?) > > In my testing (OpenJDK, > test/jdk/javax/net/ssl/TLSv1/TLSEnginesClosureTest.java), for #4, the > server could wrap the close_notify alert message for TLS 1.2 and prior > versions (CLOSED/NOT_HANDSHAKING); and wrap data for TLS 1.3 > (OK/NOT_HANDSHAKING, half-close). > > Are you using TLS 1.3 with no data in your test case in #4? Because of > the half-close policy, it may be the expected behavior if no > application data can be delivered. The problem with 4 between TLS 1.2 and your latest patch is that before there was no need to call server.closeOutbound(): as the server received the close_notify from the client, it was moving to NEED_WRAP and if wrap() was called it would generate the close_notify reply. With your latest patch, you _have_ to call server.closeOutbound() otherwise 4 will always generate 0 bytes and spin loop. That is why I prefer 2 to go into CLOSED+NOT_HANDSHAKING. When it goes into CLOSE+NEED_UNWRAP, the application will follow the instructions of SSLEngine and attempt an unwrap() immediately, while instead it should stop wrapping/unwrapping and write the close_notify to the server. > For TLS 1.3 > > Trying to close engines from Client to Server > Client wrapped 24 bytes. > Client handshake status is NEED_UNWRAP Result is CLOSED > Server unwrapping 24 bytes... > Server handshake status is NEED_WRAP Result is CLOSED > Server wrapped 16406 bytes. > Server handshake status is NEED_WRAP Result is OK > The above tells me that the server did not generate yet the close_notify reply because it is still in NEED_WRAP. Just to repeat myself I would prefer this: > Client called closeOutbound() status is NEED_WRAP > Client wrapped 24 bytes. > Client handshake status is NOT_HANDSHAKING Result is CLOSED > Server unwrapping 24 bytes... > Server handshake status is NOT_HANDSHAKING Result is CLOSED > Server wrapped 16406 bytes. > Server handshake status is NOT_HANDSHAKING Result is OK > Server called closeOutbound() status is NEED_WRAP > Server wraps 24 bytes > Server handshake status is NOT_HANDSHAKING Result is CLOSED Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: SSLEngine weird behavior in 11+21?
Hi, On Tue, Jul 31, 2018 at 4:13 PM Xuelei Fan wrote: > > The Status.CLOSED specification is defined as "The operation just closed > this side of the SSLEngine, or the operation could not be completed > because it was already closed.". My reading of the spec, the CLOSED > status means half-close. If wrap() status is CLOSED, it means write > side close; and unwrap() CLOSED is for read side close. > > I may prefer to: > 1. client.closeOutbound() then goes into NEED_WRAP. > 2. Client wraps 24 bytes, result is CLOSED, then goes into NEED_UNWRAP. > 3. Server unwraps 24 bytes, result is CLOSED, then goes into NEED_WRAP. > 4. server.closeOutbound() then goes into NEED_WRAP. > 5. Server wraps 24 bytes, result is CLOSED, then goes into NOT_HANDSHAKING. > 6. Client unwraps 24 bytes, result is CLOSED, then goes into > NOT_HANDSHAKING. Yes, we agreed that at step 2 and especially step 3 result must be CLOSED. Please consider the case where data is sent before the close_notify reply, and what would be good for you. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: SSLEngine weird behavior in 11+21?
Hi, On Tue, Jul 31, 2018 at 3:43 PM Xuelei Fan wrote: > > 1. client.closeOutbound() then goes into NEED_WRAP. > > 2. Client wraps 24 bytes, result is CLOSED, then goes into NOT_HANDSHAKING. > It might be a problem for application to make a right decision if using > CLOSED status and NOT_HANDSHAKING handshake status together. > > As the write side is close, the client now only be able to receiving > data from peer. I think NEED_UNWRAP may be more appropriate for > application development. I think the key point here is that SSLEngine tells the application what needs to be done. Because you implemented half-close, NEED_UNWRAP is not _strictly_ needed here. By staying in NOT_HANDSHAKING the normal flow will happen: data to read from the server, pass to unwrap(), it may be data or close_notify. Let's imagine it is data; after unwrapping data, do you stay in NEED_UNWRAP until the close_notify arrives? I'm fine with either NOT_HANSHAKING or NEED_UNWRAP, but I prefer NOT_HANDSHAKING. > However, the CLOSED status may be confusing as the client may still want > to read something later. > > I may prefer to use OK/NEED_UNWRAP. What do you think? I prefer CLOSED/NOT_HANDSHAKING, but OK/NEED_UNWRAP works too. > > 3. Server unwraps 24 bytes, result is CLOSED, then goes into > > NOT_HANDSHAKING. > Same reason as above, I may prefer to use OK/NEED_WRAP. Here I strongly disagree. It is fundamental for the application to know that it received the close_notify from the client, because the application needs to call closeOutbound(), eventually. The CLOSED result is the only thing that tells the application to call closeOutbound(). I think this is also good for backwards compatibility: with TLS 1.2 I bet most application were relying on the CLOSED state. I prefer CLOSED/NOT_HANDSHAKING; CLOSED/NEED_WRAP is ok too. > > 4. server.closeOutbound() then goes into NEED_WRAP. > > 5. Server wraps 24 bytes, result is CLOSED, then goes into NOT_HANDSHAKING. > > 6. Client unwraps 24 bytes, result is CLOSED, then goes into > > NOT_HANDSHAKING. > > > Agreed. > > If no objections, I will make the update: > > 1. client.closeOutbound() then goes into NEED_WRAP. > 2. Client wraps 24 bytes, result is OK, then goes into NEED_UNWRAP. > 3. Server unwraps 24 bytes, result is OK, then goes into NEED_WRAP. Nope: CLOSED, then NEED_WRAP, see above. > 4. server.closeOutbound() then goes into NEED_WRAP. > 5. Server wraps 24 bytes, result is CLOSED, then goes into NOT_HANDSHAKING. > 6. Client unwraps 24 bytes, result is CLOSED, then goes into > NOT_HANDSHAKING. Let's make another case with data, and what I prefer: 1. client.closeOutbound() then goes into NEED_WRAP. 2. Client wraps 24 bytes, result is OK, then goes into NOT_HANDSHAKING. 3. Server unwraps 24 bytes, result is CLOSED, then goes into NOT_HANDSHAKING. 4. Server wraps data bytes, result is OK, stays in NOT_HANDSHAKING. 5. Client unwraps data bytes result is OK, stays in NOT_HANDSHAKING. 6. server.closeOutbound() then goes into NEED_WRAP. 7. Server wraps 24 bytes, result is CLOSED, then goes into NOT_HANDSHAKING. 8. Client unwraps 24 bytes, result is CLOSED, then goes into NOT_HANDSHAKING. I'm fine for 2 and 5 to be NEED_UNWRAP; I'm fine for 3 and 4 to be NEED_WRAP. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: SSLEngine weird behavior in 11+21?
Hi, On Tue, Jul 31, 2018 at 11:39 AM Simone Bordet wrote: > > Hi, > On Mon, Jul 30, 2018 at 8:08 PM Xuelei Fan wrote: > > Would you mind look at the code I posted in the following thread: > > http://mail.openjdk.java.net/pipermail/security-dev/2018-July/017708.html > > JDK 11+21: > 1. client.closeOutbound() then goes into NEED_WRAP. > 2. Client wraps 24 bytes, result is CLOSED, then goes into NOT_HANDSHAKING (?) > 3. Server unwraps 24 bytes, result is CLOSED, then goes into NEED_WRAP. > 4. Server wraps 24 bytes, result is CLOSED, then goes into NOT_HANDSHAKING. > 5. Client unwraps 0 bytes (?) > > Current jdk11 tip with your patch: > 1. client.closeOutbound() then goes into NEED_WRAP. > 2. Client wraps 24 bytes, result is CLOSED, then goes into NEED_UNWRAP. > 3. Server unwraps 24 bytes, result is CLOSED, then goes into NEED_WRAP. > 4. Server wraps 0 bytes and stays in NEED_WRAP (?) > > I don't think this is right. > > While I previously complained about step 2 going into NOT_HANDSHAKING, > if you now support full half close, then I think this may be > reasonable, as the server may still send data and only later issue a > close_notify. > However, NEED_UNWRAP like it is now is also reasonable. > > At step 3, after the server unwraps the close_notify, the server > should either stay in NOT_HANDSHAKING *and* require a call to > closeOutbound() (which will move the state to NEED_WRAP), or it should > go into NEED_WRAP *and* produce the close_notify. > As it is now, SSLEngine tells the application to wrap(), but it wraps > 0 bytes, but tells again the application to wrap(), but still produces > 0 bytes, so it's going to be a tight spin loop - not good. For completeness, calling server.closeOutbound() at step 4. correctly moves SSLEngine into NEED_WRAP and a subsequent wrap() produces the 24 bytes of the close_notify and result CLOSED, then goes into NOT_HANDSHAKING. I think the current behavior (with your patch) needs to be fixed. Since you implemented half-close, my preference would be this: 1. client.closeOutbound() then goes into NEED_WRAP. 2. Client wraps 24 bytes, result is CLOSED, then goes into NOT_HANDSHAKING. 3. Server unwraps 24 bytes, result is CLOSED, then goes into NOT_HANDSHAKING. 4. server.closeOutbound() then goes into NEED_WRAP. 5. Server wraps 24 bytes, result is CLOSED, then goes into NOT_HANDSHAKING. 6. Client unwraps 24 bytes, result is CLOSED, then goes into NOT_HANDSHAKING. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: SSLEngine weird behavior in 11+21?
Hi, On Mon, Jul 30, 2018 at 8:08 PM Xuelei Fan wrote: > Would you mind look at the code I posted in the following thread: > http://mail.openjdk.java.net/pipermail/security-dev/2018-July/017708.html JDK 11+21: 1. client.closeOutbound() then goes into NEED_WRAP. 2. Client wraps 24 bytes, result is CLOSED, then goes into NOT_HANDSHAKING (?) 3. Server unwraps 24 bytes, result is CLOSED, then goes into NEED_WRAP. 4. Server wraps 24 bytes, result is CLOSED, then goes into NOT_HANDSHAKING. 5. Client unwraps 0 bytes (?) Current jdk11 tip with your patch: 1. client.closeOutbound() then goes into NEED_WRAP. 2. Client wraps 24 bytes, result is CLOSED, then goes into NEED_UNWRAP. 3. Server unwraps 24 bytes, result is CLOSED, then goes into NEED_WRAP. 4. Server wraps 0 bytes and stays in NEED_WRAP (?) I don't think this is right. While I previously complained about step 2 going into NOT_HANDSHAKING, if you now support full half close, then I think this may be reasonable, as the server may still send data and only later issue a close_notify. However, NEED_UNWRAP like it is now is also reasonable. At step 3, after the server unwraps the close_notify, the server should either stay in NOT_HANDSHAKING *and* require a call to closeOutbound() (which will move the state to NEED_WRAP), or it should go into NEED_WRAP *and* produce the close_notify. As it is now, SSLEngine tells the application to wrap(), but it wraps 0 bytes, but tells again the application to wrap(), but still produces 0 bytes, so it's going to be a tight spin loop - not good. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: SSLEngine weird behavior in 11+21?
Hi, On Fri, Jul 13, 2018 at 3:45 PM Xuelei Fan wrote: > Thank you very much, Simone. I find at least two improvements we can > take. It's really good! Great! Let know when they land in a 11+X release and we'll try them out. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: SSLEngine weird behavior in 11+21?
Hi, On Thu, Jul 12, 2018 at 11:10 PM Xuelei Fan wrote: > In TLS 1.3 closure, receiving of the close_notify is not required: >"... Both parties need not wait to receive a > "close_notify" alert before closing their read side of the > connection, though doing so would introduce the possibility of > truncation." > > TLS 1.2 also has similar specification: > "It is not required for the initiator > of the close to wait for the responding close_notify alert before > closing the read side of the connection." > > My reading the spec, the peer MUST send the close_notify, but it is not > required to close the connection. > > If the socket cannot be closed until close_notify get received, the > local may never have a chance to close the socket unless the peer > agrees. It does not sound like to me. The compatibility impact could > be serious as previous application work in a duplex-close manner. > 1. start and establish a TLS connection. > 2. client call close(), and the server will response with a close_notify > in TLS 1.2. the TLS connection can be closed gracefully. > > While for TLS 1.3, #2 does not work if the client calling of close() > closes the writing side only, and leave the reading side open. > > I see the benefits of half-close. I'm open to hear a better solution to > take the benefit of half-close and avoid the compatibility issue. Even in TLS 1.2 you cannot be sure that the server will send the close_notify (e.g. bad server). The half close problem is present in TCP, WebSocket, etc. and it's typically solved with (idle) timeouts. Client sends the close_notify, then reads, hoping to see the close_notify from the server; if it does not see it, it closes the raw socket after a timeout. The timeout could account for receiving data (i.e. it is reset if application data arrives), or not. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: SSLEngine weird behavior in 11+21?
Hi,
On Thu, Jul 12, 2018 at 11:15 PM Xuelei Fan wrote:
>
> On 7/12/2018 1:09 PM, Simone Bordet wrote:
> > Hi,
> > On Thu, Jul 12, 2018 at 10:05 PM David Lloyd wrote:
> >> But more importantly, I think this
> >> represents a chance to fix this long-standing bad behavior of the JDK.
> >
> > Indeed!
> >
> > Another thing I was about to complain is that
> > SSLSocket.shutdownOutput() still throws:
> > UnsupportedOperationException("The method shutdownOutput() is not
> > supported in SSLSocket");
> >
> This happens before JDK 11 B20, right?
No, it still happens on 11+21.
Cheers
--
Simone Bordet
http://cometd.org
http://webtide.com
Developer advice, training, services and support
from the Jetty & CometD experts.
Re: SSLEngine weird behavior in 11+21?
Hi, On Thu, Jul 12, 2018 at 9:29 PM Xuelei Fan wrote: > Per the TLS 1.3 specification: >- The server sends a dummy change_cipher_spec record immediately > after its first handshake message. This may either be after a > ServerHello or a HelloRetryRequest. > > and >- If not offering early data, the client sends a dummy > change_cipher_spec record (see the third paragraph of Section 5.1) > immediately before its second flight. This may either be before > its second ClientHello or before its encrypted handshake flight. > If offering early data, the record is placed immediately after the > first ClientHello. > > My read of the spec is that the dummy change_cipher_spec record is > generated immediately after the ServerHello in server side, and before > the 2nd flight in client side. The spec is about *sending*, while I'm about *generating*. Currently: - Server wraps 160, 6 and 907 in 3 wraps. - Server *sends* the 1073 bytes in 1 TCP write - Client unwraps 160, then goes into NEED_WRAP - Client wraps 6, then goes again into NEED_UNWRAP to finish with the 6 and 907 received by the server. What I'm suggesting: - Server wraps 160, 6 and 907 in 3 wraps. - Server *sends* the 1073 (160+6+907) bytes in 1 TCP write - Client unwraps 160, stays in NEED_UNWRAP, unwraps the 6, unwraps the 907, then goes into NEED_WRAP - Client wraps 6 and 58 - Client *sends* the 64 (6+58) bytes in 1 TCP write. The 6 bytes of the dummy change_cipher_spec are *sent* just after receiving the ServerHello in both cases, they are just *generated* at different times. By having all the unwraps() consecutive and all the wraps() consecutive you can enable a number of optimizations I described earlier. Imagine a client that would perform a TCP *send* every time the state moves away from NEED_WRAP. Currently it would: 1. TCP send for ClientHello 2. Reads 1073 from server 3. Unwrap 160, then NEED_WRAP 4. Generate 6, then NEED_UNWRAP 5. TCP send the 6 bytes 6. Unwrap 6 and 907, then NEED_WRAP 7. Generate 58 then FINISHED 8 TCP send the 58 bytes. You can see that the client issues 3 TCP sends. With what I am suggesting, the client would only issue 2 TCP sends, which seems more in line with the efforts in TLS 1.3 to be more efficient. It would just be a matter of *generating* those 6 bytes a bit later, *after* having fully unwrapped the ServerHello. > We may consider a option to turn off the middlebox compatibility mode if > it helps Jetty. No need to. > But the implementation code and the application should > be ready to accept the fact that third party's implementation may be > implemented in middlebox compatibility mode, and the change_cipher_spec > record may still come in. See above, it's not about processing the dummy change_cipher_spec, it's just about generating it a little later. > After #7 (FINISHED), if the client only send application data, but never > call read again, it still works, right? The application don't have to > read something, I think. Sure, it just can't take advantage of session resumption though. The client never knows that there are post-handshake messages to read because SSLEgine did not tell. SSLEngine just said FINISHED and NOT_HANDSHAKING. However, I see your point. Handshake messages could come at any time, just that this one seems more part of the initial handshake, especially if it's about session resumption. > in #10, you said, "Client MUST unwrap ...". Do you mean that the client > cannot send application data without read/unwrap something? It can send data, but won't be good to leave unprocessed bytes around. The application would not know that there is stuff to read from the network and to process as post-handshake messages. It may never read from the network again. > Half-close means that the server may not send the close_notify when it > receive the client close_notify. It's a very tricky policy. The client > close_notify only means the client want to close its writing side. The > server may not send the close_notify if it doesn't want to close. If we > have the client goes into NEED_UNWRAP, there is a potential dead waiting. Good point. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: SSLEngine weird behavior in 11+21?
Hi,
On Thu, Jul 12, 2018 at 10:05 PM David Lloyd wrote:
> But more importantly, I think this
> represents a chance to fix this long-standing bad behavior of the JDK.
Indeed!
Another thing I was about to complain is that
SSLSocket.shutdownOutput() still throws:
UnsupportedOperationException("The method shutdownOutput() is not
supported in SSLSocket");
Thanks!
--
Simone Bordet
http://cometd.org
http://webtide.com
Developer advice, training, services and support
from the Jetty & CometD experts.
Re: SSLEngine weird behavior in 11+21?
this point, my gut feeling is that having a single codebase handling TLS 1.2 and TLS 1.3 would be difficult. I expect a few "if (tls > 1.2)" spread out in the code because now we need additional information that we did not need before because SSLEngine and its state machine was telling the application what next step was required but now it's not always the case anymore. I know that other SSLEngine users for widely used Java network servers are around in this list :) Would be great if they can feedback about whether JDK's SSLEngine with TLS 1.3 is working well for them. Thanks! -- Simone Bordet http://cometd.org http://webtide.com Developer advice, training, services and support from the Jetty & CometD experts.
Re: SSLEngine weird behavior in 11+21?
Hi, On Thu, Jul 12, 2018 at 12:06 AM Simone Bordet wrote: > > Hi, > > I can see this (weird) behavior in SSLEngine for TLS 1.3 in JDK 11+21. > It's a simple new connection (no resumption) that performs a TLS 1.3 > handshake. > The bytes numbers are those that I get, they may be different for > others depending on certificate size, etc. > > 1. Client wraps 394 bytes then goes into NEED_UNWRAP. > 2. Server unwraps 394 bytes then goes into NEED_TASK and then into NEED_WRAP. > 3. Server wraps (in 3 wraps) 160, 6 and 907 bytes then goes into NEED_UNWRAP. > 4. Client unwraps 160 bytes then goes into NEED_TASK and then into NEED_WRAP > (?) > 5. Client wraps 6 bytes, then goes into NEED_UNWRAP. > 6. Client unwraps (in 2 unwraps) 6 and 907 bytes, then goes into > NEED_TASK and then into NEED_WRAP. > 7. Client wraps 58 bytes and goes into FINISHED (?) > 8. Server unwraps (in 2 unwraps) 6 and 58 bytes then goes into NEED_WRAP. > 9. Server wraps 72 bytes then goes into FINISHED. > 10. Client MUST unwrap those 72 bytes going again into FINISHED (which > already happened at 7). > > There are 2 things that I find weird: > > A) That at 4, the client goes into NEED_WRAP, even if it has not > finished to unwrap what the server sent. Apparently, it only goes into > NEED_WRAP to generate a CHANGE_CIPHER_SPEC (I am guessing from the > number of bytes generated), but then goes back into NEED_UNWRAP to > finish reading what the server sent. This is also not optimal as it > forces applications to do something with those 6 bytes: either put > them aside (additional data structures that may not be needed) or - > worse - write them to the server causing an additional write (after > all the effort TLS 1.3 put in to have a 1 RTT handshake). > I think that step 4 should go into NEED_UNWRAP, and that step 5 and > step 6 should be switched, so that the client would unwrap the 160, 6 > and 907 sent by the server, and only after wrapping the 6 and the 58. > To be clear, the current behavior is (u==unwrap, w=wrap): u160, w6, > u6, u907, w58. > I think it should be: u160, u6, u907, w6, w58. > > Is there any reason that the 6 bytes needs to be generated in-between > the processing of the frames sent by the server? > > B)That at 7 the client goes into FINISHED, but it is not finished > really: in fact it needs to perform step 10, but there is no > indication from the SSLEngine that it must do so. > Currently, step 7 says the client is finished and there is no clue > that it must unwrap those last 72 bytes. > I think step 7 should go into NEED_UNWRAP instead, and only at 10 go > into FINISHED. In addition to what reported above, I would like to report also the weird behavior during the close handshake (i.e. when one side decides to close the connection). 1. client.closeOutbound() then goes into NEED_WRAP. 2. Client wraps 24 bytes, result is CLOSED, then goes into NOT_HANDSHAKING (?) 3. Server unwraps 24 bytes, result is CLOSED, then goes into NEED_WRAP. 4. Server wraps 24 bytes, result is CLOSED, then goes into NOT_HANDSHAKING. 5. Client unwraps 0 bytes (?) I think at step 2 the client should go into NEED_UNWRAP to read (at step 5) the server response to the close_notify. Instead, at step 5 the client unwraps 0 bytes so we are left with those 24 bytes from the server that applications need to discard. Also, I am not sure that the wrap result at step 2 and 3 should be CLOSED, perhaps OK is better? The server is actually closed at step 4, and the client at step 5. However, this is a minor issue. Attached the debug logs as you requested. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz sslengine.log.gz Description: application/gzip
SSLEngine weird behavior in 11+21?
Hi, I can see this (weird) behavior in SSLEngine for TLS 1.3 in JDK 11+21. It's a simple new connection (no resumption) that performs a TLS 1.3 handshake. The bytes numbers are those that I get, they may be different for others depending on certificate size, etc. 1. Client wraps 394 bytes then goes into NEED_UNWRAP. 2. Server unwraps 394 bytes then goes into NEED_TASK and then into NEED_WRAP. 3. Server wraps (in 3 wraps) 160, 6 and 907 bytes then goes into NEED_UNWRAP. 4. Client unwraps 160 bytes then goes into NEED_TASK and then into NEED_WRAP (?) 5. Client wraps 6 bytes, then goes into NEED_UNWRAP. 6. Client unwraps (in 2 unwraps) 6 and 907 bytes, then goes into NEED_TASK and then into NEED_WRAP. 7. Client wraps 58 bytes and goes into FINISHED (?) 8. Server unwraps (in 2 unwraps) 6 and 58 bytes then goes into NEED_WRAP. 9. Server wraps 72 bytes then goes into FINISHED. 10. Client MUST unwrap those 72 bytes going again into FINISHED (which already happened at 7). There are 2 things that I find weird: A) That at 4, the client goes into NEED_WRAP, even if it has not finished to unwrap what the server sent. Apparently, it only goes into NEED_WRAP to generate a CHANGE_CIPHER_SPEC (I am guessing from the number of bytes generated), but then goes back into NEED_UNWRAP to finish reading what the server sent. This is also not optimal as it forces applications to do something with those 6 bytes: either put them aside (additional data structures that may not be needed) or - worse - write them to the server causing an additional write (after all the effort TLS 1.3 put in to have a 1 RTT handshake). I think that step 4 should go into NEED_UNWRAP, and that step 5 and step 6 should be switched, so that the client would unwrap the 160, 6 and 907 sent by the server, and only after wrapping the 6 and the 58. To be clear, the current behavior is (u==unwrap, w=wrap): u160, w6, u6, u907, w58. I think it should be: u160, u6, u907, w6, w58. Is there any reason that the 6 bytes needs to be generated in-between the processing of the frames sent by the server? B)That at 7 the client goes into FINISHED, but it is not finished really: in fact it needs to perform step 10, but there is no indication from the SSLEngine that it must do so. Currently, step 7 says the client is finished and there is no clue that it must unwrap those last 72 bytes. I think step 7 should go into NEED_UNWRAP instead, and only at 10 go into FINISHED. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: JDK 11+21 SSLSocket.close() deadlock?
Hi,
On Wed, Jul 11, 2018 at 2:25 AM Xuelei Fan wrote:
>
> Hi Simone,
>
> Thank you for reporting this issue. Now it is tracked in JBS:
> https://bugs.openjdk.java.net/browse/JDK-8207004
>
> In the following stacks, only one lock (on 0xac) can be observed. Can I
> understand that the read() is blocked, and then the close() is blocked
> as well? Did you have a test that we can use to reproduce this issue?
Yes, it's pretty crude but reduces at minimum the dependencies.
You have to provide a valid sslContext variable, but that should be
done for your environment.
Looking at sun.security.ssl.SSLSocketImpl.close(), I see that it tries
to grab the lock for read and then for write.
While the test case below shows a problem for reads, I believe the
same problem can be shown when the SSLSocket is blocked in writes due
to TCP congestion, and some other thread tries to asynchronously
close() it.
Thanks!
public void testDeadlock() throws Exception
{
int port = 8899;
// TODO: provide a valid sslContext here.
SSLServerSocket acceptor =
(SSLServerSocket)sslContext.getServerSocketFactory().createServerSocket(port);
Socket client = new Socket("localhost", port);
SSLSocket sslServer = (SSLSocket)acceptor.accept();
CompletableFuture.runAsync(() ->
{
try
{
sslServer.startHandshake();
}
catch (Throwable x)
{
x.printStackTrace();
}
});
// Wait for sslServer to start reading from the client.
Thread.sleep(1000);
// Hangs here.
sslServer.close();
client.close();
}
--
Simone Bordet
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless. Victoria Livschitz
JDK 11+21 SSLSocket.close() deadlock?
Hi, Please look at the stack traces below. The server code accept() a SSLSocket, then calls startHandshake() in a different thread. The client code sends the TLS handshake bytes very slowly. The server code waits for a bit for the handshake to finish, then attempts to close the socket, but it cannot due to the deadlock below. The client stops sending the TLS handshake bytes and now the server is stuck forever. I think invoking close() should wake up the read() with an exception, rather than trying to grab the same lock held by the read, as asynchronous closes from other threads happen all the times. The issue does not happen with JDK 9, 10, 11-ea+18, but does happen with 11-ea+21. Thanks! --- "main@1" prio=5 tid=0x1 nid=NA waiting for monitor entry java.lang.Thread.State: BLOCKED waiting for pool-1-thread-1@2224 to release lock on <0xac7> (a sun.security.ssl.SSLSocketImpl) at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:447) "pool-1-thread-1@2224" prio=5 tid=0x19 nid=NA runnable java.lang.Thread.State: RUNNABLE blocks main@1 at java.net.SocketInputStream.socketRead0(SocketInputStream.java:-1) at java.net.SocketInputStream.socketRead(SocketInputStream.java:115) at java.net.SocketInputStream.read(SocketInputStream.java:168) at java.net.SocketInputStream.read(SocketInputStream.java:140) at sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:464) at sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:167) at sun.security.ssl.SSLTransport.decode(SSLTransport.java:108) at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:877) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:810) - locked <0xac7> (a sun.security.ssl.SSLSocketImpl) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:383) -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: Feedback on SSLEngine.setHandshakeApplicationProtocolSelector()
Hi, On Wed, Jan 11, 2017 at 5:57 PM, Simone Bordet <simone.bor...@gmail.com> wrote: > Hi, > > I just wanted to report that I have implemented the new mechanism > provided by SSLEngine.setHandshakeApplicationProtocolSelector() in > Jetty, and it works well in a much much simpler way. > > The amount of code required now is trivial (a couple of lines), there > is no need to parse the ClientHello, and we basically tap into the > existing mechanism that Jetty had. > No big surprises here since the new JDK mechanism is very similar to > what Jetty has for JDK 8. For the record, I have implemented also the client-side mechanism using the JDK 9 APIs. Looks simple enough and works fine. -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Feedback on SSLEngine.setHandshakeApplicationProtocolSelector()
Hi, I just wanted to report that I have implemented the new mechanism provided by SSLEngine.setHandshakeApplicationProtocolSelector() in Jetty, and it works well in a much much simpler way. The amount of code required now is trivial (a couple of lines), there is no need to parse the ClientHello, and we basically tap into the existing mechanism that Jetty had. No big surprises here since the new JDK mechanism is very similar to what Jetty has for JDK 8. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS-PSK status ?
Hi, On Tue, Jan 3, 2017 at 6:50 PM, Xuelei Fan <xuelei@oracle.com> wrote: > Now JDK-8049402 is public. > > https://bugs.openjdk.java.net/browse/JDK-8049402 Thanks ! The issue is marked as "fix for 10", so is it correct to assume that nothing will be done in time for JDK 9 or even 8 (via backport) ? It is just to know and to respond to people that asked about supporting TLS-PSK support. Thanks again ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
TLS-PSK status ?
Hi, https://bugs.openjdk.java.net/browse/JDK-6476446 is closed as a duplicate of JDK-8049402, but unfortunately this latter issue is not public. Is there any status about the implementation of TLS-PSK (RFC 4279, 4785, 5489) ? Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: [9] RFR 8171443: (spec) An ALPN callback function may also ignore ALPN
Vincent, On Thu, Dec 22, 2016 at 11:38 AM, Vincent Ryan <vincent.x.r...@oracle.com> wrote: > Please review this spec change to allow an ALPN callback function to also > disable ALPN usage > and return no ALPN extension value during a TLS handshake. As I understand it, the callback needs to convey 3 results: 1. a success -> non empty string 2. a failure -> null 3. no action -> empty string I wonder if it is better to convey the failure by throwing an exception ? This would also match the case where the implementation of the function, for any reason, throws an unexpected exception such as NPE or ClassCastException, etc. I expect the SSLEngine implementation to catch Throwable from the invocation of the callback and send back a TLS close message with code 120. If the failure case is conveyed with an exception, then only 2 cases remain, and then null can be used to signal "no action" ? The SSLEngine implementation should also check that the String returned is among those sent by the other peer, so an empty string is as invalid as the string "no_proto" - hence the choice of null to signal "no action". Makes sense ? Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: [9] RFR 8170282: Enable ALPN parameters to be supplied during the TLS handshake
Hi, On Fri, Dec 9, 2016 at 3:09 AM, David M. Lloyd <david.ll...@redhat.com> wrote: > On 12/08/2016 07:34 PM, Bradford Wetmore wrote: >> >> Hi, >> >> Vinnie wrote: >>> >>> We understood when we examined these issues last year that the >>> existing ALPN API would be sufficient. However it transpired, >>> following HTTP2 server implementation efforts, that a particular >>> use-case was difficult to address without sacrificing performance. >> >> >> A few more details. >> >> The discussion/decisions we had last year ended with requiring servers >> to inspect ClientHellos for the client's max TLS >> protocol/ciphersuites/SNI/ALPNs/etc, and then do any necessary >> configuration before actually starting the SSL/TLS/DTLS handshake on the >> server side. That included enabling specific SSL/TLS/DTLS protocols, >> ordering the ciphersuites (calling >> SSLParameters.setUseCipherSuitesOrder()) and ordering the ALPN values. >> If it turned out that the resulting selected combination was incorrect, > > > How could it turn out to be incorrect, when you know in advance what > credentials you have, what cipher suites are available, and what protocols > support which cipher suites? If any logic you would run is different from JDK's, then the result may be incorrect. Say it another way, you run an old version of your code in a new JDK version: the logic that was ok long time ago may not be the same years later, and you get different results. > I'm just curious because I'm pretty sure we > haven't encountered this (I'll ask our web server folks to be sure though). That would be helpful. Apart us (Jetty), nobody has actually implemented ALPN using the new JDK9 API yet, as far as I know. > Also I'd hate to see a worst-case situation where a this API was introduced > (which is very reminiscent of ideas which were rejected for good reasons), > and it turns out that in the end it's not quite sufficient after all, > leaving a bit of dead code around. >From how I understand this change, the whole mechanism of parsing the ClientHello, running your logic, running again the ClientHello parsing and the JDK logic, resulting in calling the application callbacks (for SNI for example) twice, checking that both logics yielded the same result, and if not take recovery actions, stays there, if you want to go that way. You don't have to use the new code. -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: Issues with ALPN implementation in JDK 9
Hi, On Wed, Jun 15, 2016 at 8:12 PM, Jason Greene <jason.gre...@redhat.com> wrote: > Here is an H2 example: > > The ClientHello expressing interest in both h2, and h1, and lists the ciphers: > TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 > TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 > > So in this case the client is expressing a preference for a CBC cipher using > stronger keys, but is blacklisted by h2 due to not being GCM, but this is > allowed by h1. The JDK will follow the client hello as the TLS spec > describes, and so it will select TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, and > the protocol stack will incorrectly select h1. > > Now technically H2 clients can avoid this by ordering h2 compliant ciphers on > top, but they might not do so. > > The flaw really starts to show when you have multiple alternative protocols > with completely different requirements. As an example, if you have a > hypothetical protocol which does not perform certificate based > authentication, and instead utilizes pre-shared keys or anonymous ciphers, > that protocol can never be selected unless those algs are on the top of the > list, if they are on the top of the list then h2 can’t be selected. > I'm not sure what you want to show here. If you want to prefer ciphers, you have to give up on protocol and viceversa, that's normal business. If there is no decision, then there is a conflict; if there is a decision, then the solution I propose works equally well to what you guys propose. There is no "flaw", it is just undecidable. What am I missing ? Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: Issues with ALPN implementation in JDK 9
Hi, On Wed, Jun 15, 2016 at 1:31 PM, David M. Lloyd <david.ll...@redhat.com> wrote: > The problem is that this is quite subjective. In my opinion the current > solution is both clean and precise: > > 1) Read the hello packet > 2) Examine the protocols, host names, and cipher suites > 3) Apply arbitrary user policy to select the enabled cipher suites and even > possibly the SSL context to use based on: >* The above information >* The supported cipher suites (for each SSL context, if multiple) >* The available host key types for each host and/or protocol >* Any user-driven factor that goes beyond *just* HTTP 2 Sure, that's the duplication of what the JDK does. Change logic in the JDK, change logic above. How do you avoid to run the application SNI logic twice, once by you, and once by the JDK ? What if that SNI logic, provided by an external application, has side effects ? Your bullets above are "precise" only under the assumption that those bullets are exactly what the JDK does. If it's not the case, then they're broken. I understand for you is not a big deal to replicate the JDK logic and keeping it up to date, and banning certain versions of your code to run with certain versions of the JDK. I hope you understand it's not the case for others; for example, we don't control with what JDK versions people run Jetty. If JDK 10 changes the logic, we will have to say: "Oh, sorry, you cannot run Jetty 9 (the current version) with JDK 10". And we won't be able to change the logic in Jetty 9, because we would break previous JDKs. We will be back to sniffing what JDK it is. Yuck. > No matter what the JDK does, it will not be able to encapsulate a completely > arbitrary user policy - certainly not precisely - during handshake, nor will > it be able to allow for the selection of different SSLContexts based on that > policy (let alone proxying the connection). Can you detail a case ? > The best that can be done in the > JDK, in my opinion, is to provide various utilities to simplify implementing > the above: a helpful SSL explorer style class, which includes mapping the > protocol and cipher suite list (which I believe we agree on) and perhaps > providing information about the key type(s) supported by each cipher suite, > so the user can compare this against the available key types. Unfortunately this SSL Explorer class has not yet been provided. If it was, it would be helpful. But I think it will just hit the same problems I hit: logic duplication and running SNI twice. > Once you know what cipher suites are offered, and you have selected the > protocol, host, and enabled cipher suites, there isn't a great deal of > mystery left in the JDK: it simply carries out the negotiation that you > specified. IFF your logic is the same as the JDK. I don't want to check this every time a JDK release is made. Why you enjoy doing this this check, it's a mystery to me :) > I agree with 1 and 2, at least in concept if not in specifics, but I think 3 > is a mistake: > if you've committed to an SSL context then it's already too late to make some > possibly important decisions, and SNI can play a factor here too. > We can make this easier but I think that trying to do this all inside of the > handshake process is going to cut off some important use cases. Which use cases ? BTW, the delay of handshaker.started=true would be irrelevant to the TLS mechanism, it's just an implementation detail. The information that needs to be reported in the ServerHello is only needed when the ServerHello is being constructed. Had the JDK implementation been written with a delayed handshaker.started=true we would have the best of both worlds. You could write your own solution, we could write our own. Yours will have some restrictions, and so will ours (just different ones). Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: Issues with ALPN implementation in JDK 9
Hi, On Wed, Jun 15, 2016 at 10:15 AM, Andrew Haley <a...@redhat.com> wrote: > The problem I have with reading posts about JDK9 problems is that I > can't tell the *severity* of the problems. I don't know if something > is a total blocker or an inconvenience. When someone uses an internal > sun.* class they may be doing something which must be done in order to > get access which would be impossible otherwise. I don't understand this comment. The main issue of this thread is the current impossibility to have a clean and precise ALPN implementation with minimal code. Sure we can fallback to less optimal solutions that won't work in all cases. Sure we can duplicate the JDK logic and keep it up-to-date, invoke SNI twice, etc. But I don't see the point of settling for a less than optimal solution when there is room to make it better, and the effort is minimal. Is this issue a blocker ? Surely not, it is possible to rewrite from scratch a JSSE provider. Would I do that ? No, thanks. I am proposing 3 simple changes to be done: 1) Introduce a TLSProtocolNames class that can convert from TLS protocol bytes to TLS protocol names. 2) Introduce a TLSCipherSuiteNames class that can convert from TLS cipher suite bytes to TLS cipher suite names. 3) Delay handshaker.started=true so that it would simpler and more precise for an application to handle ALPN. I think these changes will benefit all the people involved in network servers having to play with ALPN. We *are* still in time, according to Mark Reinhold. The changes are simple. We have not heard from Oracle yet, but I can prepare a webrev if that helps. -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: Issues with ALPN implementation in JDK 9
Hi, On Wed, Jun 15, 2016 at 10:03 AM, Andrew Haley <a...@redhat.com> wrote: > On 14/06/16 14:57, Simone Bordet wrote: >> * Lack of facilities to convert TLS protocol bytes to protocol strings. >> This class already exist in JDK, sun.security.ssl.ProtocolVersion, it >> would just need to be exposed in javax.net.ssl. >> >> * Lack of facilities to convert TLS cipher bytes to cipher name strings. >> As above, sun.security.ssl.CipherSuite exists, needs to be exposed publicly. >> >> Note that for the 2 bullets above, a recent message from Mark Reinhold >> to jdk9-dev confirmed that JDK 9 is *not yet* feature complete, so I >> hope they can be considered for inclusion. > > It's very late, It's not too late, like Mark Reinhold said. > and they certainly won't be considered unless > someone proposes it. I just did. > But can't we simply clone this class anyway? Sure, but then every ALPN implementer out there will have their own, they will need to be kept up to date when a new TLS protocol or a new cipher is introduced in the JDK, etc. etc. The TLS ciphers in the JDK have non-standard names for historical reasons, etc. The classes are trivial, and can be tiny wrappers around the existing ones like TLSProtocolNames and CipherSuiteNames. > Incidentally, I can't tell which (if any) of the issues you describe > are specific to JDK 9 (i.e. they would not be problems in JDK 8.) Not sure what you mean here. Unless you parse the TLS protocol in JDK 8, there is no particular need for those classes. With JDK9 we now need to parse the TLS protocol to handle ALPN, so those classes would be handy. -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: Issues with ALPN implementation in JDK 9
Hi, On Tue, Jun 14, 2016 at 8:11 PM, Jason Greene <jason.gre...@redhat.com> wrote: > If the case is that H3 blacks all RSA, then thats an easy test right? Just > verify that keystore has an ECDSA key. I don't think it is that easy. The server logic has to mimic exactly what the JDK logic is, that is to verify the cipher and certificate compatibility. Not only, the server has to run the logic for SNI, the same logic that the JDK runs to get the right certificate. This logic would now run twice. As I said, it is doable as long as the server logic duplicates and keeps in sync with the JDK logic so that it is guaranteed that the application protocol chosen by the server ends up with a cipher, chosen by the JDK, valid for that protocol. The moment the logic is different, there is a problem. Every time the JDK updates, you and me have to go and look inside the JDK to check whether the logic is changed, and if so, update our servers. And then we have to say that version X of our server only works with JDKs up to version Y, and that version X+1 of our server only works with JDK version Y+1. Been there, done that. I'd like to move to a better model that is future proof. I don't control which JDK people use to run Jetty. I would rather not duplicate all the JDK logic into an application. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: Issues with ALPN implementation in JDK 9
Hi, On Tue, Jun 14, 2016 at 5:58 PM, Simone Bordet <simone.bor...@gmail.com> wrote: > The server could choose an ECDSA cipher for h3, the mandatory cipher > for h2, so the list of cipher is (ECDSA, RSA). If, at this point, the > server chooses the protocol *before* the JDK chooses the cipher, it > may think that h3 is a good choice, but the TLS stack chooses the RSA > cipher. > At this point, you have the h3 protocol with the RSA cipher, which may > be an invalid combination. Just to add more information here. If the server logic chooses h3 and also restricts the ciphers to only (ECDSA) to make sure that h3 is chosen, the JDK may find that the cipher is not good for the certificate, and therefore there are no ciphers in common and the connection is terminated. There is no negotiation, although h2 with RSA would have been a valid combination. If handshaker.started=true is delayed, the server could wait for the JDK to choose a cipher, and then a suitable protocol for that cipher, which in the example would be h2, and the negotiation would be successful. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: Issues with ALPN implementation in JDK 9
Hi, On Tue, Jun 14, 2016 at 5:26 PM, Jason T. Greene <jason.gre...@redhat.com> wrote: > With H2 all impls are required to support > TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 Unless it is discovered vulnerable. > so that should be selectable either by your protocol stack or the TLS layer. > If you wanted to be especially lazy you could just exclude any blacklisted > cipher list presented in the client hello, and verify that additional ones > are present, and if so set the rest as the available cipher set in SSL > params. The TLS stack will pick appropriately at that point. Not in general, see my example. The server could choose an ECDSA cipher for h3, the mandatory cipher for h2, so the list of cipher is (ECDSA, RSA). If, at this point, the server chooses the protocol *before* the JDK chooses the cipher, it may think that h3 is a good choice, but the TLS stack chooses the RSA cipher. At this point, you have the h3 protocol with the RSA cipher, which may be an invalid combination. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: Issues with ALPN implementation in JDK 9
Hi, On Tue, Jun 14, 2016 at 4:31 PM, David M. Lloyd <david.ll...@redhat.com> wrote: >> During the unwrap(), the JDK implementation picks a cipher based on >> the JDK logic. >> In particular, in my case, I had a keystore with a certificate that >> was *not* ECDSA. > > Could you not use the available keys as an input into your own protocol > selection? Granted you have to know what kind of key works with what > algorithm, but if you already have a table of cipher suites, you might as > well just add it on there... By "keys" here you mean, exactly ? >> However, this means duplicating all the JDK logic to make sure that >> the server logic *before* calling unwrap() is the same of the JDK so >> that when unwrap() is called there will be no failures. > > I don't think you have to duplicate the exact logic though. It's not really > a "black box": if you know the cipher suites supported by your available > key(s) then you should have enough information to know, based on the client > cipher suites and the per-protocol suites, and the cipher suites available > in the SSL context(s), which protocols can complete. Well, perhaps. My point is that I implement some logic in the server, say version 10. Server 10 works with JDK 9.1.0. Time passes, JDK gets updated to 9.1.173, which has now a different logic, for whatever reason. Try to run server 10 with JDK 9.1.173 does not negotiate the right protocol. >> I don't think this is maintainable; the JDK is entitled to change the >> logic following CVEs, optimizations and what not, and each such change >> risks to break existing server code. > > > What kind of change do you anticipate being a breaking change? Are you > thinking of e.g. blacklisting some known-bad cipher suite or something? Well, take ECDSA for instance. Before that, my logic on server version 10 was not caring about ECDSA. My server version 10 was working with JDK 5, but not with JDK 9 (assuming that JDK 5 did not have support for ECDSA). That is a breaking change to me. I imagine in the future ECDSA be replaced by something else, or a different DSA being used, I have to change the server code. I would like to avoid that, and keep the server logic independent of how the JDK chooses the cipher. Makes sense ? Thanks ! -- Simone Bordet http://cometd.org http://webtide.com Developer advice, training, services and support from the Jetty & CometD experts.
Issues with ALPN implementation in JDK 9
re not copied into the handshaker and are not used to generate the ServerHello. Conclusions. I could not make a reliable ALPN implementation with the current JDK 9. If I am off road, then that's good news, and I will be all ears for alternative approaches. If I am correct, I would like to discuss whether it would be possible to delay handshaker.started=true to a later time, so that SSLParameters can be changed just after the NEED_TASK step, so that server applications will be able to interact with the JDK for what pertains TLS protocols, ciphers, SNI, etc. without duplicating the logic. Comments welcome. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: Request for review: 8144093: JEP 244/8051498 - TLS Application-Layer Protocol Negotiation Extension
Hi, On Thu, Jun 2, 2016 at 8:20 PM, Bradford Wetmore <bradford.wetm...@oracle.com> wrote: > Hi Simone, > > I agree with you, it would be handy. As part of the HTTPS 2.0 prohibited > suites for ALPN, I wanted to do just such a convenience mapping in > javax.net.ssl.StandardConstants.java: > > > http://cr.openjdk.java.net/~wetmore/8051498/webrev.16/src/java.base/share/classes/javax/net/ssl/StandardConstants.java.frames.html > > public static final Map<Integer, String> cipherSuiteIntToString > > but was shot down and finally gave up. We'd need to consider more how to > expose some of the TLS internals that we don't currently do right now. > (e.g. at the API layer or as a provider (SPI) call.) > > One of the problems you may not have noticed is also historical, in that > JSSE was first released around the time of the finalization of TLSv1, so > there is a mix of SSL_/TLS_ prefix for the older algorithms. The TLS > standard name String "TLS_RSA_WITH_3DES_EDE_CBC_SHA" is actually the > SSLv3/Java Standard Name "SSL_RSA_WITH_3DES_EDE_CBC_SHA", which was set > during the original SSLv3 spec, which later became the "Historic" RFC 6101. > > FYI, in JDK 9, you will lose the ability to call valueOf() due to modular > encapsulation. > > We can file a RFE to do this. Since JDK 9 just went FC last week, it will > likely need to be a JDK 10 RFE. Oh, no ! On net-dev we are still talking about changing the APIs for HttpClient and WebSocket client, so can I please ask that this does not get delayed another 5 years ? We explicitly asked on net-dev if there was a freeze, and they replied that it was still possible to change the APIs, so I presume it would be possible to add a simple class to javax.net.ssl ? It's a super simple class to expose, call it CipherNames, whose implementation will be trivial and based on sun.security.ssl.CipherSuite. Right now I am doing the same accessing CipherSuite via reflection on its private members. I don't know if I am late to the party or way too early, but I cannot believe that all others that are interested in ALPN in Java 9 went for a workaround instead of asking here whether the cipher names could have been exposed in a standard way. Considering the historical issue you mention it is important that there are no ambiguities in the cipher names, and having the JDK as the only source is really important. Can I ask you to reconsider this small change for JDK 9 ? -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: Request for review: 8144093: JEP 244/8051498 - TLS Application-Layer Protocol Negotiation Extension
Hi,
On Wed, Dec 2, 2015 at 1:04 AM, Bradford Wetmore
<bradford.wetm...@oracle.com> wrote:
> Filling in a few more details [for server parsing]
>
> SSLContext.getInstance("protocol"); // returns a context with
> // "protocol" and possibly
> // other protocols enabled.
> SSLEngine ssle = SSLContext.createSSLEngine(...);
>
> Read ClientHello from Socket/SocketChannel/AsynchronousSocketChannel/etc.
>
> Parse ClientHello for requested protocol/alpn/ciphersuites
>
> choose protocol/alpn/ciphersuite value(s)
I'm trying to give this a go in Jetty and I stumbled into this
problem: when Jetty parses the ClientHello, it parses the ciphers in
the TLS format of 2 bytes.
For example, the ClientHello contains the cipher (0x00, 0x00).
I would need a way to convert the 2-bytes duple into the corresponding
cipher string.
For this example, it would be: "TLS_NULL_WITH_NULL_NULL".
Turns out that the JDK can do this via:
CipherSuite cipher = sun.security.ssl.CipherSuite.valueOf(byte1, byte2);
String name = cipher.name;
Unfortunately, class CipherSuite is package local, and in sun.* package.
Would be great if there was a standard way of going from the duple to
the string.
Having the cipher string would allow applications to compare what
parsed with application configurations strings, and it would match
with methods such as SSLEngine.getEnabledCipherSuites(), etc. which
handle String[].
Having the JDK to provide this facility would guarantee that the
naming is always consistent, and that it is always up-to-date with the
JDK in use.
Looking forward to your comments.
Thanks !
--
Simone Bordet
http://bordet.blogspot.com
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal v7
Bradford, On Sat, Oct 3, 2015 at 2:19 AM, Bradford Wetmore <bradford.wetm...@oracle.com> wrote: > Thanks for the comments everyone. I'm submitting the following to the CCC > (internal review board): > > http://cr.openjdk.java.net/~wetmore/8051498/webrev.17/ > > Changes: > > 1. No H2 Blacklist/Comparator > > 2. set/getApplicationProtocols() back to SSLParameters. Have you implemented this solution already ? Also for clients ? Do you have feedback on actually implementing ALPN in this way ? -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal v5
David, sorry, but I don't understand your proposal. Can you please write it down in pseudo code what a server application should do and what the JDK should do to negotiate HTTP/2 with a client ? I don't see how it is even possible for a user to decide anything *before* the handshaking is even initiated, like you say. It obviously does not have enough information. This is the current algorithm (A=app code, J=JDK code): A: sslParameters.setApplicationProtocols(H2, H1); A: (optional, only needed for H2) sort ciphers to favor H2 A: start handshake J: receive ClientHello J: ciphers = intersect client/server ciphers J: aps = intersect client/server application protocols J: for each cipher in ciphers J:for each ap in aps J:if (ap.matches(cipher & other info)) break J end // aps [A: ap.matches() calls into application code to return whether ap is good for the given info] J:if (ap was not selected) continue; // to next cipher J:if (trySetCipherSuite(cipher)) break; // success J: end // ciphers J: send ServerHello J: terminate handshake A: sslEngine/sslSocket.getApplicationProtocol() Note that the JDK provides default implementations for H1 and H2 for ap.matches(), but in general these will be implemented by application code. For an application that wants to support H2, this boils down to the first 2 lines, the rest is in the JDK. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal v5
Hi,
On Fri, Sep 25, 2015 at 3:44 AM, Xuelei Fan <xuelei@oracle.com> wrote:
> For example, a client wants to negotiate {HTTP2, HTTP1.1} or {HTTP1.1,
> HTTP2} and {TLS_RSA_WITH_AES_128_CBC_SHA,
> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384}.
> HTTP1.1/TLS_RSA_WITH_AES_128_CBC_SHA should be negotiated per the TLS
> and HTTP2 specification. If the cipher suites are sorted,
> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 would be negotiated, this is not
> what the customer really expected. The customer preference should be
> respected.
>
> I don't think we really need to re-order the cipher suites.
"We" as in the OpenJDK implementation *must not* reorder the cipher suites.
"We" as in an application that wants to use HTTP/2 *must* reorder the
cipher suites.
The comparator provided is only a help to the application to perform
this reordering.
> Let's consider the following client requests (prefer cipher suite more
> than application protocol; blacklisted_CS are HTTP2 blacklisted cipher
> suite):
>
> 1. {HTTP2, HTTP1.1} {strong_cipher_site, blacklisted_CS}
> HTTP2 and strong_cipher_site should be negotiated. Need not to re-order
> cipher suites.
>
> 2. {HTTP1.1, HTTP2} {strong_cipher_site, blacklisted_CS}
> HTTP1.1 and strong_cipher_site should be negotiated. Need not to
> re-order cipher suites.
>
> 3. {HTTP2, HTTP1.1} {blacklisted_CS, strong_cipher_site}
> HTTP1.1 and blacklisted_CS should be negotiated. Need not to re-order
> cipher suites.
Of course you need to re-order in this case.
The application has just provided a preference for HTTP/2 in the
application protocol list, it actually happens that HTTP/2 could be
negotiated because strong ciphers are present, but instead HTTP/1.1 is
chosen.
> 4. {HTTP1.1, HTTP2} {blacklisted_CS, strong_cipher_site}
> HTTP1.1 and blacklisted_CS should be negotiated. Need not to re-order
> cipher suites.
>
> 5. {HTTP2} {strong_cipher_site, blacklisted_CS}
> HTTP2 and strong_cipher_site should be negotiated. Need not to re-order
> cipher suites.
>
> 6. {HTTP1.1} {strong_cipher_site, blacklisted_CS}
> HTTP1.1 and strong_cipher_site should be negotiated. Need not to
> re-order cipher suites.
>
> 7. {HTTP2} {blacklisted_CS, strong_cipher_site}
> blacklisted_CS would be filtered out as it does not appy to HTTP2. Only
> strong_cipher_site presents in ClientHello message.
>
> HTTP2 and strong_cipher_site should be negotiated. Need not to re-order
> cipher suites.
>
> 8. {HTTP1.1} {blacklisted_CS, strong_cipher_site}
> HTTP1.1 and blacklisted_CS should be negotiated. Need not to re-order
> cipher suites.
>
> One concern may be that, the customer is not intent to negotiate HTTP2
> blacklisted cipher suite. The customer just don't know which are the
> strong cipher suites among many cipher suites. I think we may need a
> handy tool to order the cipher suites before configuration.
>
> // there are a few cipher suites are available
> String[] cipherSuites = ... // a array of cipher suites.
>
> // Q: Don't know the strength of them
> // A: OK, there is a handy tool
> cipherSuites = cipherSuiteReorder.sort(cipherSuites);
Or, with the comparator:
cipherSuites = Arrays.sort(cipherSuites,
ApplicationProtocol.H2.CIPHER_COMPARATOR);
The comparator is the handy tool.
> // configure the cipher suites
> SSLParameters.setCipherSuites(cipherSuites);
>
> The order also apply to the normally cipher suites configuration, not
> only to application protocols. The re-order should be called by
> customers explicitly. JSSE would better not sort them automatically.
>
> I think, the handy sort tool cannot be simply bind to application
> protocol. For example, HTTP2 has a blacklisted cipher suites. OK,
> ApplicationProtocol.H2BLACKLISTCOMPARATOR is expected to make the sort.
> If, in the future, a new application protocol (AP_NEW) has a different
> blacklist cipher suites, a new
> ApplicationProtocol.APNEWBLACKLISTCOMPARATOR would be defined. If both
> {HTTP2, AP_NEW} would be requested, which comparator for the sorting
> would be used? None of them can sort the cipher suite properly. The
> comparator design will not work any more.
Sure it does.
Because you explicitly set a preference in the order of the
application protocol, you only sort to favour the best protocol.
In case you have [HTTP/2, AP_NEW, HTTP/1.1], then you can simply
compose the comparators to sort first with the H2.CIPHER_COMPARATOR,
then with AP_NEW.CIPHER_COMPARATOR.
cipherSuites = Arrays.sort(cipherSuites,
ApplicationProtocol.H2.CIPHER_COMPARATOR.thenComparing(AP_NEW.CIPHER_COMPARATOR));
> We may need a handy tool for the sorting for stronger cipher suites.
> But, AFAIU, the tool does not belong to ALPN.
Indeed, it belongs to ApplicationProtocol.H2, not to ApplicationProtocol.
--
Simone Bordet
http://cometd.org
http://webtide.com
Developer advice, training, services and support
from the Jetty & CometD experts.
Re: TLS ALPN Proposal v5
Hi, On Fri, Sep 25, 2015 at 11:47 AM, Xuelei Fan <xuelei@oracle.com> wrote: > Here is the question to answer, which preference should be respected > firstly between cipher suite and application protocol? If application > protocol are preferred at first, of course, application preference > should be respected at first; otherwise, cipher suite preference should > be respected at first. The answer to this question has been decided when the algorithm has been chosen to be: for each cipher for each application protocol end end All the rest being equal, ciphers dominate application protocol selection. Are you suggesting to change this to: for each application protocol for each cipher end end ? It's in the hands of the role that configures application protocols and ciphers to decide whether it's more important to prefer a protocol or a cipher. Put it in a different way: If the role prefers application protocols, it has to sort the ciphers to influence that. If the role prefers ciphers, it has to sort the ciphers. No matter what, it has to sort the ciphers. > Therefore, personally, I think application may want a handy tool to sort > the cipher suite for the strength for general purpose, but not for > application protocol. Because HTTP/2 would probably be popular given the success of its predecessor, it would be handy to have a HTTP/2 comparator to influence the selection of the HTTP/2 protocol. Nothing forbids to offer a comparator by cipher strength too. -- Simone Bordet http://cometd.org http://webtide.com Developer advice, training, services and support from the Jetty & CometD experts.
Re: TLS ALPN Proposal v5
Hi,
On Fri, Sep 25, 2015 at 3:20 PM, Xuelei Fan <xuelei@oracle.com> wrote:
> For the complication, I posted the comments in previous mail here:
>
> -
>> In case you have [HTTP/2, AP_NEW, HTTP/1.1], then you can simply
>> compose the comparators to sort first with the H2.CIPHER_COMPARATOR,
>> then with AP_NEW.CIPHER_COMPARATOR.
>>
>> cipherSuites = Arrays.sort(cipherSuites,
>>ApplicationProtocol.H2.CIPHER_COMPARATOR.
>>thenComparing(AP_NEW.CIPHER_COMPARATOR));
>>
> Let's look at an example. application_protocol_1 prefer cipher_suite_1,
> and application_protocol_1 prefer cipher_suite_2.
>
> The comparator for application_protocol_1 would set the preference as
> {cipher_suite_1, cipher_suite_2}. and the comparator for
> application_protocol_2} would set the preference as {cipher_suite_2,
> cipher_suite_1}.
>
> The result to sort 1 and then 2, and the result to sort 2 and then 1 are
> different.
>
> The call sequence to the comparators, and the call to each comparator
> would result in difference result. That's may be not the expected behavior.
The example is malformed, since it does not specify which ciphers are
good for which application protocol, and neither the order of the
application protocols.
Let me rewrite it:
application protocols: [ap1, ap2]
ciphers: [c1, c2]
ap1 requires c1, does not work with c2
ap2 requires c2, does not work with c1
Now the question is: you have to configure your system, what you want to do ?
If you want to favor ap1, then you sort [c1, c2]
If you want to favor ap2, then you sort [c2, c1]
If you want to favor c1, then you sort [c1, c2]
If you want to favor c2, then you sort [c2, c1]
If you want to favor ap1 *and* c2, you have to decide what is more
important between the two, because you cannot have both.
I don't see any problem, really.
That the results are different, sure, but they are predictable.
When the configuration uses one comparator, it will always be that
result, and same for the other comparator.
But you configure the comparators in base of what you want to do.
--
Simone Bordet
http://bordet.blogspot.com
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless. Victoria Livschitz
[no subject]
Hi, On Fri, Sep 25, 2015 at 4:19 PM, Xuelei Fan <xuelei@oracle.com> wrote: > Actually, it was a puzzle to me: whether a concrete server can support > both HTTP/2 and HTTP/1.1, or not. If HTTP/2 mode of the server does not > work, is it OK to fall-over to use HTTP/1.1 mode? I did not get the > answer from the HTTP/2 spec. Yes, there was an answer to Bradford's email: https://lists.w3.org/Archives/Public/ietf-http-wg/2015JulSep/0160.html. The answer is "yes". -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal v5
Hi, On Fri, Sep 25, 2015 at 1:45 AM, Bradford Wetmore <bradford.wetm...@oracle.com> wrote: > I think that a textual name will be better than: > > // Output: javax.net.ssl.ApplicationProtocol$1@1b9e1916 > > System.out.println(ApplicationProtocol.H2); > > and there's no UTF-8 ambiguity. Sure, but then I would just keep getNetworkSequence(), and remove getProtocolName(), since toString() is sufficient. To have to implement getProtocolName(), I see it as a non-strictly-needed burden to developers that implement ApplicationProtocol. I know of companies that want to use ALPN extensively because they use many different communication protocols internally, so it won't be a rare occasion to implement ApplicationProtocol. > http://cr.openjdk.java.net/~wetmore/8051498/webrev.15/ I gather that the Map parameter can't be solved in other ways, right ? Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal v5
Hi,
On Wed, Sep 23, 2015 at 7:04 AM, Bradford Wetmore
<bradford.wetm...@oracle.com> wrote:
>
>> This new proposal still requires that ciphers are sorted in a way that
>> matches the ApplicationProtocol order.
>> Would be nice if, along with the HTTP/2 blacklist, there is a HTTP/2
>> comparator that sorts ciphers putting the blacklisted ones at the end.
>
> Hm...is the sample code at the end of the initial class description
> insufficient? Adding a comparator seems a little heavyweight in that it
> could require access to the ciphersuite internals and would add a lot of
> complexity for this one known use case. When TLSv1.3 is done, the blacklist
> stuff in HTTP/2 goes away.
Sure, but until TLS 1.3 widely deployed, applications will have to
sort the ciphers to put HTTP/2 ones before the blacklisted ones.
Providing this comparator is as trivial as providing
ApplicationProtocol.HTTP2BLACKLIST, so I thought to mention it.
>> I also don't understand why there are 2 methods for the protocol name
>> ? What value does it bring to have 2 methods for the same thing ?
>
> Please see the IANA registry:
>
> http://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids
>
> for RFC 7301:
>
> http://www.rfc-editor.org/info/rfc7301
>
> getProtocolName() is the IANA/IETF textual representation of the protocol
> name (i.e. "Protocol" column), for example "HTTP/1.1", "SPDY/3", and "HTTP/2
> over TLS". I suppose toString() could be used instead, but thought it might
> eventually output additional ALPN value state. I don't have any concrete
> plans at this point.
>
> getNetworkSequence() is the identification sequence for the protocol (i.e.
> "Identification Sequence" column), and represents the actual byte
> identifiers that will travel the network in an ALPN extension.
>
> 0x68 0x74 0x74 0x70 0x2f 0x31 0x2e 0x31 ("http/1.1")
> 0x73 0x70 0x64 0x79 0x2f 0x33 ("spdy/3")
> 0x68 0x32 ("h2")
>
> When client wants to send the extension over the network, it grabs the
> ApplicationProtocols values from the SSLParameters, then calls
> getNetworkSequence() on each ApplicationProtocol to obtain the actual opaque
> ProtocolName(1..2^8-1) to send. Likewise on the server side, we match the
> incoming active ALPN opaque values with the list of mutually agreeable ALPN
> values. And of course, send back the final selected value.
Sure, but application will have to implement two methods instead of
one, and AFAIU the JDK implementation is never calling
getProtocolName() since it's just a description for humans.
> I've updated the webrev to include an SSLSocket test variant, and added a
> few more comments.
>
> http://cr.openjdk.java.net/~wetmore/8051498/webrev.14/
>
> Hopefully things are more clear now. Thanks for your review/comments.
I see now, thanks for the pointers !
Indulge me a bit more below on the Map passed as parameter to
ApplicationProtocol :)
IIUC, by the time we are executing the code that calls
ApplicationProtocol.match(), the TLS protocol is already chosen and
it's available in SSLSession.
When remains is the transient value of cipher that is being chosen.
Because we already have modified the API to support the application
protocol transient value (by adding
SSLEngine.getHandshakeApplicationProtocol()) to be used by
KeyManagers, I was wondering if we cannot either:
A) add: String SSLEngine.getHandshakeCipherSuite(), to be used by
ApplicationProtocol
or
B) add: Map<String, String> SSLEngine.getHandshakeParameters() or
perhaps: Map<String, String> SSLParameters.getHandshakeParameters().
Thanks !
--
Simone Bordet
http://bordet.blogspot.com
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal v5
Hi,
On Sat, Sep 19, 2015 at 7:15 AM, Bradford Wetmore
<bradford.wetm...@oracle.com> wrote:
> Here is the new webrev:
>
> http://cr.openjdk.java.net/~wetmore/8051498/webrev.12/
>
> Unless there are major objections, we need to get ALPN into JDK very soon to
> make JDK 9. We can still tweak the APIs if something is found later.
>
> Major changes:
>
> 1. ApplicationProtocols interface + H2 + HTTP/1.1 impls
>
> Both client/server call SSLParameters.setApplicationProtocols(List<>) to set
> their preferences.
>
> The matches method can be used for:
>
> . During server handshaking to determine an acceptable ALPN value.
>
> . potentially during client preparation to eliminate known bad
> ciphersuites/protocols combos. If you are going to only request TLSv1.1 and
> earlier, you can disable H2 because it won't match.
>
> In this second case, you may not have a SSLSocket/SSLEngine yet, but you
> still want to do the checks based on ciphersuite/protocol, so the second
> parameters can be null.
>
> 2. ApplicationProtocolSelector gone.
>
> 3. SSLParameters AP_HTTP_1_1/AP_H2 effectively moved to
> ApplicationProtocol. Selection logic added to ApplicationProtocol.
>
> 4. Moved ALPN values from SSLSession to SSLSocket/SSLSEngine. Added a
> "handshaking in progress" variant.
>
> 5. SSLSession.getPeerApplicationProtocols() and
> getHandshakeCipherSuiteList() are gone. These are all handled internally.
This new proposal still requires that ciphers are sorted in a way that
matches the ApplicationProtocol order.
Would be nice if, along with the HTTP/2 blacklist, there is a HTTP/2
comparator that sorts ciphers putting the blacklisted ones at the end.
I don't like the first parameter of ApplicationProtocol.match() to be
a Map<String, String>; I would prefer a full blown class at this
point, that contains all the parameters, for example:
inner class ApplicationProtocol.Info
{
tlsProtocol
cipher
sslEngine
sessionIsResuming
etc.
}
I also don't understand why there are 2 methods for the protocol name
? What value does it bring to have 2 methods for the same thing ?
I would just use:
class ApplicationProtocol
{
public String getName()
}
RFC 7301 hints that the bytes to send over the network are the UTF-8
bytes from that string.
There are still a lot of details missing, such as where is the chosen
ALPN value stored (and how it can be retrieved by the KeyManager, for
example), as well as the webrev not showing any real implementation,
and how exactly the ApplicationProtocol instances are called, etc.
For example, client sends ["h2"], server has ["http/1.1", "h2"]. Will
the instance of ApplicationProtocol corresponding to "http/1.1" be
invoked at all ?
If not, there is a missing step in your algorithm above, where the
implementation intersects the ALPN values from both peers.
Finally, I think this API may be suitable for the server, but not much
for the client, which is equally important.
I don't see how a client application can figure out what protocol has
been chosen by the server, because there is no final notification
about that and the API seems totally geared towards server "matching"
more than client notification. For me this is a blocker.
Can you please provide an example of how a client application would
use this new API to be notified that the server has chosen protocol
"foo" ?
I still remain convinced that - so far - the Jetty API proposal (or
the similar version that was proposed here 2-3 proposals ago) is
superior to the latest ones.
Thanks !
--
Simone Bordet
http://bordet.blogspot.com
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal v4
ake sure that whatever proposal enters JDK 9, session resumption is supported. My preference would go to the previous proposal (akin to Jetty - I know I am biased) where protocol selection was happening in isolation *after* cipher selection. It is much simpler, and has the only drawback of not allowing certificate selection based on application protocol (for which we never had request from the community). This proposal seems less clear to me, a bit of compromise between a full fledged multi-selector API (it is in fact a 2-selector API for cipher and application protocol) and the previous proposal (that was a select-after-the-cipher selector API). If the times are tight, I would go for the simpler approach and leave the full fledged multi-selector API for JDK 10. Thanks ! -- Simone Bordet http://cometd.org http://webtide.com Developer advice, training, services and support from the Jetty & CometD experts.
Re: TLS ALPN Proposal v3
Hi, On Fri, Jul 24, 2015 at 9:38 PM, Jason Greene jason.gre...@redhat.com wrote: The truth is that there is a gap between the current capabilities of TLS stacks and what the specs are trying to achieve. Ultimately the desired semantic the specs are trying to achieve is that every ALPN protocol can have its own TLS requirements. In this case H2 wanted TLS 1.3 behavior before it was released. This is basically social engineering, the newer protocol is the carrot for updating your TLS stack. However there are other potential scenarios, such as protocols which desire to be encrypted but unauthenticated. To truly resolve this gap in a future proof manner, TLS stacks need to support cipher suite selection based on the combination of ALPN protocol and TLS version. Had TLS 1.3 been released before H2, we would not need to choose the cipher suite based on ALPN + TLS version, because any TLS 1.3 cipher would do, and the support for ALPN would be much simpler (probably akin to SNI). The current situation is a temporary glitch produced by the H2 specification, and requires clients and servers to implement hacks to support H2 (sorting ciphers, etc.). Another thing to remember is that clients and servers may implement different ways of selecting the various TLS parameters. While server have to choose among TLS protocol versions, ciphers, application protocols, etc., the client can only react to what the server chose. For a TLS implementation, it may be simpler to implement a client (like browsers do) than a server; I think the server algorithm would work for clients, but not necessarily viceversa. Tradeoff between A) change radically the OpenJDK implementation to support an iterative processing of TLS protocols, extensions (all of them), ciphers, aliases, etc. that would be future proof (if that is even possible) and B) hack in just enough of what is needed to support H2 today (for which we already have working examples) ? Would it be possible to fit the ALPN solution that would have been implemented if TLS 1.3 was out before HTTP/2 to support the current situation ? -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal v3
Hi, On Thu, Jul 9, 2015 at 1:42 AM, Bradford Wetmore bradford.wetm...@oracle.com wrote: SSLParameters is a configuration class which is used to configure SSLSockets/SSLEngines. SSLSession/ExtendedSSLSession is a class which holds negotiated Session values. getReceivedApplicationProtocols() represents the Application Protocol values received from the peer, thus belongs in the SSLSession. I suggest to rename ExtendedSSLSession.getRequestedApplicationProtocols() to ExtendedSSLSession.getPeerApplicationProtocols() The protocols are not really requested, they are more offered, but IIUC the reason to add Requested to this method is to distinguish it from SSLParameters.getApplicationProtocols() which returns the local protocols, and in that spirit I think Peer is better for ExtendedSSLSession. Xuelei/Simone wrote: Per my understanding, application protocol should be negotiated before cipher suite and protocol version negotiated. This is not possible for HTTP/2. Application protocol negotiation MUST happen *after* the TLS protocol and the TLS cipher are negotiated. Yes, that's my understanding as well. Well, to be precise, either the application protocol is negotiated after the cipher (and you need cipher sorting to influence the cipher selection towards the application protocol you would like to choose), or it must happen at the same time - that is, cipher and application protocol must be chosen at the same time - but this implies that the action of choosing that tuple may be invoked multiple times with the current OpenJDK implementation. Note that I don't know if the fact that cipher selection is an iterative process is an OpenJDK implementation detail. If other implementations are not iterative, then perhaps they have a single moment where the tuple is chosen. I support Xuelei in that you should ask confirmation to the HTTP/2 editor. Also, remember that Firefox, Chrome, OpenSSL, nghttp2, etc. are all open source and their code is available to verify the behavior. IIUC, the HTTP/2 blacklist is just strongly recommended (...SHOULD NOT use any of the cipher suites...black list), but not required. Such potential peers must also support such a configuration, but in general, it will not. See section 9.2.2. I think it's still considered compliant to the spec tho. From experience, if a server breaks this SHOULD NOT, it won't work with any browser. We had our share of pain trying to figure out interoperability with browsers for Jetty :) Sure, it's a SHOULD NOT, but it's like a MUST NOT if you want a browser to talk to a Java server (or a Java client to talk to a current HTTP/2 server). Simone wrote two different ways to do selection: 1) ... so that TLS protocol, cipher (possibly the alias too) and application protocol are chosen together, or 2) we separate the TLS protocol and cipher negotiation (and alias) in one step, and we perform application protocol selection afterwards. Rather than completely redo the JSSE selection mechanism with the (version/ciphersuite/ALPN)-tuple idea (which would be a much more involved API and behavior change), I think the more straightforward solution (2) is better. That's what we do in Jetty's ALPN implementation too. Be aware that it rules out some possibility such as those mentioned by Michael from RFC 7301. Also, it is critical to detail how the mechanism work. Example implementations for SSLFunction would be great to have: the typical HTTP/2 case is to select the application protocol based on the TLS protocol and the already negotiated cipher. I have a working test example which shows how the ALPN APIs can be used for HTTP/2 clients and servers. It is a minor configuration tweak to the jdk/test/javax/net/ssl/templates/SSLEngineTemplate.java test that we use as the basis for JSSE SSLEngine testing. Do you have a Java 9 server that can negotiate h2 with current browsers ? http://cr.openjdk.java.net/~wetmore/8051498/webrev.04/test/javax/net/ssl/ALPN/SSLEngineAlpnHttp2.java.html See the configuration in createSSLEngines() around line 265 and 280. http://cr.openjdk.java.net/~wetmore/8051498/webrev.04/test/javax/net/ssl/ALPN/Http2ApplicationSelector.java.html Note the HTTP/2 blacklist and reordering code. The code is not actually working yet (haven't merged API/impl repos yet), but shows how to configure/use this API. Just a reminder that the cipher blacklist is only valid for TLS 1.2. For example, if the negotiated TLS protocol is 1.3, there is no need to look at the ciphers (nor to sort them). Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal v2
Hi, On Fri, Jun 5, 2015 at 2:36 PM, Xuelei Fan xuelei@oracle.com wrote: See more inlines, please. Please help on one question I'm not sure of. Per HTTP/2 specification, Does H2 server allow fallback to HTTP/1.1 if client requests a HTTP/2 connection? I did not find the answer from RFC 7540. Yes. The intended behavior is exactly to fall back to http/1.1 if h2 cannot be negotiated. This is implicitly explained in the ALPN spec, RFC 7301, see http://tools.ietf.org/html/rfc7301#section-3.2. In TLS, if client requests to negotiate TLS v1.2, and server supports TLS 1.2, it is not allowed to fallback to TLS v1.1. If there is not suitable cipher suite to negotiate TLS 1.2, the connection would be terminated immediately. I'm not sure what's the spec for HTTP/2, HTTP/1.1 and HTTP/1.0. HTTP/2 does not behave like TLS in this sense. ALPN is for *application* protocol selection. A client can send: [h2, spdy/3.1, http/1.1] Failing h2, spdy/3.1 is attempted, which is a completely different protocol, with different restrictions, etc. Per RFC 7540, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 is a mandatory cipher suite for H2. No. It's a mandatory cipher for *TLS 1.2 deployments only*. If the client uses TLS 1.4 does not have to have that cipher, and hence there can be an empty intersection of ciphers with the server. That cipher is only mentioned because all the mandatory TLS 1.2 ciphers have been blacklisted by HTTP/2. If TLS 1.3 was specified before HTTP/2, that cipher would not even be mentioned, and the HTTP/2 spec would have referenced only TLS 1.3 as mandatory: no black lists, no special cipher. The TLS and HTTP specifications will evolve independently, and there cannot be a requirement that whenever TLS changes, an update to HTTP/2 must be published. Hope this clarifies. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal v2
Hi, On Fri, Jun 5, 2015 at 4:46 PM, Xuelei Fan xuelei@oracle.com wrote: If H2 is not supported, SPDY/3.1 would be attempted, of SPDY/3.1 is not supported HTTP/1.1 would be attempted. Correct. If H2 is supported in both side, but H2 does not work, it is a H2 problem that need to be addressed in H2 layer. If both client and server have h2 as a potentially supported protocol, but the cipher to use h2 is not valid, then h2 is not supported for that particular connection. At that point, like you said above, spdy/3.1 is attempted, and so on. No application protocol fallback in TLS layer if the application protocol is supported. Your interpretation of supported is not what browser and server implementors mean :) I understand your concerns now. I think we have different understanding of the ALPN protocols. It's a good thing to understand the actually requirements of the industry, I think. Thank you! So where does this leave us know ? By the way, while I have participated in the RFC 7540 discussions, and implemented HTTP/2 in Jetty to be interoperable with a variety of other clients and servers, feel free to ask clarifications to the RFC 7540 and RFC 7301 mailing lists, or even directly to the editors of those RFCs; they are typically open to answer questions, I guess especially so if they come from the OpenJDK team that is implementing those specification. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal v2
Hi, On Thu, Jun 4, 2015 at 6:50 PM, Xuelei Fan xuelei@oracle.com wrote: Hm, I see your point now. But I may not agree with your ALPN MUST happen after protocol/cipher suite negotiation conclusion. I parse this section as, a H2 server must be strong enough(comply to RFC7540), and a H2 client must also be strong enough (comply to RFC7540). Otherwise, both side may terminated the connection, and cannot declare as complying to H2. It is not necessary for an application protocol selector to detect whether a H2 server/client comply to H2 or not. If H2 is requested, it means that the client supports H2. Otherwise, it's a client application bug. Not that simple, see below. If H2 is selected by a selected, it means that the selected server supports H2. Otherwise, it is a selector implementation bug. If something wrong in either client or server, it is expected to terminate the connection immediately, rather than downgrade to a not-strong enough level. From the points above, I think an application protocol selector may not need to know the negotiated protocol version and cipher suite. No. The client may send ciphers that are valid for http/1.1 (but invalid for h2), along with ciphers that are good for h2 (as well as http/1.1 of course), plus the list of protocols it supports. The client has no idea what the server supports. When the server sees that the client supports h2, it MUST pick a cipher that is valid for h2. Alternatively, the ciphers on the server are sorted so that those valid for h2 have higher priority (they are attempted before all the others), so that there is a high chance that a h2 valid cipher is chosen (but no guarantee) before choosing the application protocol. When the application protocol selector callback is invoked, it can only pick h2 IFF the cipher is h2 valid, otherwise it has to fallback to http/1.1. With your reasoning, the client can send [h2_invalid_cipher, h2_valid_cipher], the server may pick h2_invalid_cipher, then the application protocol selector is invoked, which will only look at the protocols, pick h2 since it's supported by client and server, and now you have an invalid connection: the h2 protocol with h2_invalid_cipher. We have been through these issues for months in the RFC 7540 expert group, and the outcome is that protocol selection, for h2, depends on the cipher. We have also been through a number of scenarios where both the client and the server send h2 valid ciphers, but their intersection is empty (this may happen when a very old client talks to a very new server, think TLS 1.2 vs TLS 1.4, or viceversa). Same outcome: to pick h2 you MUST have a h2 valid cipher in common between client and server, so application protocol selection, for h2, depends on the cipher. A bit of warning here: we are designing an API for ALPN, not for HTTP/2. The ALPN API should be flexible enough to implement *at least* HTTP/2, possibly even more complex scenarios (for example alias selection), but IMHO it should not be tied to HTTP/2. Again, I see 2 cases: either the JDK implementation picks the TLS protocol, the cipher and the alias like it does now, and then invokes the callback to pick the application protocol (current Jetty ALPN behavior), or the implementation must be reviewed to perform TLS protocol, cipher, alias and application protocol selection at once, with a callback that will be invoked possibly multiple times until it can find the right tuple to return. The latter would be the optimal solution, the former has certainly working implementations. Hope this clarified. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal v2
Hi, On Thu, Jun 4, 2015 at 5:53 PM, Xuelei Fan xuelei@oracle.com wrote: On 6/4/2015 8:19 PM, Simone Bordet wrote: This is not possible for HTTP/2. Application protocol negotiation MUST happen *after* the TLS protocol and the TLS cipher are negotiated. Why? Is it a spec of HTTP/2? It is a point I don't understand now. Please help with more details. http://tools.ietf.org/html/rfc7540#section-9.2 You can only speak h2 if the cipher is strong enough as defined by RFC 7540. -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal v2
Hi, On Thu, Jun 4, 2015 at 3:08 PM, Michael McMahon michael.x.mcma...@oracle.com wrote: On 04/06/15 13:19, Simone Bordet wrote: Hi, On Wed, Jun 3, 2015 at 8:23 AM, Xuelei Fan xuelei@oracle.com wrote: Per section 4, RFC 7301: ... The application_layer_protocol_negotiation ServerHello extension is intended to be definitive for the connection (until the connection is renegotiated) and is sent in plaintext to permit network elements to provide differentiated service for the connection when the TCP or UDP port number is not definitive for the application-layer protocol to be used in the connection. By placing ownership of protocol selection on the server, ALPN facilitates scenarios in which certificate selection or connection rerouting may be based on the negotiated protocol. Per my understanding, application protocol should be negotiated before cipher suite and protocol version negotiated. This is not possible for HTTP/2. Application protocol negotiation MUST happen *after* the TLS protocol and the TLS cipher are negotiated. What do you mean by after? As far as TLS is concerned, all of this negotiation happens in one negotiation. The client proposes a list of ciphers and application protocols. The server chooses a cipher and application protocol and sends back its choices. Currently, IIUC, the cipher selection is an iterative process where a cipher is attempted until one is negotiated. In this process, there is no looking at the application protocol. Here we're trying to find a solution for ALPN, and either we 1) look the application protocol in this iterative process (and therefore the SSLFunction is invoked multiple times), so that TLS protocol, cipher (possibly the alias too) and application protocol are chosen together, at once (for each iteration); or 2) we separate the TLS protocol and cipher negotiation (and alias) in one step, and we perform application protocol selection afterwards. The latter is how Jetty's ALPN works, and that's what I mean with *after*. For HTTP/2 it won't work to pick the application protocol before the cipher. Either at the same time, or after. That is why I asked to specify how the mechanism worked. The HTTP/2 RFC specifically warns against splitting this negotiation with the example that a client could propose a mandatory TLS 1.2 cipher, but which is black-listed by HTTP/2. If (internally) the server chooses that cipher first, without knowing the application protocol is going to be HTTP/2 then you end up with a non-compliant connection that will probably have to be closed for reason of insufficient security. If the server chooses a blacklisted cipher, and then h2 as protocol, it's a non compliant server. Communication of clients with compliant (and properly configured) servers is guaranteed if the application protocol is chosen after (or at the same time of) the cipher. If I understand you correctly, you are proposing to rework the internal JDK code to perform TLS protocol, cipher, alias and application protocol in one point, which is then possibly iterated multiple times until a satisfactory tuple is selected ? I am fine with this approach too, but I guess the API will be very different from Jetty's ALPN and this current proposal (which is fine as well - does not have to be similar to Jetty's). I don't know enough about the TLS implementation to say how much work it is. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal v2
Hi,
On Wed, Jun 3, 2015 at 2:56 AM, Bradford Wetmore
bradford.wetm...@oracle.com wrote:
Hi,
I have just posted the second iteration of the ALPN proposal which hopefully
addresses all of the comments raised here. It is in javadoc format, but
things can certainly be adjusted:
http://cr.openjdk.java.net/~wetmore/8051498/webrev.00/
Please see the archive [1] for previous design discussion. I will be
writing up usage examples in the next few days.
The significant changes:
ExtendedSSLSession
public ListString getReceivedApplicationProtocols() {
This will allow applications to examine which protocol names were
sent.
This is a constant once the application protocols are received, so it
can be surely retrieved from SSLParameters.
I don't understand why it is replicated here ?
SSLParameters
mentions both ALPN/NPN as possible protocols. I removed the
discussion about server and client since ALPN/NPN essentially
reverse the roles.
mentions appropriate character sets for String-byte[] conversions
such as UTF-8 for ALPN.
The application protocol selector is now a @FunctionalInterface
(i.e. lambda-ready) called SSLFunction. It is to throw an
SSLException if no values are supported, or null if you want to
treat as an unknown extension.
Defined constants for HTTP/1.1 and HTTP/2.
SSLSession
Called out that getHandshakeSession's ciphersuite may vary until
selected.
SSLBase
A new marker interface that SSLEngine/SSLSocket will implement.
This will allow for a single SSLFunction instead of having
SSLFunctionSSLEngine and SSLFunctionSSLSocket. It does require
that the lambda do a instanceof, unless we were to move the common
Socket/Engine APIs into this class.
I'm not sure about this one being a marker interface.
I could understand if it extracted the common functionality of
SSLEngine and SSLSocket, but a marker interface does not really add
much, and perhaps I would prefer it entirely gone.
On the same note, why is SSLFunction generic at all ?
Also, it is critical to detail how the mechanism work.
Will SSLFunction be invoked multiple times, or only once ? When,
exactly, with respect to cipher selection and alias selection ?
Example implementations for SSLFunction would be great to have: the
typical HTTP/2 case is to select the application protocol based on the
TLS protocol and the already negotiated cipher.
Thanks !
--
Simone Bordet
http://bordet.blogspot.com
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal v2
Hi, On Wed, Jun 3, 2015 at 8:23 AM, Xuelei Fan xuelei@oracle.com wrote: Per section 4, RFC 7301: ... The application_layer_protocol_negotiation ServerHello extension is intended to be definitive for the connection (until the connection is renegotiated) and is sent in plaintext to permit network elements to provide differentiated service for the connection when the TCP or UDP port number is not definitive for the application-layer protocol to be used in the connection. By placing ownership of protocol selection on the server, ALPN facilitates scenarios in which certificate selection or connection rerouting may be based on the negotiated protocol. Per my understanding, application protocol should be negotiated before cipher suite and protocol version negotiated. This is not possible for HTTP/2. Application protocol negotiation MUST happen *after* the TLS protocol and the TLS cipher are negotiated. And the connection may be rerouted (even to different machines) for further operation. The requested application protocols list should be the only information for the selection of a suitable application protocol. Not sure what you exactly mean here, but you can't pick the HTTP/2 protocol unless you have the TLS protocol and TLS cipher available. So *only* the list of protocol sent by the client is not enough for HTTP/2, we would need additional contextual information. What a HTTP/2 aware load balancer written in Java that offloads TLS would need to do is to negotiate the TLS protocol, negotiate the TLS cipher, *then* negotiate the application protocol (whether h2 or http/1.1), and with the last information pick a backend server, typically forwarding clear text bytes to the backend. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal
Hi,
On Tue, May 26, 2015 at 8:46 PM, Bradford Wetmore
bradford.wetm...@oracle.com wrote:
I am not sure I follow this. Can you please sketch the steps/algorithm
that will be done in your proposed solution ?
I'm assuming you are talking about 1a and not 1b.
Sure, most of the heavy lifting takes place in ServerHandshaker.
ServerHandshaker.java
=
In the SunJSSE code:
// currently line 330.
private void clientHello(ClientHello mesg) throws IOException {
// Was an ALPNExtension received?
ALPNExtension alpnExt = (ALPNExtension)
mesg.extensions.get(ExtensionTyp.EXT_ALPN);
...
// Currently line 706 in JDK9.
session = new SSLSessionImpl(protocolVersion,
CipherSuite.C_NULL,
getLocalSupportedSignAlgs(),
sslContext.getSecureRandom(),
getHostAddressSE(), getPortSE());
...
session.setALPNNames(alpnExt.getNames());
...
// choose cipher suite and corresponding private key
// This function is at 987 currently.
chooseCipherSuite(mesg);
...
// Only adds an ALPN extension if non-empty.
m1.extensions.add(applicationProtocolSelector.select(...));
...
Above, chooseCipherSuite() iterates through the collection of suites. Inside
that, trySetCipherSuite() attempts to verify that the suite is acceptable to
use. Inside that, setupPrivateKeyAndChain() does the actual KeyManager
calls.
if (conn != null) {
alias = km.chooseServerAlias(algorithm, null, conn);
} else {
alias = km.chooseEngineServerAlias(algorithm, null, engine);
}
Now in the Application's Code:
If you want the KeyManager to take this info into account, you need to
create your own customer KM.
public String chooseEngineServerAlias(String keyType,
Principal[] issuers,
SSLEngine engine) {
ExtendedSSLSession session = engine.getHandshakeSession();
String protocol = session.getProtocol();
String ciphersuite = session.getCipherSuite();
ListString alpns =
session.getRequestedApplicationProtocolNames();
// Some logic for which key to use...
return choose(protocol, ciphersuite, alpns);
}
And then your actual ALPN selector:
public String select(...) throws SSLProtocolException {
ExtendedSSLSession session = engine.getHandshakeSession()
String ciphersuite = session.getCipherSuite();
ListString alpns =
session.getRequestedApplicationProtocolNames();
// Some logic for which key to use...
return choose(protocol, ciphersuite, alpns);
}
Hopefully that is clear.
Let me see if I understand correctly.
In chooseEngineServerAlias() I will have a proposed cipher and the
list of app protos from the client.
The goal would be to choose the alias based on the app proto, as stated by 7301.
However, the app proto is not yet chosen here. If I don't have other
constraints to choose the app proto (e.g. SNI), the usual algorithm
applies: pick the first server app proto that is in common with the
client.
Let's assume this is h2; but looking at the cipher, it's not good, so
we have to pick another app proto, say http/1.1. Now the cipher is
good and we return the alias.
This is similar to what happens now with Jetty's ALPN: the cipher will
be chosen, then the ALPN callback invoked and there, by looking at the
cipher, we know we have to use http/1.1.
Let's assume now I have the constraint to choose h2 (e.g. I have a SNI
of http2.domain.com).
In chooseEngineServerAlias() I will look at SNI, and therefore choose
h2, then look at the proposed cipher, which is not good, so return
null.
Method chooseEngineServerAlias() will be called repeatedly for other
ciphers, until a cipher good for h2 is found, otherwise it's an alert
120.
Are you proposing to call select(...) from chooseEngineServerAlias(),
and give the ALPN callback a semantic that it may be called multiple
times, each time with a different cipher, and give access to the ALPN
callback implementation to SNI to choose the right protocol based on
that ?
Thanks!
--
Simone Bordet
http://bordet.blogspot.com
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal
Hi, On Tue, May 26, 2015 at 2:30 AM, Bradford Wetmore bradford.wetm...@oracle.com wrote: Darn those Chicken/Eggs [1]! Yes, you are correct. The steps for the current server code: 1. The ClientHello is parsed, and the SNI matcher callback is called. It does not return which value was matched in the ServerHello, just whether a SNI name was matched or not: The extension_data field of this extension SHALL be empty. 2. Begin generating the ServerHello, choose the Protocol Version. 3. Iterate through the list of client's ciphersuites and call the Server's KeyManager (KM) callback until the KM returns key material we can use. A return string selects the proposed ciphersuite. So we currently don't know the selected ciphersuite until the KM has been called (possibly multiple times). If we choose the ALPN before the ciphersuite, the ciphersuite selection may end up being inappropriate (HTTP/2 blacklist). If we choose the ciphersuite first, then the ALPN value wasn't used to drive the certificate selection. Two suggestions in preferred order below. In each of these cases, unfortunately there is currently no indication of the proposed Ciphersuite, so we need to modify the behavior of getHandshakeSession().getCipherSuite() to fill in the proposed CipherSuite before the call to the KM. This seems ok with the current wording, but we'd need to make that explicit. This value will change for each ciphersuite/KM choice attempt. Each suggestion below is followed by our previously proposed ALPN callback to make the actual ALPN value selection: 1a. Add a parallel method to ExtendedSSLSession: public ListString getRequestedApplicationProtocolNames(); along with the previously proposed selected name: public String getApplicationProtocol() (I'll be changing these names. I'm open to suggestions). When the KM is called, the TLS protocol (e.g. TLSv1.2) has already been selected. Both of the major selection parameters (protocol/proposed ciphersuite) are now available, and applications have access to the ordered ALPN list to see what the client's requested values were. -or- 1b. Keep API as is, and make two callbacks. This first is an advisory value, the TLS protocol version and proposed ciphersuite will be available in getHandshakeSession(). The second callback sets the final value that will be sent. I think 1.a is my preference. I am not sure I follow this. Can you please sketch the steps/algorithm that will be done in your proposed solution ? -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal
Hi, On Mon, May 25, 2015 at 12:08 PM, Michael McMahon michael.x.mcma...@oracle.com wrote: Hi Brad, A couple of initial comments/questions. 1) Certificate selection is one feature envisaged by ALPN. ie a client or a server ought to be able to choose a different certificate depending on the application name that gets negotiated. Is that possible with this API? Interesting. I can definitely see choosing the ALPN protocol based on the SNI name sent by the client. For example, a server able to speak http/1.1 and h2 receiving a request for http1.domain.com wants to force http/1.1. This would be possible, IIUC, using sslEngine.getHandshakeSession().getRequestedServerNames() in the ApplicationProtocolSelector implementation. I see less common choosing the certificate given the application protocol, but I understand it's mentioned in RFC 7301. ALPN definitely needs the cipher to be negotiated to support HTTP/2, so I hope it's not a chicken-egg problem. -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal
Hi, On Mon, May 25, 2015 at 3:57 PM, Michael McMahon michael.x.mcma...@oracle.com wrote: Perhaps, though it seems there are specific ALPNs for HTTP/1.1 (http/1.1) and for HTTP/2 (h2). So, I think you would use ALPN itself to do that negotiation. An incoming TLS connection without the ALPN extension is just assumed to be HTTP/1.1 Sure, but I can see a client looping over a list of domain names (e.g. to check whether the sites support HTTP/2) and therefore not having any knowledge of whether it should or not add the ALPN extension. This client will always add it, but the server must force http/1.1 for http1.domain.com. There aren't very many different applications envisaged yet. There are a couple of NAT related protocols registered. But, actually having thought about it again, client certificate selection happens in the X509ExtendedKeyManager class and that implementation could presumably call sslEngine.getHandshakeSession().getApplicationProtocol() and select the client cert using that information. It doesn't do that now obviously but I think it could in future if necessary. Sure, unless the protocol is not available because ALPN code has not run yet. For what I understand, currently cipher selection and certificate selection happen at the same time, please correct me if I am wrong. If that's the case, then we have a chicken-egg problem: you can't call ALPN code before the cipher is selected, but you need ALPN to select the certificate. If those two steps can be split, then ALPN code could be put in between and all is solved. -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal
Hi,
On Sat, May 23, 2015 at 3:13 AM, Bradford Wetmore
bradford.wetm...@oracle.com wrote:
Thanks for the thorough reviews and comments, I really appreciate it and
always learn something. FunctionalInterface (@since 1.8) is something I
haven't really explored yet, so off to the books.
Just to be clear, this is what I am proposing:
class SSLParameters
{
...
ListString getApplicationProtocols();
void setApplicationProtocols(ListString protocols);
void setApplicationProtocolSelector(ApplicationProtocolSelector selector);
ApplicationProtocolSelector getApplicationProtocolSelector();
}
@FunctionalInterface
interface ApplicationProtocolSelector
{
String select(ListString protocols) throws SSLException;
}
In this way, there is no need for a SSLBase to converge SSLSocket and
SSLEngine, the ApplicationProtocolSelector can be specified as a
lambda expression on SSLParameters, and everything is much simpler.
--
Simone Bordet
http://bordet.blogspot.com
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal
from including SPDY/*, since they are on their way out now, and NAT/STUN since there hasn't been any call for it so far. I agree that only http/1.1 and h2 deserve a constant. -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal
Hi,
On Fri, May 22, 2015 at 8:54 PM, Sean Mullan sean.mul...@oracle.com wrote:
There's actually a bunch of methods in common, so one possibility is to
create a new interface containing the common methods (say SSLBase for now
for lack of a better name). Then you could change SSLEngine and SSLSocket to
implement SSLBase (which should be a compatible change), and then have a
single method on ApplicationProtocolSelector. Question is if there are
enough common methods to do what you want? I don't know enough about ALPN to
answer that question.
There are 2 things that conflate here: one is to provide the mechanism
to support ALPN, and the second is to make sure that it is possible to
support HTTP/2.
ALPN per se, for example, does not need any information about
SSLEngine or TLS protocol versions or ciphers.
It just needs the protocol strings, so the ApplicationProtocolSelector
would just need select(ListString).
For HTTP/2, however, the choice of the protocol will depend on the TLS
version and the cipher, so we will need a way to obtain that
information.
Typically, in the code where you pass the ApplicationProtocolSelector
you have the SSLEngine available, e.g.:
sslEngine.getSSLParameters().setApplicationProtocolSelector(new
ApplicationProtocolSelector()
{
public String select(ListString clientProtocols)
{
// Here sslEngine will be in lexical scope
}
});
In this way, ApplicationProtocolSelector won't depend on SSLEngine or
SSLSocket, and can be reduced to a FunctionalInterface.
--
Simone Bordet
http://bordet.blogspot.com
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless. Victoria Livschitz
Re: TLS ALPN Proposal
Hi, On Fri, May 22, 2015 at 9:14 PM, Bernd Eckenfels e...@zusammenkunft.net wrote: I would suggest to make this encoded in latin1 instead. This is supposed to be a 8bit clean encoding (and will be compatible to all ASCII only strings). It is still ugly and needs to be documanted cleanly that the string you get back might not be a string at all. RFC 7301 hints that protocol string could be UTF-8 encoded: http://tools.ietf.org/html/rfc7301#section-6 -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: JEP 244: TLS Application-Layer Protocol Negotiation Extension
Hi, On Mon, Apr 13, 2015 at 6:22 PM, David M. Lloyd david.ll...@redhat.com wrote: Do you know of a Java TLS implementation that has APIs like this already? I am also interested in this for the ability to implement authentication mechanisms (GSSAPI and SASL) that rely on channel binding, so I would like to see such an API materialize as well. I posted a while back such APIs from 3rd party JSSE implementations: http://mail.openjdk.java.net/pipermail/security-dev/2014-August/011014.html (at the end). The problem that has been raised is that if you offer a generic TLS extensions API, then the extension may have a semantic that it's not implemented. Imagine this TLS extensions API already existed, to add extensions to SSLEngine. Now, ALPN comes along as a new TLS extension. An application could create their own ALPNExtension subclass (extending a standard one provided by the TLS extensions API), and add it to the ClientHello. But there is no code in the JDK that calls the application, asking (on the server) to select one of the protocols, for then send back the chosen protocol to the client. This could be solved by a callback API at the moment the ClientHello is received by the server (and the ServerHello by the client), so the application can examine the ALPN protocols. The NPN extension was doing something even more complicated, creating an additional TLS message that needed to be sent at the right time. It may be that a TLS extensions API (to add/remove/query TLS extensions) *and* a callback API to analyze hello messages when they are received is enough to cover a lot of cases, perhaps even all currently existing ones. I asked for feedback some time ago about the status of the ALPN implementation; would be great if the security team could update the current status. -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: JEP Review Request: TLS Application-Layer Protocol Negotiation Extension
Hi, On Mon, Feb 2, 2015 at 8:27 PM, Bradford Wetmore bradford.wetm...@oracle.com wrote: The draft JEP “TLS Application-Layer Protocol Negotiation Extension” is now available for community review: https://bugs.openjdk.java.net/browse/JDK-8051498 This JEP is to add support for the Application Layer Protocol Negotiation (ALPN) TLS Hello extension [1] in JSSE. ALPN provides a mechanism for declaring the application protocols that are supported over a TLS connection. We need this functionality to make JDK 9, so this JEP needs to get into the JEP pipeline soon. Community review is a precursor in the process before it can move to Submitted. For now, there is a simple API proposed (similar to JDK 8 SNI), but I'm parsing the discussions that took place on security-dev in August[2], September[3], and November 2014[4], and the current API is likely not flexible enough. I have taken a quick look at the proposed API as reported here: https://bugs.openjdk.java.net/browse/JDK-8062848. I don't think it is enough, since what needs to be provided by the API is the ability to run application code. That is the API must be something like: sslParameters.setALPNSelector(protocols - protocols.get(0)); That is, it should take a lambda of the form: public String select(ListString protocols); Also, the lambda would be different for clients and servers. See my November message for further details: http://mail.openjdk.java.net/pipermail/security-dev/2014-November/011399.html Somehow orthogonal to this, the HTTP2 working group has finally stabilized the requirements for negotiating HTTP2. For a server that receives a list of ALPN protocols, and that wants to negotiate h2, it has not only check that both itself and the client support the h2 protocol, but also that the cipher requirements for HTTP2 are satisfied. Note that this requirement only exist for negotiating HTTP2. While I agree that the major use case for ALPN is HTTP2, ALPN is nevertheless a generic mechanism to negotiate the application protocol over TLS, so the HTTP2 cipher selection should not be hardcoded. So perhaps the JDK should also provide a utility class, say HTTP2ALPNServerSelector, which implements the before mentioned lambda adding the specific HTTP2 cipher logic, although this may be out of scope for this ALPN work. I mentioned it as a reminder that the API cannot be too closed, as the protocol selection may need to access contextual information such as the TSL protocol version, the negotiated cipher, possibly the server name indication, etc. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: A fully fledged TLS Extensions API ?
Hi, On Mon, Nov 10, 2014 at 2:07 PM, Florian Weimer fwei...@redhat.com wrote: On 11/07/2014 02:06 PM, Simone Bordet wrote: This email is about the idea to introduce in JDK 9 a fully fledged TLS Extensions API. Adding ALPN [0] support to JDK 9 requires, differently from other TLS extensions, to provide application code that will be run in the context of the TLS implementation, rather than just values such as booleans or strings. That's going to be interesting if you need to support non-blocking operation for use with SSLEngine. In the case where you have to load the configuration from a place where it may take a lot of time, you can easily change the ALPN API to something like: For the client: void selected(String protocol, CompletableFutureVoid callback) For the server: void select(ListString protocols, CompletableFutureString callback) and when you're done call callback.complete() or callback.completeExceptionally(). This would mean that the TLS implementation has to wait for those callbacks to be completed before proceeding. On one side, having application code to complete the callbacks is, in our experience, error prone (i.e. applications forget it) On the other side, applications that make use of this API are typically on the implementer side rather than on the developer side (here I mean people that implement a server, rather than a developer of a webapp), so I guess a callback, albeit more complex, may give implementers more room for different implementations. IMHO this chance can be lifted to provide a full TLS Extensions API. I don't think this is possible because TLS extensions can alter the TLS handshake, result in additional messages being exchanged, and generally alter the protocol in unforeseeable ways. On top of that, the concrete TLS implementation is not fixed, it can be swapped out, so tying the extension API to the existing OpenJDK internals will not work for everyone. Can you expand what you mean here ? If a TLS extensions API is provided by JDK 9, would it not be implemented by providers like they do with the other APIs they have to implement to be a compliant provider ? It's a little more effort for providers, but with a public API should be implementable by any provider and an application would be portable across providers. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: A fully fledged TLS Extensions API ?
Hi, On Mon, Nov 10, 2014 at 10:36 PM, Bernd Eckenfels e...@zusammenkunft.net wrote: Hello, the question is, what is a extension API actually helping with: the JSSE implementation is slow to adopt to typical advances in TLS browser wars. While this is not so bad as nobody implements a Web Browser in Java anyway, it does make it harder for server vendors to offer a full stack (without native libraries or reverse proxies). Well, JEP 110 (http://openjdk.java.net/jeps/110) is about a HTTP client that supports h2. I think it's not too far-fetched to expect jfx/swing widgets, JAX-RS clients, etc. to support h2 in a close future, so I would not rule out h2 support for clients. ALPN would be a prerequisite for h2, since unlikely servers will implement h2 in the clear. Having an API to handle extensions could help with this a bit, however a lot of the extensions cannot be bolted on, they alter the handshake sequence, introduce new messages or alter interpretatrion of existing enums. So while ALPN can be handled pretty easily, it would be harder for things like eliptic_curves or supported_signatures or SRP (or Poly authenticator) - or TLS_FALLBACK_SCSV (or simple things like rate limiting the renegotiations). For example for the NPN extension to work, it requires collaboration with the TLS implementation to send a new TLS message at the right moment. A JDK 9 client application would be able to add the NPN extension via the API, but then the JDK would not be able to send the new message at the right time because NPN won't be implemented in JDK 9. There are several ways to handle these cases, but I agree that they may expose an overspecification of the API similar to that of the collections API: throw UnsupportedOperationException if you can't handle it (e.g. Iterator.remove()). Whether there is more advantage for applications to have the API and risk an UOE, or not having the API and be stuck, I don't know (and it's the reason for this email, to discuss it :) A declaration of the impact of an extension on the TLS protocol, akin to Spliterator.characteristics() may help. Some extensions have no impact and may be freely added, others require the TLS implementation cooperation. A couple more callbacks for [Client|Server]Hello received, for applications to interact with the extensions for SSLSocket (similar to HandshakeCompletedListener) would be needed too. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
A fully fledged TLS Extensions API ?
Hi, I was writing the email about the ALPN API proposal when I realized that it would have been better to split the arguments in different emails. This email is about the idea to introduce in JDK 9 a fully fledged TLS Extensions API. Adding ALPN [0] support to JDK 9 requires, differently from other TLS extensions, to provide application code that will be run in the context of the TLS implementation, rather than just values such as booleans or strings. IMHO this chance can be lifted to provide a full TLS Extensions API. Alternative providers such as IAIK offer a private API such as [1]: SSLSocket.getActiveExtensions() SSLSocket.getPeerExtensions() Similarly, BouncyCastle offers [2]: DefaultTlsClient.getClientExtensions() In the JDK there is a class named SSLParameters that contains some of the configuration values for TLS. Some of those are duplicated in SSLSocket and SSLEngine (e.g. wantClientAuth, needClientAuth, etc), with some temporal dependency (call this before the other, if I have to trust the comments of SSLSocketImpl.setHost()). Eventually all values get forwarded to the handshaker, but from the API point of view it's not very clear what API one should call (the one on SSLEngine or the one on SSLParameters), nor where the ALPN setup should be done. The idea would then be to introduce a fully fledged TLS extensions API to handle all the TLS extensions related things, such as renegotiation, SNI, elliptic curves, NPN, ALPN, session tickets etc. Both applications and the JDK implementation would be able to leverage this new API. Note that the bulk of this API already exists in sun.security.ssl. My question is really whether JDK 9 could take in consideration such TLS Extension API be exposed publicly for applications to use it. See the other emails about ALPN and the interaction between ALPN and HTTP 2 for further examples of where this TLS Extension API would come handy. Thanks ! [0] http://tools.ietf.org/html/rfc7301 [1] http://javadoc.iaik.tugraz.at/isasilk/current/iaik/security/ssl/SSLSocket.html [2] https://www.bouncycastle.org/docs/docs1.5on/org/bouncycastle/crypto/tls/DefaultTlsClient.html -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
ALPN HTTP2
Hi, below you can find few considerations about the interaction between HTTP 2.0 (from now onwards just h2) and ALPN. The h2 group at the IETF is having a big debate about section 9.2.2 of draft-14 [0]. This section basically put a very strong requirement on the ciphers that can be used to talk h2. However, clients are entitled to offer weak ciphers and strong ciphers, but only the strong ones can be used to talk h2. With this, there is a dependency between the ALPN protocol chosen and cipher selection; note that cipher selection should happen before the ALPN server lambda invocation (as I proposed in the other email). The algorihtm proposed by the h2 editors would be that h2 could only be chosen by the ALPN server lambda IFF: !cipher.isStream() !cipher.isBlock() cipher.supportsForwardSecrecy() There is a big resistance among the h2 experts to have this algorithm defined in the h2 spec, for many reasons. However, the h2 editors are very resistant to changes (I have to say, backed by the TLS editor). The fact that almost all TLS implementations out there don't have at least one (or all) of the methods above (so basically the algorithm above is not implementable) is not seen as a good enough reason to remove section 9.2.2. Furthermore, there is not direct correlation, in the TLS handshake, between ciphers and protocols. Speaking of the server side that I know better, the case is that I take Jetty and run it in the shiny just released JDK 9. The TLS implementation of JDK 9 adds new ciphers, one of which is named XYZ. Jetty receives a TLS connection request, with cipher XYZ and ALPN listing h2. The JDK Jetty runs on has XYZ, but Jetty has no idea whether this cipher is good for h2 or not. Jetty may pick XYZ and h2, thus succeeding at both cipher negotiation (both client and server can use that cipher) and at protocol negotiation (both client and server can speak h2), but when the client receives the ServerHello, the client may say oh, no, XYZ is not good for h2, I offered it for HTTP/1.1, but now it's too late and the only possible solution is to close the connection, and reopen another one excluding h2 from the protocols sent to the server (or excluding the cipher XYZ - but how the client knows if a different cipher will succeed ?). Ironically, 9.2.2 may hamper h2 adoption. Hopefully the h2 group will come to a better decision about section 9.2.2, perhaps without entangling protocol negotiation with cipher negotiation, but we don't know yet. In any case, I would suggest that the OpenJDK security-dev leaders consider to add the capabilities to query a cipher for its properties to OpenJDK. There already exist a javax.crypto.Cipher class that perhaps may be enhanced to expose those methods, along with a way to obtain the selected Cipher instance from SSLSocket/SSLEngine. On the same theme, it would not be too far fetched to think that a server may want to provide h2.domain.com over h2, and h1.domain.com over HTTP/1.1 for the same certificate *.domain.com. For this to happen, the server ALPN lambda would need to extract the SNI extension name, compare it with some virtual host configuration, and then take a decision about the protocol to speak. Again, a full fledged TLS Extension API would simplify this: String sni = SSLEngine.getRemoteExtensions().find(SNIExtension.class).getName(); SNIMatcher won't be able, for example, to match regexp'ed virtual hosts. In conclusion I'd like to point out the fact that a ALPN API into JDK 9 is a mandatory requisite to implement h2, but may not be sufficient, since additional methods on Cipher and other TLS classes may be needed. Thanks ! [0] http://tools.ietf.org/html/draft-ietf-httpbis-http2-14 -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS extensions API, ALPN and HTTP 2.0
Hi, Another update on the subject. It seems that the HTTP 2.0 working group has relaxed, after a excruciating battle, requirements on ciphers and their entanglement with the ALPN protocols advertised. This results in many less requirements to have ALPN and HTTP 2.0 work together. I am reviewing the JDK API and other open source API and will come up soon with a proposal for the ALPN API. Sorry for the long delay about this, but the outcome of the HTTP 2.0 working group was important to decide how much was needed in the JDK for an implementation to be able to implement HTTP 2.0. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS extensions API, ALPN and HTTP 2.0
Hi, On Sat, Sep 27, 2014 at 4:23 AM, Xuelei Fan xuelei@oracle.com wrote: I used to think it is better to have SSLExplorer as a public utility but not a part of JSSE, because the extract is not involved in TLS transactions. I am not sure I understand this. Can you expand ? A number of TLS Extensions are designed to be consumed by applications: SNI, ALPN, NPN, renegotiation, etc. Eventually these features crop up in the API (e.g. SSLParameters.setServerNames()), or even applications need to have control on whether these extensions have to be added or not (e.g. ALPN) in a dynamic way that cannot be expressed via system properties. So I would rather prefer a full fledged extensions API exposed by the JDK (also to cope with future extensions). Please let me know if the SSLExplorer cannot meet your requirements. Definitely not :) It has too many assumptions on how many bytes have been read, it does not work for async reads that may read only part of TLS frames, it parses again the TLS bytes, etc. Sure we can do all that in the application in order to fullfill SSLExplorer requirements, but I really hope that is not the way to solve this problem. Every application out there would have its own tweaked version of SSLExplorer, tweaks may introduce vulnerabilities, etc. The whole point is to have a standard API to rely on. Internally OpenJDK already does the parsing of the SNI extension, so I see no point in forcing application to do it again: just expose what's already been read. I think we should go the extra mile and just expose in the right way what's already in the JDK: there already is a ServerNameExtension class, hopefully there will be NPNExtension, ALPNExtension classes, etc, all part of the same API, which is available to applications. As I said before, it seems that ALPN is only one piece required to support HTTP/2. The HTTP/2 spec editors are being adamant on cipher requirements, so for example an application would need to know the type of cipher as defined by RFC 5246 section 6.2.3: something like cipher.isStream(), cipher.isBlock(), etc, along with whether the cipher supports forward secrecy (currently ephemeral key exchange). I will try to come up with the list of necessary things for HTTP/2, but I would appreciate any thought on introducing a full fledged TLS extensions API available to applications to the JDK. I'd like to know in advance if this is not possible at all. Thanks ! -- Simone Bordet http://cometd.org http://webtide.com http://intalio.com Developer advice, training, services and support from the Jetty CometD experts. Intalio, the modern way to build business applications.
Re: TLS extensions API, ALPN and HTTP 2.0
Hi, On Fri, Sep 26, 2014 at 8:03 PM, Sean Mullan sean.mul...@oracle.com wrote: On 09/17/2014 01:18 PM, Simone Bordet wrote: For the server to differentiate between those 2 connections he would need the SNI information, which I don't think it's currently available in JDK 8, right ? No. It is. We added support for SNI in JDK 8. See: http://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html#SNIExtension I understand one cannot extract the string with the SNI name into the application, you can only match for certificates via SNIMatcher; and that is the reason for SSLExplorer - to extract the SNI names. Am I missing something ? For example, how can I negotiate h2 via ALPN only for certain domains ? ListString allowedDomains = ... // provided by some server configuration SNIServerName sniName = ... // what here ? if (allowedDomains.contains(sniName)) doALPN(); Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS extensions API, ALPN and HTTP 2.0
Hi, On Tue, Sep 2, 2014 at 11:11 AM, Vincent Ryan vincent.x.r...@oracle.com wrote: Your OCA is still being processed. When that has completed your name will be listed at: http://www.oracle.com/technetwork/community/oca-486395.html#b Until then, we can discuss these TLS/HTTP issues but we cannot include your APIs or source code. Just an update on this... While ALPN should offer mechanism independent from the protocol advertised or the cipher used in the TLS connection, it seems that the HTTP/2.0 spec has put some constraint that link the protocol advertised to the ciphers negotiated (http://tools.ietf.org/html/draft-ietf-httpbis-http2-14#section-9.2.2). Currently it seems that this HTTP/2 requirement is difficult, if not impossible, to implement in the JDK. I am following the HTTP/2 expert group to see if this issue is resolved, and working on the Jetty code to implement this feature. The idea being to wait a bit to define the ALPN APIs until this issue is resolved, to see if the resolution requires changes in the ALPN APIs. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS extensions API, ALPN and HTTP 2.0
Hi, On Wed, Sep 17, 2014 at 12:57 PM, Michael McMahon michael.x.mcma...@oracle.com wrote: Hi Simone, I'm interested to understand why you think this Http 2 requirement is difficult or impossible to implement in the JDK currently. I thought, cipher suite selection would be independent of the ALPN mechanism. Indeed they should be independent, but section 9.2.2 poses constraint and links the h2 protocol with a particular subset of ciphers. The client can offer 2 ciphers, cipher1 that is valid per 9.2.2 and will work for h2, and cipher2 that is not valid for h2, but will work for http/1.1. At this point, the server has to have a knowledge about what ciphers are good for what protocol. Writing this code in a future-proof way is difficult. Even preferring client ciphers, which is only possible in JDK 8, is not enough, as we may choose the wrong one if the client sends them without a particular order. For example two browsers may send one (cipher1, cipher2), and another (cipher2, cipher1). Say that in future cipher FOO is discovered and found to be suitable for h2. An old h2 client is not aware of FOO, and won't accept it as valid for h2. But when the old h2 client is deployed in JDK 9, which does offer FOO, a server can select FOO because the client offered it, negotiate h2, but the client will close the connection because it won't recognize FOO as a valid cipher for h2. My statement about the difficulty of implementation went a bit beyond about strictly ALPN, but also taking in account server implementations of h2. We are trying to have section 9.2.2 modified so that it drops this linking and leaves everything to TLS (for example by just stating that a particular version (or greater) of TLS is mandatory for h2, without mentioning ciphers). For reference the discussion happens at http://lists.w3.org/Archives/Public/ietf-http-wg/2014JulSep/2112.html. -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS extensions API, ALPN and HTTP 2.0
See also: https://github.com/http2/http2-spec/issues/612 On Wed, Sep 17, 2014 at 3:17 PM, Simone Bordet simone.bor...@gmail.com wrote: Hi, On Wed, Sep 17, 2014 at 12:57 PM, Michael McMahon michael.x.mcma...@oracle.com wrote: Hi Simone, I'm interested to understand why you think this Http 2 requirement is difficult or impossible to implement in the JDK currently. I thought, cipher suite selection would be independent of the ALPN mechanism. Indeed they should be independent, but section 9.2.2 poses constraint and links the h2 protocol with a particular subset of ciphers. The client can offer 2 ciphers, cipher1 that is valid per 9.2.2 and will work for h2, and cipher2 that is not valid for h2, but will work for http/1.1. At this point, the server has to have a knowledge about what ciphers are good for what protocol. Writing this code in a future-proof way is difficult. Even preferring client ciphers, which is only possible in JDK 8, is not enough, as we may choose the wrong one if the client sends them without a particular order. For example two browsers may send one (cipher1, cipher2), and another (cipher2, cipher1). Say that in future cipher FOO is discovered and found to be suitable for h2. An old h2 client is not aware of FOO, and won't accept it as valid for h2. But when the old h2 client is deployed in JDK 9, which does offer FOO, a server can select FOO because the client offered it, negotiate h2, but the client will close the connection because it won't recognize FOO as a valid cipher for h2. My statement about the difficulty of implementation went a bit beyond about strictly ALPN, but also taking in account server implementations of h2. We are trying to have section 9.2.2 modified so that it drops this linking and leaves everything to TLS (for example by just stating that a particular version (or greater) of TLS is mandatory for h2, without mentioning ciphers). For reference the discussion happens at http://lists.w3.org/Archives/Public/ietf-http-wg/2014JulSep/2112.html. -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS extensions API, ALPN and HTTP 2.0
Hi, On Wed, Sep 17, 2014 at 4:11 PM, Michael McMahon michael.x.mcma...@oracle.com wrote: Okay, I see the point you are making. It's more a question of whether the constraints themselves are appropriate. And convince the HTTP/2 editors :( I've another question. In the work you've done so far, did you allow for the possibility of separate certificates to be selected per ALPN instance? I'm guessing that if multiple applications are to be supported over one TCP port (client or server) then different certificates might be needed for each application. Or is that not a reasonable assumption? If I understand you correctly, you are saying that you want a connection with SNI=foo.domain.com to *not* trigger ALPN, but a connection with SNI=bar.domain.com to trigger ALPN ? We don't support this right now in Jetty, but the current ALPN API probably does: at the moment of selecting the protocol, the application can retrieve the SNI and decide what protocol to select. See for example: http://git.eclipse.org/c/jetty/org.eclipse.jetty.project.git/tree/jetty-alpn/jetty-alpn-server/src/main/java/org/eclipse/jetty/alpn/server/ALPNServerConnection.java#n49 There you can call getSSLEngine(), which will be able to return a number of information about the handshake (such as the cipher chosen). Makes sense ? -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: [undertow-dev] NoSuchMethod in 9ea-b30 getRawHostnameSE
Hi, On Wed, Sep 17, 2014 at 5:04 PM, Bernd Eckenfels e...@zusammenkunft.net wrote: Hello, thanks Stuart, I should have known to look for this. I mentioned before, that this is a quite important thing for JSSE to support (or actually the SSL API). But I did not expect that the JEtty workaround is already used. In Jetty we have a mechanism to start the server with ALPN only if we have the right version of ALPN for the JDK that is running the server. For example, Jetty would refuse to start with a message in JDK 9 with ALPN enabled. A bit harsh, but at least you know what's going on :) But yes, having ALPN support in JDK 9 will definitely be great. -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS extensions API, ALPN and HTTP 2.0
Hi, On Wed, Aug 20, 2014 at 1:03 PM, Simone Bordet simone.bor...@gmail.com wrote: On Tue, Aug 19, 2014 at 4:11 PM, Vincent Ryan vincent.x.r...@oracle.com wrote: Finally, please confirm that you have already signed the OCA [1] I have not yet, it's running through legals ATM. I'll notify when this is done. I emailed the signed OCA. Does it need to be processed before we can talk about the design, or ... ? Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: TLS extensions API, ALPN and HTTP 2.0
Hi, On Tue, Aug 19, 2014 at 4:11 PM, Vincent Ryan vincent.x.r...@oracle.com wrote: Hello Simone, Thanks for re-raising this issue. There are indeed plans to add support for ALPN in JDK 9 but they’re still at an early stage so the timing is good. We would be interested in examining your implementation to explore how it fits into the current TLS APIs. Removing the dependency on a boot class path setting is something we should address in JDK 9. Would you be interested in co-operating on defining a new public API in JDK 9 for ALPN usage in TLS? Yes. Typically, we use the SSLParameters class to support setting the proposed TLS parameters and use the SSLSession class to support getting the negotiated TLS parameters. We could easily adopt a similar model for ALPN. I'll have a look, although ALPN is a connection property rather than a session property. Finally, please confirm that you have already signed the OCA [1] I have not yet, it's running through legals ATM. I'll notify when this is done. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
TLS extensions API, ALPN and HTTP 2.0
Hi, I was suggested to restart the discussion about the topic on this list. I would like to reboot a discussion around an improved TLS extensions API in order to support ALPN (see http://tools.ietf.org/html/rfc7301), which is the mechanism required by HTTP 2.0 to negotiate the new version of the HTTP protocol. I sent a previous message hoping that such work would have been included in JDK 8, but it was too late, see http://mail.openjdk.java.net/pipermail/jdk8-dev/2013-March/002197.html. I think this needs to be addressed so that a future version of the Servlet specification can be implemented without requiring the hacks described below. Under the Jetty project, we have implemented ALPN as a set of patches to OpenJDK classes, producing a jar that must be prepended to the boot classpath in order for ALPN to work, see https://www.eclipse.org/jetty/documentation/current/alpn-chapter.html. The downside of this is that for every JDK release the ALPN jar may need to be rebuilt incorporating JDK changes. The patches are available at https://github.com/jetty-project/jetty-alpn, while the API provided to applications is here: http://git.eclipse.org/c/jetty/org.eclipse.jetty.alpn.git/ While this solution works, it would be great to have a clear API in the JDK that would allow to add the required TLS extension without requiring patched classes and boot classpath jars. This would allow applications to manage easily and in one standardized way TLS extensions like SNI, ALPN or NPN, chiper suite negotation, renegotiation, etc. Most of the classes are already in the sun.ssl.* package, so the effort would be to clean them up and move them to a standard package. A) Is there any plan to add a generic TLS extensions API to JDK 9 ? B) Is there a plan, perhaps in concert with the Servlet EG, to prepare to support ALPN in order to support HTTP 2.0 ? C) What would be the process to start the effort to add a TLS extensions API to the JDK ? Start a new JEP ? Note that proprietary TLS Extentions API have been provided by other JSSE implementers such as IAIK, see http://javadoc.iaik.tugraz.at/isasilk/current/iaik/security/ssl/ExtensionList.html and its usage in http://javadoc.iaik.tugraz.at/isasilk/current/iaik/security/ssl/SSLSocket.html. We are happy to contribute to this effort, but we seek guidance from the experts in this list. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Re: Next Protocol Negotiation TLS Extension
Hi, On Fri, Mar 22, 2013 at 1:08 AM, Brad Wetmore bradford.wetm...@oracle.com wrote: Hi Simone, I haven't looked at the proposal yet, but just from a scheduling point of view, unfortunately we're finishing up the implementation of the last of the planned features for JDK 8, so getting this into 8 is likely not possible. Likely not possible means totally impossible or if you do this and that, then it's possible ? :) Do you know the status of JEP 114 ? Will it be shipped in JDK 8 ? We have an open bug for this (JDK-8007785) and it's on the radar for JDK 9. Ok, if impossible I will eventually pop up again when JDK 8 is released and JDK 9 work started. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Next Protocol Negotiation TLS Extension
Hi all, [cross-posted to jdk8-dev and security-dev] I am a member of the Jetty servlet container (http://eclipse.org/jetty) team and the implementor of the Next Protocol Negotiation (NPN) TLS Extension used by Jetty to support the SPDY protocol (API at http://git.eclipse.org/c/jetty/org.eclipse.jetty.npn.git/ and implementation at https://github.com/jetty-project/jetty-npn). The SPDY protocol has been chosen as the basis for HTTP 2.0. I would like to ask for suggestions for what would be the best way to have NPN support in OpenJDK 8 rather than via the Jetty NPN implementation. Currently, the Jetty implementation is kind of hacky in that it is the smallest possible hack (in a positive meaning) to make NPN work in OpenJDK. It modifies 5 sun.security.ssl.* classes and introduces 5 new classes. These modifies classes must be put in the bootclasspath. The API of public classes like SSLEngine is not modified; instead the current implementation relies on a static class that maps SSLEngine (or SSLSocket) with application code that is invoked at the right time during the TLS handshake when NPN data is detected. Currently, the Jetty project maintains the NPN implementation locked with OpenJDK releases: every time the sun.security.ssl.* classes are modified, we pull in the changes from OpenJDK, re-patch these classes with NPN support and make a new release of the NPN jar. The NPN TLS extension requires an API exposed to applications (usually web servers, but they are applications for the Java runtime). In this sense, JEP 114 (http://openjdk.java.net/jeps/114, SNI TLS extension) is similar: I am guessing it also has to expose an API to applications. It seems to me that both NPN and SNI would require a standard way to access TLS extensions at the proper time during the TLS handshake. In light of this, it would be great if NPN could be piggybacked on JEP 114, exposing a standard TLS extensions API provided by OpenJDK that application can use to plug in their code for NPN and/or SNI. Now, I understand that designing a TLS extensions API is not as simple as including the current Jetty NPN implementation in OpenJDK, but I would rather see a generic solution in OpenJDK rather than a hacky solution like current Jetty NPN's included in OpenJDK. A private TLS extensions API already exists in the sun.security.ssl.* classes, but it's mostly package private and of course under sun.* packages. So perhaps the work to be done is not a from-scratch effort. I would like to get a discussion started on how NPN can be supported in OpenJDK 8. Ideally, my best plan would be: * NPN included in JEP 114. * JEP 114 designing a standard TLS extensions API that can serve for both NPN and SNI (and, generically, others TLS extensions) * JEP 114 shipped in OpenJDK 8. We're happy to keep Jetty NPN up-to-date for OpenJDK 7 releases, but we will really like to see NPN in OpenJDK 8. We are open to comply with any legal papers that needs to be in place for this contribution. I will be more than happy to provide detailed information about the implementation of Jetty NPN and have it discussed (or even reviewed) by security experts. Thanks ! -- Simone Bordet http://bordet.blogspot.com --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
