IBM Connections Mail and Ephemeral Diffie-Hellman key size error – part 2

I wrote about the effects using DHE ciphers can have depending on the size of the SSL certificate used by iNotes when IBM Connections Mail is in play in IBM Connections Mail and Ephemeral Diffie-Hellman key size error

In this blog I suggested the work around was to use the following notes.ini setting.

SSL_DH_KEYSIZE=2048

Our Domino admins weren’t too keen on lowering the key size so I had to look into a way of forcing the server to use a different cipher instead of one of the DHE ciphers.

This is the output from Domino when the DHE cipher is in play.

[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> Client requested RSA_WITH_AES_128_CBC_SHA (0x002F)
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> Best common cipherspec 0x002F (so far)
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> Best common non-EC cipherspec 0x002F (so far)
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> Client requested RSA_WITH_AES_256_CBC_SHA (0x0035)
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> Best common cipherspec 0x0035 (so far)
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> Best common non-EC cipherspec 0x0035 (so far)
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> Client requested DHE_RSA_WITH_AES_128_CBC_SHA (0x0033)
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> Client requested DHE_RSA_WITH_AES_256_CBC_SHA (0x0039)
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> Best common cipherspec 0x0039 (so far)
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> Best common non-EC cipherspec 0x0039 (so far)
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> Client requested DHE_DSS_WITH_AES_128_CBC_SHA (0x0032)
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> Client requested DHE_DSS_WITH_AES_256_CBC_SHA (0x0038)
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> Client requested RSA_WITH_3DES_EDE_CBC_SHA (0x000A)
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> Client requested DHE_RSA_WITH_3DES_EDE_CBC_SHA (0x0016)
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> Client requested Unknown Cipher (0x0013)
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> Client requested TLS_EMPTY_RENEGOTIATION_INFO_SCSV (0x00FF)
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> TLS_EMPTY_RENEGOTIATION_INFO_SCSV found
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> Extensions found in this message
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> Processing TLS signature algorithms extension
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> Client supports hash mask 0x007E; server cert chain has mask 0x0030
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> hash/alg in certchain  fSupHasAlg:0000
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessClientHello> We selected cipher DHE_RSA_WITH_AES_256_CBC_SHA (0x0039)
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLProcessHandshakeMessage Exit> Message: ClientHello (1) State: HandshakeServerIdle (3) Key Exchange: 9 Cipher: DHE_RSA_WITH_AES_256_CBC_SHA (0x0039)
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLAdvanceHandshake Enter> Processed: ClientHello (1) State: HandshakeServerIdle (3)
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLAdvanceHandshake client_hello> SGC FLAG: 0   Count = 2
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLAdvanceHandshake calling SSLPrepareAndQueueMessage> SSLEncodeServerHello
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLEncodeServerHello> Sending empty renegotiation_info (0xff01) extension
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLAdvanceHandshake calling SSLPrepareAndQueueMessage> SSLEncodeCertificate
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLEncodeCertificate> Generating a certificate message with 3 certs
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLAdvanceHandshake calling SSLPrepareAndQueueMessage> SSLEncodeServerKeyExchange
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLEncodeDHKeyParams> Server RSA key size 4096 bits
[00403:00011-2285692672] 07/15/2016 11:07:54.94 AM SSLEncodeDHKeyParams> Using a DH key size of 4096 bits
[00403:00011-2285692672] 07/15/2016 11:07:55.01 AM SSLEncodeRSAServerKeyExchange> Signing ServerKeyExchange using RSAWithSHA256
[00403:00011-2285692672] 07/15/2016 11:07:55.04 AM SSLAdvanceHandshake calling SSLPrepareAndQueueMessage> SSLEncodeServerHelloDone
[00403:00011-2285692672] 07/15/2016 11:07:55.04 AM SSLAdvanceHandshake Exit> State HandshakeClientKeyExchange (11)
[00403:00011-2285692672] 07/15/2016 11:07:55.04 AM SSL_Handshake> After handshake state = HandshakeClientKeyExchange (11); Status = -5000
[00403:00011-2285692672] 07/15/2016 11:07:55.04 AM int_MapSSLError> Mapping SSL error -5000 to 4176 [SSLHandshakeNoDone]
[00403:00011-2285692672] 07/15/2016 11:07:55.06 AM SSLProcessProtocolMessage> Record Content: Alert (21)
[00403:00011-2285692672] 07/15/2016 11:07:55.06 AM SSLProcessAlert> Got an alert of 0x50 (internal_error) level 0x2 (fatal)
[00403:00011-2285692672] 07/15/2016 11:07:55.06 AM SSL_Handshake> After handshake2 state HandshakeClientKeyExchange (11)
[00403:00011-2285692672] 07/15/2016 11:07:55.06 AM SSL_Handshake> SSL Error: -6994
[00403:00011-2285692672] 07/15/2016 11:07:55.06 AM int_MapSSLError> Mapping SSL error -6994 to 4171 [SSLFatalAlert]

The idea was to remove the DHE_RSA_WITH_AES_256_CBC_SHA (0x0039) from the list of supported ciphers.

You can do this by dictating all the ciphers Domino uses using the SSLCipherSpec notes.ini setting.

I stopped Domino and added to the notes.ini the following and then started Domino.

SSLCipherSpec=C030009FC02F009EC028006BC014C0270067C013009D009C003D0035003C02F000A

You can see in the string 0039 is not listed. This means that Domino will not use DHE_RSA_WITH_AES_256_CBC_SHA and another cipher will be negotiated.

On restart you can see that the cipher RSA_WITH_AES_256_CBC_SHA is now selected and that is being used which works.

[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLInitContext> Ignoring invalid SSLCipherSpec value F0
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLInitContext> User is forcing 0xFFF3800 cipher spec bitmask for 15 ciphers
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSL_TRUSTPOLICY>  bits for signature hashes: 0030
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM int_MapSSLError> Mapping SSL error 0 to 0 [SSLNoErr]
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSL_Handshake> outgoing ->protocolVersion: 0303
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessProtocolMessage> Record Content: Handshake (22)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessHandshakeMessage Enter> Message: ClientHello (1) State: HandshakeServerIdle (3) Key Exchange: 0 Cipher: Unknown Cipher (0x0000)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessHandshakeMessage client_hello> SGC FLAG: 0 CTX state = 3 SGCCount = 0
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> clientVersion: 0303
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> SSL/TLS protocol clientVersion 0x0303, serverVersion 0x0303
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> 10 ciphers requested by client
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> Client requested RSA_WITH_AES_128_CBC_SHA (0x002F)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> Client requested RSA_WITH_AES_256_CBC_SHA (0x0035)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> Best common cipherspec 0x0035 (so far)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> Best common non-EC cipherspec 0x0035 (so far)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> Client requested DHE_RSA_WITH_AES_128_CBC_SHA (0x0033)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> Client requested DHE_RSA_WITH_AES_256_CBC_SHA (0x0039)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> Client requested DHE_DSS_WITH_AES_128_CBC_SHA (0x0032)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> Client requested DHE_DSS_WITH_AES_256_CBC_SHA (0x0038)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> Client requested RSA_WITH_3DES_EDE_CBC_SHA (0x000A)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> Client requested DHE_RSA_WITH_3DES_EDE_CBC_SHA (0x0016)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> Client requested Unknown Cipher (0x0013)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> Client requested TLS_EMPTY_RENEGOTIATION_INFO_SCSV (0x00FF)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> TLS_EMPTY_RENEGOTIATION_INFO_SCSV found
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> Extensions found in this message
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> Processing TLS signature algorithms extension
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> Client supports hash mask 0x007E; server cert chain has mask 0x0030
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> hash/alg in certchain  fSupHasAlg:0000
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessClientHello> We selected cipher RSA_WITH_AES_256_CBC_SHA (0x0035)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessHandshakeMessage Exit> Message: ClientHello (1) State: HandshakeServerIdle (3) Key Exchange: 1 Cipher: RSA_WITH_AES_256_CBC_SHA (0x0035)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLAdvanceHandshake Enter> Processed: ClientHello (1) State: HandshakeServerIdle (3)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLAdvanceHandshake client_hello> SGC FLAG: 0   Count = 2
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLAdvanceHandshake client_hello> Using resumed SSL/TLS Session
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLAdvanceHandshake calling SSLPrepareAndQueueMessage> SSLEncodeServerHello
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLEncodeServerHello> Sending empty renegotiation_info (0xff01) extension
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLAdvanceHandshake calling SSLPrepareAndQueueMessage> SSLEncodeChangeCipherSpec
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLAdvanceHandshake calling SSLPrepareAndQueueMessage> SSLEncodeFinishedMessage
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLCalculateTLS12FinishedMessage Enter> senderID: server finished, PRF using SHA256
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLAdvanceHandshake Exit> State HandshakeChangeCipherSpec (13)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSL_Handshake> After handshake state = HandshakeChangeCipherSpec (13); Status = -5000
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM int_MapSSLError> Mapping SSL error -5000 to 4176 [SSLHandshakeNoDone]
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessProtocolMessage> Record Content: Change cipher spec (20)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSL_Handshake> After handshake2 state HandshakeFinished (14)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM int_MapSSLError> Mapping SSL error -5000 to 4176 [SSLHandshakeNoDone]
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessProtocolMessage> Record Content: Handshake (22)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessHandshakeMessage Enter> Message: Finished (20) State: HandshakeFinished (14) Key Exchange: 1 Cipher: RSA_WITH_AES_256_CBC_SHA (0x0035)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLCalculateTLS12FinishedMessage Enter> senderID: client finished, PRF using SHA256
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLProcessHandshakeMessage Exit> Message: Finished (20) State: HandshakeFinished (14) Key Exchange: 1 Cipher: RSA_WITH_AES_256_CBC_SHA (0x0035)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLAdvanceHandshake Enter> Processed: Finished (20) State: HandshakeFinished (14)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSLAdvanceHandshake Exit> State HandshakeServerIdle (3)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSL_Handshake> After handshake2 state HandshakeServerIdle (3)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSL_Handshake> Using resumed SSL/TLS session
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSL_Handshake> Protocol Version = TLS1.2 (0x303)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSL_Handshake> Cipher = RSA_WITH_AES_256_CBC_SHA (0x0035)
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSL_Handshake> KeySize = 256 bits
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSL_Handshake> Server RSA key size = 4096 bits
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM SSL_Handshake> TLS/SSL Handshake completed successfully
[06035:00011-2986616576] 07/15/2016 12:30:35.85 PM int_MapSSLError> Mapping SSL error 0 to 0 [SSLNoErr]

The string below includes all the ECDHE ciphers which is detailed in https://www-10.lotus.com/ldd/dominowiki.nsf/dx/TLS_Cipher_Configuration but not the DHE cipher that was tripping me up.

SSLCipherSpec=C030009FC02F009EC028006BC014C0270067C013009D009C003D0035003C02F000A

It work’s now and I have tested it with all major browsers. I’m happy and so are the Domino guys too 🙂