Wireshark · Wireshark-dev: Re: [Wireshark-dev] Support for TLS1.2 decryption using derived keys

[webpentest <webpentest@xxxxxxxxx>]

Hello Peter, thanks for your answer. I have truncated some of the quoting in order to avoid inflating the size of the message.

On 30.04.2020 12:58, Peter Wu wrote:

> This
>       would be the ideal approach as access to the master secret
>       provides
> full functionality. Apart from the links shared before, I found these
> references in my notes which might help with such an implementation:
> https://reverseengineering.stackexchange.com/questions/211/decryping-tls-packets-between-windows-8-apps-and-azure
> https://reverseengineering.stackexchange.com/questions/2681/is-it-possible-to-decrypt-an-ssl-connection-short-of-bruteforcing
>
> Yes it will require some privileges, but someone with more expertise on
> Windows might know their ways.
>

Thanks for these additional resources! Just to clarify: extraction of secrets from lsass.exe is absolutely possible; I was able to do that successfully for on current windows 10.

The problem here is the increasing layers of protection from accessing lsass memory, such as Protected Process Light and others. Disabling these protections is not trivial and in some cases might even be impossible. Thus the main objective of my research was to explore ways to not touch the memory of lsass.exe.

> > 2. The derived keys, however, are available to the actual process that
> > uses schannel. Notably, I can extract the client and server write keys
> > and IV's, as well as the client random (see section 6.1 of RFC 5246,
> > page 17).
> >       
> In my notes I also found this reference describing extraction of the
> "write keys", I suppose that is what you are using?
> https://social.technet.microsoft.com/Forums/en-US/4041d78a-21bb-44fd-9a96-6579ea8129d1/obtaining-sslkeylogfilelike-data-from-edge-et-al-schannel-clients
>

Thanks for this link, I've never encountered it before. It also has some additional motivation on why it might be preferable to lift the secrets from the process itself, not lsass.

My approach is a little bit different to the one suggested, it does not envolve calls to ExportSecurityContext. Using a debugger I monitor local RPC communication between mstsc (in my example case) and lsass.exe. These messages contain both the client random and the write keys/ivs I am interested in.

> Just to be clear about what "derived keys" are. You are referring to the
> "write keys". The "derived keys" in that [3] link for TLS 1.3 are
> comparable to the "master secret" in TLS 1.2. In TLS 1.2 and before the
> master secret is sufficient to derive all those write keys that may be
> used in the TLS session. In TLS 1.3, there are many more different
> secrets for handshake encryption, application data encryption, and so
> on: https://tools.ietf.org/html/rfc8446#page-93
>
> The same write keys are present for TLS 1.3 though (see RFC 8446,
> Section 7.3).
>

Thanks for the clarification. So, if I understood correctly, tls 1.3 derives a set of intermediate secrets from master secret and then derives the keys for encryption from those secrets. And the keylog for tls1.3 contains the intermediate secrets, not the (write) keys themselves.

I must admit that I haven't yet looked closely into TLS1.3 -  I'm not yet ready to comment in detail on this and on  your following suggestions regarding TLS1.3 and QUIC.

> The idea could certainly be considered. There are limitations, but I
> suppose that these would be preferable over having no decryption
> capability at all. I think that this idea could be extended for TLS 1.3
> as well.
>
> Microsoft added an experimental implementation of TLS 1.3 in Windows 10,
> version 1909:
> https://docs.microsoft.com/en-us/windows/whats-new/whats-new-windows-10-version-1909
>     
> ............
>
>     typedef struct _SEC_TRAFFIC_SECRETS {
>         wchar_t SymmetricAlgId[SZ_ALG_MAX_SIZE];     // Negotiated symmetric key algorithm. e.g. BCRYPT_AES_ALGORITHM.
>         wchar_t ChainingMode[SZ_ALG_MAX_SIZE];       // Negotiated symmetric key algorithm chaining mode. e.g. BCRYPT_CHAIN_MODE_GCM or BCRYPT_CHAIN_MODE_CCM.
>         wchar_t HashAlgId[SZ_ALG_MAX_SIZE];          // Negotiated hash algorithm. e.g. BCRYPT_SHA256_ALGORITHM or BCRYPT_SHA384_ALGORITHM.
>         unsigned short KeySize;                      // Size in bytes of the symmetric key to derive from this traffic secret.
>         unsigned short IvSize;                       // Size in bytes of the IV to derive from this traffic secret.
>         unsigned short MsgSequenceStart;             // Offset of the first byte of the TLS message sequence to be protected with a key derived from TrafficSecret. Zero to indicate the first byte of the buffer.
>         unsigned short MsgSequenceEnd;               // Offset of the last byte of the TLS message sequence to be protected with a key derived from TrafficSecret. Zero if the secret is for the encryption of application data or decryption of incoming records.
>         SEC_TRAFFIC_SECRET_TYPE TrafficSecretType;   // Type of traffic secret from the TRAFFIC_SECRET_TYPE enumeration.
>         unsigned short TrafficSecretSize;            // Size in bytes of the traffic secret.
>         unsigned char  TrafficSecret[ANYSIZE_ARRAY]; // Traffic secret of type TrafficSecretType, TrafficSecretSize bytes long, used to derive write key and IV for message protection.
>     } SEC_TRAFFIC_SECRETS, *PSEC_TRAFFIC_SECRETS;
>
> The traffic secret is described in
> https://tools.ietf.org/html/rfc8446#section-7.3, it is the same
> handshake and application data secret that Wireshark already supports!
> So for TLS 1.3 support, hopefully you can use this mechanism without
> requiring further modifications to Wireshark.
>

So, just to make sure that I understand you correctly, what you are implying is that for TLS1.3 connections I might be able to extract not the write keys, but the derived secrets, which would be preferable for wireshark as it would enable using already-existing sslkeylog-parsing functionality.

I agree, but I fear that these structures you mention above are, alike the tls1.2 master key, may exist only in the memory of lsass, but not the application that uses schannel API - because of the same key isolation feature.

> Maybe something similar is
> available for TLS 1.2 and earlier support. If not, then we could add
> support for write key/iv, but only as a last resort.
>

Seems to me that we have two separate (although, related) tasks at hand:

1. A generic way to export schannel key material in SSLKEYLOG-like format using elevated privilege and lsass.exe debugging / memory. Preferably - the data that wireshark supports already - master secret for tls <= 1.2 and the intermediate traffic secrets for tls 1.3

2. A generic way to export schannel key material for a particular process _without_ using elevated privilege. Here I'm almost certain that for TLS <= 1.2 exporting master secret is not possible, and I strongly suspect that for TLS1.3 exporting traffic secrets is not possible as well (but this I haven't yet verified). So if wireshark is to support keys extracted this way, we need a patch that will allow using write key/iv from a keylog - both for tls <= 1.2 and 1.3

As far as I understand you comments, currently solving the first task is more desirable. But the tasks aren't really mutually exclusive, i.e. if the first one is solved, the second still remains an interesting target at least in some circumstances.

> I made some suggestions above on the high-level approach. If you have a
> concrete patch I'd suggest submitting it for review per
> https://www.wireshark.org/docs/wsdg_html_chunked/ChSrcContribute.html
>     
    Thanks for taking time to review my hacks! Submitting a proper patch
    for review is my goal.
> Some quick comments:
>
> - The dummy CLIENT_RANDOM is not needed, the state tracking has to be
>   updated instead. Right now it expects either a premaster secret, a
>   master secret, or an encrypted premaster secret + RSA private key.
>   You'd have to extend this to support the fourth case.
>

Yes, the dummy CLIENT_RANDOM is ugly and I used it just to get to the working POC faster.

So, i would need to extend ssl_session->state with an additional flag for cases where only the write keys are available (say, SSL_WRITE_KEYS) and account for this situation in ssl_generate_keyring_material.

I'll also have to extend SslDecryptSession to accomodate for write keys and ivs, because currently I pass these values using ssl_master_key_map_t parameter that I added to ssl_generate_keyring_material prototype, and changing the prototype is probably ill-advised?

> - Do not allow the QUIC_ prefix in the regex. It is deprecated and will
>   be removed later.
>     
    Thanks for clarification.
> - The write key and write IV are usually available at the same time.
>   What about defining a single format such as
>   "WRITE_KEY <crand> <client key> <client iv> <server key> <server iv>"?
>   This would reduce the number of hashtables required as well. For
>   non-AEAD ciphers there is also a client/server MAC key for verifying
>   the decryption result. In theory these could also be added to ensure
>   full functionality. Not sure how important it is.
>     
    Again, separate lines for separate values were just easier to
    implement quickly. Considering the use of one hashtable - I'll have
    to place a there a struct of four StringInfo* (instead of just
    StringInfo* as it currently is for all hashtables that are involved
    in keylogfiles parsing). This in turn will require special handling
    in tls_keylog_process_lines (currently the code expected all
    hashtables to contain StringInfo*). Other option would be to somehow
    serialize the keys and ivs into one StringInfo, but given their
    variable length this is also a little bit ugly. Are there any other
    options that I'm missing here.
>