Wireshark · Wireshark-dev: Re: [Wireshark-dev] Memory consumption in tshark

[Evan Huus <eapache@xxxxxxxxx>]

On 2013-08-28, at 3:24 AM, Dario Lombardo <dario.lombardo.ml@xxxxxxxxx> wrote:

On Tue, Aug 27, 2013 at 10:38 PM, Evan Huus <eapache@xxxxxxxxx> wrote:

We already discard a great deal of state in (single-pass) tshark that we keep around in Wireshark (or two-pass tshark). We do need to keep some, though. It's only a bug if we're keeping more than we actually need, and that's not determinable from the information we have here. Dario, if you could get us a memory profile of tshark in this situation (through valgrind's massif tool, for example) that would help us debug further.

For sure. But I'd need exactly the commands to run and what I should give you back.

It's dependant on platform and setup, but I'll assume a from-source build on Linux. In theory all you have to do is prefix your normal command with "libtool --mode=execute valgrind --tool=massif" and then the usual ./tshark etc.

Valgrind takes a bunch more memory though, so you'll almost certainly want to use editcap to split the capture, and then run this on just a subset.

It will produce an output file massif.out.PID which you can pass to the ms_print command for human-readable output. That output would be useful to us.

I dislike the idea of two-pass by default for exactly this reason: people expect tshark to be relatively state-less. This is already not the case, but it's a lot worse in two-pass mode. It might even make sense to add a --state-less flag to tshark that disables all options which require state. I don't know how feasible that would be however.

Evan

FYI, 10G file is a giant DNS capture. Maybe the state kept in the queries (for conversations creation) triggers the memory consumption.

That's almost certainly what it is - the question is which state is the culprit of using so much memory :)

___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev@xxxxxxxxxxxxx>
Archives:    http://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
            mailto:wireshark-dev-request@xxxxxxxxxxxxx?subject=unsubscribe