Wireshark-dev: Re: [Wireshark-dev] I have a 5.8GB capture and it is taking an enormous amount o
On Dec 10, 2018, at 5:16 PM, Richard Sharpe <realrichardsharpe@xxxxxxxxx> wrote:
> It has taken 20 minutes or more to load this monster (looks like about
> 5M packets)
"It has taken 20 minutes or more..." as in "it took 20 minutes or more to load it" or as in "it has, so far, taken 20 minutes or more to load it, and it's still loading it"?
> and I dread making searches through it.
>
> The Wireshark process is only consuming about 1.7GB of memory at the
> moment and 17% of CPU on my laptop with NVMe and oodles of memory and
> a 4-core Xeon.
"At the moment" as in "after it finished loading, it's only consuming..." or "while it's loading, it's consuming..."?
The dissection process is currently not multi-threaded, so extra cores don't help. Making it multi-threaded is possible, but it would involve a lot of work to parallelize an in-principle-sequential process, including finding dependencies (e.g., a case where dissecting packet N involves looking up something in a table that might be modified by the dissection of packet M, for M < N, in which case the dissection of packet N has to block waiting for packet M to be dissected).
Just out of curiosity, what happens if you read the file with TShark without the -2 flag, sending the output to the null device, and then with the -2 flag, again sending the output to the null device? What's the difference in the memory consumption of TShark in those cases, and how does it differ from the memory consumption of Wireshark for the same file? (That could indicate how much memory is used by the frame table - which shouldn't be constructed by TShark without -2, but should be constructed by TShark with -2 and by Wireshark - and how much is used by the GUI display widget showing the packet list, which will only be in Wireshark.)
