Wireshark-dev: Re: [Wireshark-dev] Wireshark memory handling
From: Guy Harris <guy@xxxxxxxxxxxx>
Date: Fri, 9 Oct 2009 18:48:29 -0700

On Oct 8, 2009, at 1:15 PM, Erlend Hamberg wrote:

The "problem", as we have understood it, and as we have seen it to be, is that Wireshark keeps its internal representation (from reading a capture file) in
memory.

The data Wireshark currently keeps in its address space that could grow in size as the capture file grows are:

the frame_data structure (epan/frame_data.h) - one structure instance per packet;

the text for some or all of the columns in all of the rows of the packet list (all, in current releases of Wireshark; some, in the development branch);

	various per-packet private data attached to some frames by dissectors;

	various per-dissector private data structures;

	the results of reassembly.

The data from the frames in the capture file are not kept in Wireshark's address space - they are read in as necessary, into a small number of buffers (one for the main window, and one for each packet window opened). *HOWEVER*, if data from a frame is reassembled into a higher-level multiple-frame packet, the result of the reassembly is, as noted, kept in Wireshark's address space.

I write "problem" in quotes, because in most use cases I guess that
this is not a problem at all,

People complain about it enough that, while in *most* cases it might not be a problem, we frequently get mail from people who have to split up capture files to read them - I'd call it enough of a problem that we should work on it (ideally, by reducing the amount of address space required by the aforementioned data items).

So, on OS X (and possibly other modern OS-es), as long as you have available harddisk space, a process will not run out of memory, ever? (A process can
have address space of ~18 exabytes on 64-bit OS X. [1])

On modern OSes that support 64-bit userland, and that can dynamically grow the swap space, running on a machine with enough disk space, I would expect that a process running 64-bit would not run out of address space unless there are some in-kernel data structures that scale with address space size and the kernel's maximum address space size limits the size of those data structures enough to stop address space growth before you hit another limit. 32-bit programs aren't going to be so lucky.