Wireshark-dev: Re: [Wireshark-dev] Reduce memory consumption by re-reading data from file for r
From: Guy Harris <guy@xxxxxxxxxxxx>
Date: Wed, 3 Nov 2010 11:50:58 -0700
On Nov 3, 2010, at 11:25 AM, Jakub Zawadzki wrote:

> Temp files are also kind of memory, so you won't decrease memory by using them :>

That depends on what type of memory you're talking about.

If you're talking about the address space of Wireshark - which, at least for 32-bit versions of Wireshark, could be a significant constraint - temp files don't increase consumption of address space.

If you're talking about the use of physical memory, overconsuming that won't cause Wireshark to fail to allocate memory unless you also overconsume swap space - it'll cause thrashing, but not malloc failures.  Pages from temp files will just get evicted if necessary - and if they're dirty, they'll get pushed back to the temp file, not to swap space, so they won't *themselves* consume swap space.

> Btw. it's like reinventing swap, so if you want to use your disk as memory,
> it's enough to create some big file and do mkswap & swapon.
> (Well maybe on 32-bit systems it's not so easy...)

Yes - adding swap space (which happens automatically in a fashion similar to the one you describe in the UN*X running on the machine on which I'm typing this) won't make a 32-bit address space bigger than 2^32 bytes.

> I like Jeff Moriss idea about mmap()ed files.

...which consume address space, so, again, if the problem is running out of address space, rather than running out of main memory+backing store for anonymous pages, that won't help.