Wireshark-dev: Re: [Wireshark-dev] New packet list - out of memory?
Date Index · Thread Index · Other Months · All Mailing Lists
Date Prev · Date Next · Thread Prev · Thread Next
From: Jeff Morriss <jeff.morriss.ws@xxxxxxxxx>
Date: Tue, 06 Oct 2009 12:21:07 -0400
Anders Broman wrote:

Hi,
Experimenting loading the file there is 163 calls to VirtualAlloc()
~1.630 GB memory used
Changing #define EMEM_ALLOCS_PER_CHUNK (EMEM_PACKET_CHUNK_SIZE / 512) 
to:
#define EMEM_ALLOCS_PER_CHUNK (EMEM_PACKET_CHUNK_SIZE / 64)

Gives only 26 VirtualAlloc() so it seems like we waist a huge amount of
memory...
15 VirtualAlloc() with 32 bytes.

Is there a reason not to change to:
#define EMEM_ALLOCS_PER_CHUNK (EMEM_PACKET_CHUNK_SIZE / 32) ?
Well, the disadvantage is that the overhead associated with the chunk headers grows (when using canaries). For example, with the current /512 number sizeof(emem_canary_t) is 184,328 which means 1.47% of the total allocation for the chunk goes to overhead. Changing that to /64 means the canary header grows to 1,474,568 and total overhead grows to 14.1%. Those numbers are not counting the actual canaries at the end of each allocation.
BUT, the waste that goes into those headers is vastly overshadowed by the waste in unused buffer space... 


Loading is actually 2s slower with se_alloc for col_text...
Are we better off using glibs memory functions?
Another question: do we really need to use se_ allocations here? Do we need the canaries or are we just using se_ allocations because they get cleaned up for us "automatically?" 

Should we have a new allocator without the canaries?