Wireshark-bugs: [Wireshark-bugs] [Bug 5821] New: Reduce per-packet memory requirements
https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=5821
Summary: Reduce per-packet memory requirements
Product: Wireshark
Version: SVN
Platform: All
OS/Version: All
Status: NEW
Severity: Normal
Priority: Low
Component: Wireshark
AssignedTo: wireshark-bugs@xxxxxxxxxxxxx
ReportedBy: guy@xxxxxxxxxxxx
Build Information:
Trunk
--
I read in a huge capture file (a 265.5 megabyte gzipped file - that's 265.5MB
after gzipping) with a 64-bit version of Wireshark on OS X 10.6.7, and Activity
Monitor reported that Wireshark's virtual address space was over 1GB.
I ran it with the MallocStackLogging environment variable set to 1, and ran
malloc_history on it; it produced 11 MB of output. Some of the top items are
207570 calls for 421782240 bytes: thread_7fff70cf6ca0 |start | main | cf_read |
read_packet | g_slice_alloc | slab_allocator_alloc_chunk | posix_memalign |
malloc_zone_memalign
319526 calls for 158484896 bytes: thread_7fff70cf6ca0 |start | main | cf_read |
new_packet_list_thaw | gtk_tree_view_set_model | gtk_tree_view_build_tree |
_gtk_rbtree_insert_after | _gtk_rbnode_new | g_slice_alloc |
slab_allocator_alloc_chunk | posix_memalign | malloc_zone_memalign
20757 calls for 42178224 bytes: thread_7fff70cf6ca0 |start | main | cf_read |
read_packet | g_slice_alloc | slab_allocator_alloc_chunk | posix_memalign |
malloc_zone_memalign
1 calls for 33554432 bytes: thread_7fff70cf6ca0 |start | main | cf_read |
read_packet | add_packet_to_packet_list | new_packet_list_append |
packet_list_append_record | g_ptr_array_add | g_ptr_array_maybe_expand |
g_realloc | realloc | malloc_zone_realloc
1 calls for 33554432 bytes: thread_7fff70cf6ca0 |start | main | cf_read |
read_packet | add_packet_to_packet_list | new_packet_list_append |
packet_list_append_record | g_ptr_array_add | g_ptr_array_maybe_expand |
g_realloc | realloc | malloc_zone_realloc
686 calls for 22500800 bytes: thread_7fff70cf6ca0 |start | main | cf_read |
wtap_read | libpcap_read | libpcap_read_rec_data | file_read | fill_out_buffer
| g_malloc | malloc | malloc_zone_malloc
38871 calls for 19280016 bytes: thread_7fff70cf6ca0 |start | main | cf_read |
read_packet | add_packet_to_packet_list | new_packet_list_append |
packet_list_append_record | g_signal_emit | g_signal_emit_valist |
boxed_proxy_collect_value | gtk_tree_iter_copy | g_slice_alloc |
slab_allocator_alloc_chunk | posix_memalign | malloc_zone_memalign
9686 calls for 4804256 bytes: thread_7fff70cf6ca0 |start | main | cf_read |
new_packet_list_thaw | gtk_tree_view_set_model | gtk_tree_view_build_tree |
_gtk_rbtree_insert_after | _gtk_rbnode_new | g_slice_alloc |
slab_allocator_alloc_chunk | posix_memalign | malloc_zone_memalign
1 calls for 1920808 bytes: thread_7fff70cf6ca0 |start | main | cf_read |
read_packet | add_packet_to_packet_list | dissect_packet | call_dissector |
call_dissector_work | call_dissector_through_handle | dissect_frame |
dissector_try_uint_new | call_dissector_work | call_dissector_through_handle |
dissect_eth_common | ethertype | dissector_try_uint_new | call_dissector_work |
call_dissector_through_handle | dissect_ip | dissector_try_uint_new |
call_dissector_work | call_dissector_through_handle | dissect |
decode_udp_ports | dissector_try_uint_new | call_dissector_work |
call_dissector_through_handle | dissect_rx | call_dissector |
call_dissector_work | call_dissector_through_handle | dissect_afs |
g_hash_table_insert_internal | g_hash_table_resize | g_malloc0 | calloc |
malloc_zone_calloc
46 calls for 1508800 bytes: thread_7fff70cf6ca0 |start | main | cf_read |
wtap_read | libpcap_read | libpcap_read_header | file_read | fill_out_buffer |
g_malloc | malloc | malloc_zone_malloc
1 calls for 1146508 bytes: thread_7fff70cf6ca0 |start | main | geoip_db_init |
GeoIP_open | malloc | malloc_zone_malloc
That top item is for the frame_data structure; that structure is 144 bytes
long, and, with about 280-290 million packets in the capture, that adds up to
about 421MB. Some of the items we might be able to re-fetch from the capture
file rather than store in the frame_data structure; some of the time stamps, if
we don't or can't re-fetch them, could be calculated from other time stamps.
If we explicitly bunch the structures together, rather than letting a slab
allocator do so, we could have next and prev pointers for the chunks, rather
than for each structure. (Pointers are 8 bytes, and nstime_t's are 16 bytes,
in most LP64 environments.)
It might be interesting to see whether we need PacketListRecords or whether we
can just use the frame_data structure for that, and save some memory (the
pointer to the frame_data structure in the PacketListRecord, if nothing else).
This one:
686 calls for 22500800 bytes: thread_7fff70cf6ca0 |start | main | cf_read |
wtap_read | libpcap_read | libpcap_read_rec_data | file_read | fill_out_buffer
| g_malloc | malloc | malloc_zone_malloc
might be the fast-seek data structures, but we might want to make sure of that.
There's probably not much we can do about the GtkTreeView data structures.
AFS scores high because it's an AFS trace.
--
Configure bugmail: https://bugs.wireshark.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
