Wireshark · Wireshark-dev: Re: [Wireshark-dev] tvb allocator (was: Re: [Wireshark-commits] master b6d20a2: Optimize reseting ep

[Evan Huus <eapache@xxxxxxxxx>]

On Fri, Jul 11, 2014 at 6:07 PM, Anders Broman <a.broman58@xxxxxxxxx> wrote:

Den 11 jul 2014 23:13 skrev "Bálint Réczey" <balint@xxxxxxxxxxxxxxx>:

>
> Hi All,
>
> Please provide the input data for letting others reproduce the results
> or perform the performance tests on pcap files already available to
> the public

Ok I'll see if we can use something from the wiki instead.

>
> I'm not a fan of implementing custom memory management methods because
> partly because I highly doubt we can beat jemalloc easily on
> performance and custom allocation methods can also have nasty bugs
> like the one observed in OpenSSL:
> http://www.tedunangst.com/flak/post/analysis-of-openssl-freelist-reuse
>

We have gone through a set of memory allocation schemes already to try to improve performance (g_slice, emen and now wmem)

Technically, emem was originally added to fix memory leaks caused by exceptions. Wmem (as just a better-architected replacement for emem) is similar.

The block allocator was added later as an optimization because it provided a *significant* improvement in performance over a simple linked-list of OS-alloced blocks (5-10x faster in my benchmarks).

are you saying that you are opposed to that?

As wmem seems to be the faster scheme for packet scope memory allocation /free, why not use it in all possible places where  the scope is "packet"?

> Please don't sacrifice protection for 2% speedup. Please keep wmem
> usage for cases where it is used for garbage collecting (free() after
> end of frame/capture file) not when the allocation and deallocation
> are already done properly.

? A slow scheme might be working well but that in it self does not warrant to not replace it with a faster one. With this reasoning we shouldn't have introduced ep memory in the first place.

What percentage if improvement do you think makes a change worthwhile?

The set of improvements Jacub and I have done lately has given a reduction of 40-50 percent compared to 1.10 measuring with the sample file. The problem is that each improvement only yeald a percent or 2. Agreed that we shouldn't complicate the code for a small speed gain.

In your blog you say that people would accept double the execution time with increased security - I'm not so sure. If say the reformatting of a video takes one hour instead of 30 minutes.

Just my 2 cents
Anders

>
> Thanks,
> Balint
>
> 2014-07-11 8:58 GMT+02:00 Jakub Zawadzki <darkjames-ws@xxxxxxxxxxxx>:
> > Hi,
> >
> > On Sat, Jun 21, 2014 at 10:12:48PM +0200, Jakub Zawadzki wrote:
> >> If we're in topic of optimizing 'slower' [de]allocations in common functions:
> >>
> >> - tvb allocation/deallocation (2.5%, or 3.4% when no filtering)
> >>
> >>    243,931,671  *  ???:tvb_new [/tmp/wireshark/epan/.libs/libwireshark.so.0.0.0]
> >>    202,052,290  >   ???:g_slice_alloc (2463493x) [/usr/lib64/libglib-2.0.so.0.3600.4]
> >>
> >>    291,765,126  *  ???:tvb_free_chain [/tmp/wireshark/epan/.libs/libwireshark.so.0.0.0]
> >>    256,390,635  >   ???:g_slice_free1 (2435843x) [/usr/lib64/libglib-2.0.so.0.3600.4]
> >
> >> This, or next week I'll try to do tvb.
> >
> > ... or maybe this week:
> >
> > ver0 | 18,055,719,820 (-----------) | Base version 96f0585268f1cc4e820767c4038c10ed4915c12a
> > ver1 | 18,185,185,838 (0.6% slower) | Change tvb allocator g_slice_* to wmem with file scope
> > ver2 | 17,809,433,204 (1.4% faster) | Change tvb allocator g_slice_* to wmem with file/packet scope
> > ver3 | 17,812,128,887 (1.3% faster) | Change tvb allocator g_slice_* to simple object allocator with epan scope
> > ver4 | 17,704,132,561 (2.0% faster) | Change tvb allocator g_slice_* to simple object allocator with file scope
> >
> > I have uploaded patches & profiler outputs to: http://www.wireshark.org/~darkjames/tvb-opt-allocator/
> >
> > Please review, and check what version is OK to be applied.
> >
> >
> > P.S: I'll might find some time to do ver5 with slab allocator, but it'll look like object allocator API with epan scope.
> >
> > Cheers,
> > Jakub.
> > ___________________________________________________________________________
> > Sent via:    Wireshark-dev mailing list <wireshark-dev@xxxxxxxxxxxxx>
> > Archives:    http://www.wireshark.org/lists/wireshark-dev
> > Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
> >              mailto:wireshark-dev-request@xxxxxxxxxxxxx?subject=unsubscribe
> ___________________________________________________________________________
> Sent via:    Wireshark-dev mailing list <wireshark-dev@xxxxxxxxxxxxx>
> Archives:    http://www.wireshark.org/lists/wireshark-dev
> Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
>              mailto:wireshark-dev-request@xxxxxxxxxxxxx?subject=unsubscribe

___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev@xxxxxxxxxxxxx>
Archives:    http://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
             mailto:wireshark-dev-request@xxxxxxxxxxxxx?subject=unsubscribe