Wireshark · Wireshark-dev: Re: [Wireshark-dev] speed optimizations

[Anders Broman <a.broman@xxxxxxxxxxxx>]

Didier skrev 2013-10-31 20:34:
> Hi,
> On Thu, 31 Oct 2013 11:57:35 +0000, Anders Broman wrote
> > > Are wireshark's developers interested in reviewing and committing these
> patches?
> > reviewing ,yes certainly . Committing, depends I suspect :-)
> >
> > > -1 Base on Yami's work "snapshot feature: apply filter upon previous
> displayed packets only" (without the SAT, only using a true table). help a lot
> when >users are building filters incrementally or move a lot between them.
> > > 2 When filtering don't dissect packet if not necessary. Help a lot, can
> speed filtering up to 2 orders of magnitude (twice to three time as fast common).
> > Could you give a brief description on how it works or is it simpler
> > to just submit a patch? Seems like a waste to do a patch and having
> > it rejected for some reason. I have some ideas on 1 and 2 too and it
> > would be interesting to see if they are similar.
> for 1) It add a new op CHECK_HISTORY in filter virtual machine, keep a cache
> with the last 64 filter expressions and a per packet 64 bits filters result in
> frame_data structure.
>
> When using a new filter it try first to factorize from the cache history up to
> 8 sub expressions, ex:
> filter history:
> tcp.stream eq 6
>
> second filter
> !(tcp.stream eq 6)
> is substituted with
> !(CHECK_HISTORY 1)
>
> then it build a three-states true table for the filter expression, states are
> TRUE (passed), FALSE (not passed), UNKNOWN (must dissect packet)
> for the above there's one history variable and the table is:
> V1  R
> 0   TRUE  passed
> 1   FALSE not passed
>
> If there's no taps, post-dissectors or column-info there's no need to dissect
> packets (minus packet which passed and have dependent frames) the filter
> result is known only by looking at the packet bitfield in rescan_packet.
>
> Another filter:
> tcp.stream eq 6 and http ==> (CHECK_HISTORY 1) and http
>
> the true table is:
> V1  R
> 0   FALSE   known to be false without decoding packets
> 1   UNKNOWN must decode packets.
>
> and it only dissect (tcp.stream eq 6) packets
>
> I have a very rough first patch from our version working with current svn.
Sounds interesting, why not attach it to a bug report even in it's rough 
state? I had some vauge idea to use the
"ignore packet flag" to have cumulative filtering e.g filter on "UDP or 
TCP" then the next filter SIP would only look
in the packets that passed the first filter as the rest would have the 
ignore flag set.
> ==========
> 2) Is doing the same but with protocols ie: during the first pass (when
> visited is false) it save in a per packet bitfield its protocols.
>
> When compiling wireshark it's also extract which dissectors don't call
> subdissectors or only call them via wireshark API
>
> Then when filtering it can:
>
> - from the packet protocols bitfield bypass packet decoding (there's no point
> to decode and filter on a http field if you already know there's no http in
> the packet).
>
> during dissect:
> - bypass some 'leaf' dissectors, if a dissector don't call subdissectors and
> has no interesting fields for the filter evaluation.
>      
> - set tree to NULL when calling dissectors with no 'interesting fields'. I'm
> not sure this one is worth it without the AOT transform which convert a
> dissector to a bunch of get_tvb..., offset += .., new_tvb_subset ...,
> call_dissector(next_dissector)

Why do we need to know if a dissector has sub dissectors? What if we 
kept a Gslist of all protocol handles in a
frame, much like the "protocols string" but always populated on the 
first pass one could then match (some) filters
against that list before deciding if the frame should be dissected or not.

> When compiling wireshark it's also extract which dissectors don't call
> subdissectors or only call them via wireshark API

If this is similar to a previous patch it requres changes to many 
dissectors.

> Regards
> Didier
>
>
> ___________________________________________________________________________
> Sent via:    Wireshark-dev mailing list <wireshark-dev@xxxxxxxxxxxxx>
> Archives:    http://www.wireshark.org/lists/wireshark-dev
> Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
>               mailto:wireshark-dev-request@xxxxxxxxxxxxx?subject=unsubscribe