On Tue, 2008-01-22 at 00:20 +0100, Ulf Lamping wrote:
> Stephen Donnelly schrieb:
> > On Mon, 2008-01-21 at 22:00 +0100, Ulf Lamping wrote:
> >> Gianluca Varenni schrieb:
> >>> http://www.winpcap.org/pipermail/ntar-workers/2006-March/000122.html
> > I believe part of the idea behind allowing the timestamp precision to be
> > specified as a binary fraction (2^-X) is to allow the use of Endace ERF
> > timestamps in pcapng/ntar natively.
> >
> Well, my complain in the first place is that the specification text is
> pretty easy to misunderstand (just as I did), which is usually a very
> bad thing for a specification.
Certainly true.
> > Timestamps in network traces are typically 64-bit, comprising a 32-bit
> > 'seconds' part with a UNIX epoch, and a 32-bit 'subsecond' field
> > counting milli/micro/nanoseconds withing the second. The most
> > significant few bits of the subsecond are not used as the conuter
> > saturates before using all the available bits.
> >
> I'm pretty much aware of that - I'm the author of
> http://wiki.wireshark.org/Development/LibpcapFileFormat and I've
> implemented the Wireshark switch from millisecond to nanosecond internal
> resolution :-)
Very much appreciated. In fact the current Wireshark code for reading
ERF records converts the binary fractions to nanoseconds, a definite
improvement.
> > ERF timestamps are also 64-bits, and can be considered to be a 64-bit
> > value in units of 2^-32 seconds, with a UNIX epoch. Alternatively it can
> > be considered a fixed point count of seconds from the UNIX epoch.
> > Conveniently the 32 MSB are equivalent to the conventional 'seconds'
> > count, while the 32 LSB can be considered a binary fraction of seconds.
> > The lowest few LSB may not be used by specific hardware depending on the
> > available clock resolution.
> >
> Thanks for explaining it. But it just doesn't really help to explain it
> here, it must be explained in the spec! Otherwise the next one that
> graps the spec and implements stuff will be trapped.
I would be happy for this to be put into the spec, if there is
consensus.
> > Using ERF timestamps directly in pcapng avoids the requirement to
> > convert the timestamp formats on the fly at capture time. This can be a
> > significant load at extreme packet rates.
> Full ACK.
>
> But as a drawback, now the former libpcap timestamp format is not
> directly supported and needs conversion "on the fly" while writing
> packet data.
My understanding of the pcapng proposal from some time ago was that the
timestamps could be second/subsecond or second/fraction on a per-block
basis, with some indication in the block header which was in use.
Pcapng would read either format, and accessor functions would provide
the user with their preferred format at read-time.
I'm not sure if the specification still supports this concept, I should
look closer.
I could look at an ERF <> pcapng format converter also. In principle
editcap could do this, but only once Wireshark internally supports
concepts like multiple interfaces per device/file.
> > It does suggest that timestamp
> > accessor functions should be available so that the any timestamp
> > (record/block) can be requested in any format.
> >
> That reminds me of the big/little endian problem, which is solved by
> writing stuff in the native format and being able to detect and then
> read (and probably convert) whatever comes in.
>
> For timestamps this means: write libpcap, ERF, or whatever comes in and
> being able to detect and then read (and probably convert) the timestamp
> format.
Right, essentially that idea but with some indicator to suggest which
format exists, as heuristic detectors could be awkward and unreliable.
> I'm pretty much aware that this adds complexity to the programs reading
> the format, but as you've said writing performance is the key.
Right, my opinion would be that in general writing data is more time
critical than reading it. In many cases device read accesses are faster
than writes which offsets any conversions necessary at read time.
Stephen.
--
-----------------------------------------------------------------------
Stephen Donnelly BCMS PhD email: sfd@xxxxxxxxxx
Endace Technology Ltd phone: +64 7 839 0540
Hamilton, New Zealand cell: +64 21 1104378
-----------------------------------------------------------------------
