Wireshark · Wireshark-dev: [Wireshark-dev] Sub-nanosecond timestamps

[David Mirabito <davidm@xxxxxxxxxxxx>]

Hi List,

We would like to contribute awareness to Wireshark of better-than-nanosecond precise timestamps.

Currently our MetaWatch product will tap + aggregate upto 30 x 10G Ethernet streams, appending a trailer to each frame through an (up to) 32GB buffer should total ingress bandwidth exceed egress/capture rate. The basic trailer format contains metadata such as Rx Device & Port IDs, as well as a nanosecond-precise timestamp. 

Recently we have added support for sub-nanosecond precision to these timestamps in an extension to the trailer. This is an optional config which also adds a fractional-ns value to each frame, growing the trailer by 4-bytes and including 24 bits of additional timestamp. 

The on-wire units are 1 bit == 1/2^24th of a ns (approx 60 atto S) although in practice we see meaning down to around the picosecond level.

My first thought was to extend the nstime_t struct, either adding a new field or extending ns to be represented by some 64bit fixed-point scheme. However quite a lot of code reaches into the internals of this struct for initialisation or arithmetic, and would require updating for the new layout. At best this would require updating a few hundred files, and possibly would break external plugins and lua scripts so I suspect this might not be the ideal way forward.

Our current proof-of-concept instead, keeps nstime as-is, and extends frame_data to include a ts_subs field:

* Define enums for WTAP_TSPREC_PSEC, TS_PREC_FIXED_PSEC,  and TO_STR_TIME_RES_T_PSECS, etc

* Add ts_subnano field to frame_data. This allows anyone else using nstime_t directly to continue to do so unmodified but those who can respect ps can do so.

* Add "Picoseconds" to view->timestamp format ui/qt menu

* Handle the ps enums in throughout column-utils.c and to_str.c

To obtain useful values, I'm also running a change to our already existing packet-metamako dissector, which provides the option to take our timestamp from the trailer, and nstime_copy() it over pinfo->fd->abs_ts. It can also set tsprec if the subnano trailer extension is present.

This seems to work, although I haven't yet convinced myself that it's guaranteed to be safe to splat over fd->abs_ts from a trailer dissector. It was pointed out to me today that previous dissectors may have previously copied-by-value the old abs_ts, say for request/response latency measurements. We're open to other alternatives (such as hooking at the wiretap level and setting abs_ts nice and early?). This is somewhat orthogonal to subnanos as we'd also like to do this for the basic ns timestamps without requiring users to pre-process their tracefile to promote our timestamps into the packet header for consumption by standard tools. At this stage I'm not sure if that should be a separate email thread or makes sense to discuss in the context of this work.

Work along these lines would also help other packet/wiretap formats. We are aware that the Endace ERF consumer for one, throws some precision away when ingesting that format: Timestamps there are specified in terms of 1/2^32th of a second, or around 233ps, of which some is lost when rounding to the nearest ns as is done currently. 

I guess at this stage I'm interested in soliciting feedback. I intend to continue on the current PoC until it's in a reviewable format, but would certainly appreciate any comments on the approach as stated. There might be interactions and gotchas we haven't yet considered.

I'm happy enough to use both our trailer format and ERF files as vehicles to ingest precise timestamps for now. It's not currently blocking us, but it might worthwhile to kick off a parallel discussion on how to represent these in pcapng for instance. What do you think?

Finally, we have a booth in the Reef at Sharkfest'18 this week, and I'd love to talk details either on this list, or in person at the conf.

Thanks,

David Mirabito.