Wireshark · Wireshark-dev: Re: [Wireshark-dev] Sub-nanosecond timestamps

[David Mirabito <davidm@xxxxxxxxxxxx>]

In my playpen right now, it's in 2^-24th's of a ns, and then I convert to a picosecond value on demand for display. The main reason for that choice (it actually wastes a byte) is that it happens to be our wire format so was the first thing I thought of. I'd be happy to change to something more general - once the things to changes have been identified, using a different in-memory representation is a straightforward edit so long as the pretty-printers can interpret.

We've thought a bit about potential alternatives:

These three have the property that they're all less than one nanosecond, so zero is an acceptable number and timestamp producers who leave uninitialised remain correct, and consumers who *are* aware still get correct results for their calculations.

* 2^-32 of a nanosecond

   + provides way more precision that anyone needs for the foreseeable future

* picoseconds (i.e max 1000, before rolling into abs_time.nsec)

    + Is what we'd like right now

    + easy to eyeball, to_string is trivial

    - Will we have to do this exercise again at some point in the future?

* attoseconds (max 1B before rolling into abs_time.nsec)

    ? Ridiculously futureproof

    + Follows the 10^-9 progression already established by seconds/nanoseconds

Or:

* 2^-64th of a second

    ? even more futureproof (not sure it matters at that level anymore .. 10zs)

    + only need to roll over from this field into seconds, no need to then carry from ns into s

    - Duplicates information already present in the nsec field of nstime_t. We could define that when tsprec == ps it's the setter's responsibility to keep in sync?

There I'm assuming the constraint that we can't change nstime_t's layout/meaning. There are dozens of files which do things manually with it's members which at least need to be inspected for continued correctness and likely need changes; i.e places that statically initialise 1 millisecond with "{0,1000000}".

If we *can* change nstime, then it'd become a 16byte structure, perhaps:

* seconds:nanoseconds:attoseconds

    + consist powers of 10^-9

    - Math becomes annoying since there's two places one may have to carry

* seconds:nanoseconds:subnanoseconds

    + Can treat as 64bit fixed-point ns, so don't need to worry about overflow from the new bytes (This would be my preference)

* seconds:64bit-subseconds

    + nstime becomes 128bit fixed-point seconds

    - If guint128 isn't a thing, we'd still have emulate anyway so may as well use one of the above.

I'd be fine to take a pass this if it's agreed to be worthwhile, but I'm more nervous about out-of-tree plugins and scripts which could silently break. On the flip side, when done, all existing calcs (response time, iograph...) suddenly get can use the extra bits.

Would it make sense to provide enough helper functions/macros so folks won't need/want to poke inside anymore, perhaps as a pre-pass to actually making any format change?

Cheers,

David M

On 27 June 2018 at 16:26, Stephen Donnelly <Stephen.Donnelly@xxxxxxxxxx> wrote:

Sub-ns support sounds fine in principle, we can make use of a few bits at least.

 

I’m not sure if supporting both ns and ps in parallel is the best idea? What would your ts_subs have in it?

 

Stephen

 

From: Wireshark-dev <wireshark-dev-bounces@wireshark.org> On Behalf Of David Mirabito
Sent: Wednesday, 27 June 2018 2:29 AM
To: wireshark-dev@xxxxxxxxxxxxx
Subject: [Wireshark-dev] Sub-nanosecond timestamps

 

Hi List,

 

We would like to contribute awareness to Wireshark of better-than-nanosecond precise timestamps.

 

Currently our MetaWatch product will tap + aggregate upto 30 x 10G Ethernet streams, appending a trailer to each frame through an (up to) 32GB buffer should total ingress bandwidth exceed egress/capture rate. The basic trailer format contains metadata such as Rx Device & Port IDs, as well as a nanosecond-precise timestamp. 

Recently we have added support for sub-nanosecond precision to these timestamps in an extension to the trailer. This is an optional config which also adds a fractional-ns value to each frame, growing the trailer by 4-bytes and including 24 bits of additional timestamp. 

 

The on-wire units are 1 bit == 1/2^24th of a ns (approx 60 atto S) although in practice we see meaning down to around the picosecond level.

 

 

My first thought was to extend the nstime_t struct, either adding a new field or extending ns to be represented by some 64bit fixed-point scheme. However quite a lot of code reaches into the internals of this struct for initialisation or arithmetic, and would require updating for the new layout. At best this would require updating a few hundred files, and possibly would break external plugins and lua scripts so I suspect this might not be the ideal way forward.

 

 

Our current proof-of-concept instead, keeps nstime as-is, and extends frame_data to include a ts_subs field:

 

* Define enums for WTAP_TSPREC_PSEC, TS_PREC_FIXED_PSEC,  and TO_STR_TIME_RES_T_PSECS, etc

* Add ts_subnano field to frame_data. This allows anyone else using nstime_t directly to continue to do so unmodified but those who can respect ps can do so.

* Add "Picoseconds" to view->timestamp format ui/qt menu

* Handle the ps enums in throughout column-utils.c and to_str.c

 

To obtain useful values, I'm also running a change to our already existing packet-metamako dissector, which provides the option to take our timestamp from the trailer, and nstime_copy() it over pinfo->fd->abs_ts. It can also set tsprec if the subnano trailer extension is present.

 

This seems to work, although I haven't yet convinced myself that it's guaranteed to be safe to splat over fd->abs_ts from a trailer dissector. It was pointed out to me today that previous dissectors may have previously copied-by-value the old abs_ts, say for request/response latency measurements. We're open to other alternatives (such as hooking at the wiretap level and setting abs_ts nice and early?). This is somewhat orthogonal to subnanos as we'd also like to do this for the basic ns timestamps without requiring users to pre-process their tracefile to promote our timestamps into the packet header for consumption by standard tools. At this stage I'm not sure if that should be a separate email thread or makes sense to discuss in the context of this work.

 

 

Work along these lines would also help other packet/wiretap formats. We are aware that the Endace ERF consumer for one, throws some precision away when ingesting that format: Timestamps there are specified in terms of 1/2^32th of a second, or around 233ps, of which some is lost when rounding to the nearest ns as is done currently. 

 

 

I guess at this stage I'm interested in soliciting feedback. I intend to continue on the current PoC until it's in a reviewable format, but would certainly appreciate any comments on the approach as stated. There might be interactions and gotchas we haven't yet considered.

 

I'm happy enough to use both our trailer format and ERF files as vehicles to ingest precise timestamps for now. It's not currently blocking us, but it might worthwhile to kick off a parallel discussion on how to represent these in pcapng for instance. What do you think?

 

 

Finally, we have a booth in the Reef at Sharkfest'18 this week, and I'd love to talk details either on this list, or in person at the conf.

 

Thanks,

David Mirabito.

 

 

___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev@xxxxxxxxxxxxx>
Archives:    https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
             mailto:wireshark-dev-request@wireshark.org?subject=unsubscribe