Wireshark · Wireshark-dev: Re: [Wireshark-dev] Wiretap changes for pcapng

[Guy Harris <guy@xxxxxxxxxxxx>]

On Aug 31, 2015, at 2:56 PM, Hadriel Kaplan <the.real.hadriel@xxxxxxxxx> wrote:

> On Mon, Aug 31, 2015 at 2:43 PM, Guy Harris <guy@xxxxxxxxxxxx> wrote:
>> 
>> For example, in bug 4221
>> 
>>        https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=4221
>> 
>> Paul Long of Microsoft says that we discard interface information in Network Monitor files *and* that, ideally, the NetMon record containing interface information would show up in the "packet" list, so that packet numbers in NetMon files as displayed by NetMon would be the same as packet numbers in those files when displayed by Wireshark.  That would mean that libwiretap would have to present a single "IDB" record with information for *multiple* interfaces.
> 
> I'm not sure I fully grok that. I think what you're saying is the
> Network Monitor file has IDB information, but that the NetMon tool
> displays that information as a single frame in its display (because
> internally in the file format it happens to be an array of interface
> info inside a single file "NetworkInfo record"). So he'd like
> wireshark to do the same in order to keep the frame numbers
> consistent.

Yes.

> What would a Network Monitor file containing interface info for 3
> interfaces look like if it were "converted" to a pcapng file, by some
> tool... *without* losing any semantic info?  Like for example a tool
> built by Microsoft. Or put it another way, what if the NetMon program
> could save in pcapng format directly?
> 
> I think the answer is: it would be a pcapng file with 3 IDBs, each IDB
> with a bunch of Custom Options for things not in the current pcapng
> spec (Gateway addr, DHCP server, etc.);

Custom options or, if we deem that information generally useful, regular options.

> and the pcapng file would also
> contain a Custom Block which represents the single "NetworkInfo
> record", which internally has indices to the IDBs. Those Custom
> Options and Custom Block would use Microsoft's PEN number, and they'd
> decide what their internal format for them is.
> 
> In our case, we can't use Micorsoft's PEN obviously, so we'd use
> Wireshark's PEN and define the internal structure for that ourselves.
> And then we'd modify wiretap's netmon file reader - for example,
> changing its netmon_read() function to return that Custom Block when
> it finds this "NetworkInfo record" in the file. It would also adjust
> the file cursor so that on the next call to netmon_read() it can
> return the first IDB, and the third call to netmon_read() it can
> return the second IDB, etc.

That'd work.

When *writing* NetMon files, resumably we'd ignore NetworkInfoEx records handed to us, and synthesize one from the initial set of interface descriptions instead.  (If further interfaces show up in a one-pass program, we'd just ignore them; the NetworkInfoEx block is informative, but not necessary - each packet has a link-layer header type in newer versions of the file format, and older versions have only a per-file format.)

> (this would undoubtedly require some
> internal state to be kept in the netmon reader, but we do that already
> for other file readers)

Given that we're going to have to supply synthesized "section header" and "interface description" information for file formats such as pcap, we'll have to do that.

If we continue to have offsets in the uncompressed data stream as the file offsets supplied to the seek-read routines, then, for files that have a file header, we could use different offsets into the file header as the fake offsets for the synthesized beginning-of-the-file blocks.  We could use offsets within the NetworkInfo record as the fake offsets for the synthesized IDBs.

That won't work for files that *don't* have file headers, so that an offset of 0 might be the offset of the first record in the file, which might be a packet record.