Ethereal-dev: Re: [ntar-workers] Re: [Ethereal-dev] Re: NTAR - PCAP next generationdump file f
Comments marked with --LEGO--
On 6/28/05, Gianluca Varenni <gianluca.varenni@xxxxxxxxxxxx> wrote:
> Comments marked with --GV--
>
> Have a nice day
> GV
>
> ----- Original Message -----
> From: "LEGO" <luis.ontanon@xxxxxxxxx>
> To: "Gianluca Varenni" <gianluca.varenni@xxxxxxxxx>
> Cc: "ronnie sahlberg" <ronniesahlberg@xxxxxxxxx>; "Ethereal development"
> <ethereal-dev@xxxxxxxxxxxx>; "ntar-workers" <ntar-workers@xxxxxxxxxxx>
> Sent: Monday, June 27, 2005 4:21 AM
> Subject: [ntar-workers] Re: [Ethereal-dev] Re: NTAR - PCAP next
> generationdump file formatimplementation
>
>
> I was not clear at all in my last mail so I'll re-expose myself.
>
> Please note that I'll refer to "chunks" for segments of data
> containing more records and to "records" to refer to atoms. I find the
> term block confusing (in my forma-mentis a block seems something that
> contains more records).
>
> The essential point I tried to make is to have the marker-records at
> predictable locations in the file.
> That would be N*chunk_size bytes.
>
> --GV--
> That's clear.
> --GV--
>
>
> To archieve that we need:
> A padding record to fill in the space between the last packet and the
> location of the next marker record.
> And, a marker record that tell us what we have so far.
>
> --GV--
> Well, I don't see why you cannot use the sections for the same purpose. You
> can jump between jumps easily, and we can easily add some information
> (either in the interface description block or by means of a sort of marker
> block) that tells number of packets/bytes, timestamps... In my opinion, this
> is one of the main uses of sections. Any other opinion about that?
--LEGO--
You might use hsbs as "chunk markers" I find that more than OK. As far
as the hsb contains the offset up to the next one (if any) [see my
note about padding bellow].
That means that we will not have to have a fixed chunk size for all
the file but still we could skip from one hsb to the next. As far as
enough info to know where we are and where to go its conveyed in the
hsb that's ok.
The problem, I did not consider up to now is how while skipping do I
find where the interface descriptions are?
Once we sought forwrad several chunks most probably we've had skipped
some. This needs a solution.
We could replicate all the known interface descriptions after every
HSB. I don't like it... any other ideas?
--LEGO--
> Moreover, the idea of having a padding block is troublesome: by definition,
> the smallest block in pcap-ng is 12 bytes. If we create a padding block
> because we want to support these markers at fixed locations, we need to be
> *very* careful of where to put these padding blocks (i.e. we need to take
> care of having either 0 bytes left before a new marker, or >= 12 bytes,
> otherwise we are screwed up).
--LEGO--
That's just a little more waste, the solution is somewhat trivial.
if (packet_len + 12 > remaining && packet_len != remaining) {
write_padding(remaining);
write_hsb();
}
write_packet();
BTW. I believe that zero padding would be a better solution.
--LEGO--
> Finally, as I said before having these markers at fixed locations is a real
> pain in the neck when you have a file with multiple sections *and* the SHB
> does not contain the length of the section.
---LEGO---
The SHB is the right place for the chunk lenght option.
It's just a matter whether it could, should or MUST be there.
I believe it must be there. Even more, I would say it's value MUST be
between 2Mb and 16Mb.
---LEGO---
> --GV--
>
> Another point is not to have to seek backwards to fill in fields while
> dumping. Neither I like the Idea of having to keep a whole chunk in
> memory nor to have to keep more than few very essential state
> variables.
>
> --GV--
> I totally agree with you.
> --GV--
>
> I think either the file header or the HSB should tell which chunk-size
> we are using.
> The whole dump-file should use one an only one chunk-size.
>
> --GV--
> No. A file contains multiple sections (possibly coming from multiple cat'ted
> files). So *if* we really want to use this chunks approach, the chunk size
> should be saved in the SHB.
> --GV--
--LEGO--
I volunteer to write a ntar-cat that either takes the chunk_size of
the first hsb or allows the user to set it.
--LEGO--
>
> On 6/27/05, Gianluca Varenni <gianluca.varenni@xxxxxxxxx> wrote:
> > In general, I prefer your solution over Ronnie's one: the basic idea of
> > pcap-ng was to create not too much nesting in the file, so at the moment
> > we
> > basically have two levels, sections and blocks. In the future it's
> > possible/probable that we will have some nesting, especially to support
> > compressed and cyphered blocks.
>
> I like it that way. Nesting stuff would mean either to seek back to
> write the length or to keep in memory a whole meta-record before
> writing. I do not like either solution.
>
> [snip]
> > In my opinion the marker should contain the following informations
> > - number of packets up to the next "marker block"
> This is a forward reference
> > - number of bytes up to the next "marker block"
> my point is we should know this before even reading the first marker-record.
> > - timestamp of the first and last packet up to the next "marker block"
> I would put those before the marker record.
> > - offset to the next "marker block"
> In my scenario we know it beforehand
> > - other??
>
> _______________________________________________
> ntar-workers mailing list
> ntar-workers@xxxxxxxxxxx
> https://www.winpcap.org/mailman/listinfo/ntar-workers
>
>
--
This information is top security. When you have read it, destroy yourself.
-- Marshall McLuhan
