> From: Guy Harris [mailto:guy@xxxxxxxxxx]
> Sent: Monday, March 13, 2000 3:06 PM
> To: Eric Bresie
> Cc: ethereal-dev@xxxxxxxx
> Subject: Re: [ethereal-dev] High Level Filter/Plugin and Net-ML
>
>
> > On a project I am starting to work with, we have a mail messaging format
> > that we are trying to capture, database and analyze. I have seen
Ethereal
> > working and I thought this might be a starting point for the project.
> >
> > How difficult is it, based on the network packets being captured, to re
> > assemble these messages, given that some of them may be either
> incomplete or
> > duplicates (box a is sending the same message to box b and box c)
> >
> > Would this be good to develop a plugin to handle this?
> >
> > Could the message be recreated based on the TCP header info (if sent via
> > TCP - some are some aren't)?
>
> "Recreated" in what sense?
We have a predefined priopriety messages format (we'll call it Generic
Message Format-version1 or GMF-1 and Generic Message Format-version2 or
GMF-2) which are autofilled (with and without user input) by other systems
and sent between systems. The hope is to capture and database the mail
messages (header and body of the mail message).
This transmission may occur over different transmission media as well (over
radio for example) speaking different network protocols. With the
databased message we can analyze, based on the content, the actions that
occurred and how they should or could have been conducted - for training
purposes.
> TCP segment boundaries (which almost always are equivalent to link-layer
> frame boundaries; only if the maximum segment size for a TCP connection
> is incorrectly chosen, so that TCP segments have to be fragmented, is
> that not the case) don't necessarily correspond to higher-level message
> boundaries:
>
> 1) a higher-level message might take more than one TCP segment;
This is our case. A single mail message could be broken up into multple TCP
segments (or other protocols but I will keep with this for the moment).
> 2) a single TCP segment might contain pieces of more than one
> message, in that it might contain, for example, the end of
> message N and the beginning of message N+1.
Which is also a concern. We are interested in Mail message 1 being sent
from box A to box B (and possibly box C,etc) which will be broken into
multiple TCP segments. We want to completed the mail message.
> Currently, the Ethereal TCP dissector analyzes individual frames as
> independent entities, so there's no provision for handling higher-level
> messages that take more than one TCP segment (and Ethereal's IP
> dissector doesn't reassemble fragmented IP datagrams).
Is there something that does this already? Would this require a Fragement
Reassembler plugin perhaps?
> > Does the TCP header provide a time stamp
> > (absolute or relative?) indicating a time for the source/destination
machine?
>
> What the standard TCP header provides is documented in RFC 793,
> "TRANSMISSION CONTROL PROTOCOL, DARPA INTERNET PROGRAM PROTOCOL
> SPECIFICATION". It includes no time stamps whatsoever.
> There are also "options" that can be added to the TCP header, after the
> fixed part of the header, and I think there are time stamps that can be
> added, but there's no guarantee that they are added, and, as they're
> added by the sender, obviously the time on the destination machine isn't
> there, as the sender doesn't know that time.
> There *are* time stamps in the headers in capture files; those are what
> are used to display the time in the time column in the packet list
> display, and in the "frame" layer in the packet tree display for a
> packet.
Would a "generic mail message plugin" to give a "Content of the GMF-1
message" be needed to get the interact between the "Contents of TCP stream".
> > Doesn't the reliability and completeness of the message get handled by
TCP level?
>
> The TCP level of your machine's network stack, or the TCP level of
Ethereal?
>
> Currently, Ethereal's TCP dissector doesn't do any inter-packet stuff,
> as noted above - the only stuff of that sort that's currently done is in
> the "Follow TCP Stream" code.
>
> And even your machine's network stack merely ensures that the stream of
> bytes sent over the TCP connection either arrives, in order, or an error
> is reported, in the sense that if stuff takes too long to arrive, the
> connection is eventually broken. The TCP code has no clue which data is
> part of which message.
>
I'm gonna take the XML into a different thread..
Eric Bresie
ebresie@xxxxxxx
