Wireshark · Wireshark-dev: Re: [Wireshark-dev] continuous packet capture

[Brian Daniel <daniel_brian@xxxxxxxxxxxx>]

Jon,

 

Is your goal to output directly from wireshark into a MySQL database instead of a .cap file? Then be able to generate .cap files out of the database based on date, time range, and other standard filters. Keep us posted on your progress.

 

For now, I have to capture 40 .cap files each at 512 Meg and tell it to rotate. When making 041.cap the 001.cap goes away, etc... But the problem is no one has time nor do they want to analyze all that data before it goes away. We want to be able to have a packet capture available if a user says they had a problem at (specified date and time) then I'll make a copy of the files related to that date & time range and analyze them. Yes, I can do this already with the existing wireshark but a database would be nice.

 

Once the database grows to a specified size, will you be able to automatically drop oldest packets to make room for the new ones. And the big question: while you're deleting oldest mysql records, will "zero" newest capture packets get dropped or not be added to the database?
 

Good Luck. Cool Project!

Brian

 

On Thu, Mar 5, 2009 at 2:36 PM, Jon Polacheck <jonpol@xxxxxxxxxxxxx> wrote:

I am working on a continuous packet capture application (think
Infinistream, Gigastor, NetVCR) written in Perl (only because that's
what I know). Here is what I have so far.


This works (or seems to).  More optimization possible by removing
unnecessary trailing spaces from each packet hexdump.  Hacked
Data::Hexdumper some, more probably doable.

Used mkfifo to create the named pipe.  Perl sees it as a disk file
(that I called qtfifo).  ENQUE.pl dumps packet hexdumps to the fifo.
DEQUE.pl reads lines from the fifo.  /^0000 / acts as the delimiter.
mysql compression worked with the standard OpenSuSE install, no
recompiling or other mucking about necessary.

Lines used for debugging marked as such.

ENQUE.pl

use Net::Pcap;
use Data::Hexdumper qw(hexdump);

$dev = "eth0";

# used a 50 packet cap file to make sure what came out matched what went in
#$dump = "ip.cap";
#$pcap = Net::Pcap::open_offline($dump, \$err) or die "Can't read
'$dump': $err\n";

# live, real-time feed
$pcap = Net::Pcap::open_live($dev, 1514, 1, 0, \$err);

Net::Pcap::loop($pcap, -1, \&process_pkt, ""); # <- subroutine call

sub process_pkt {
   open(QT, "> qtfifo");
# $_[2] is the third element of the default array "@_" which was created
# by the subroutine call "&process_pkt"
   my $pkt=$_[2];
   $results = hexdump( data ="" $pkt
       , number_format => 'C',
   );
   print QT $results;
   close(QT);
    $i++; # debug
    &stop_run if $i > 100; #debug
}

# all debug below
sub stop_run {
 print "stop_run\n";
 open(QT, "> qtfifo");
 print QT "\nx\n";
 close(QT);
 print "enque ended\n";
 exit;
}


DEQUE.pl

use Time::HiRes ( nanosleep );
use DBI;

$hostname="127.0.0.1";
$database="cpc";
$port="3306";

$dsn = "DBI:mysql:database=$database;host=$hostname;port=$port";

$dbh = DBI->connect($dsn,
   "root",
   "",
   {'RaiseError' => 1});

# call the Net::Packet collector script
system(q{perl ENQUE.pl&});

open(EQT, " < qtfifo");

$i = 0; # debug
$pc = 0; # debug

while(1) {
   $i++; # debug
   $line = readline(EQT);
   if ($line =~ /^0000 / ) {
       $dbh->do(qq{INSERT INTO cpc VALUES ( compress("$pkt"))});
       $pc++ if defined($pkt); # debug
       print "packet $pc:\n$pkt\n" if defined($pkt); # debug
       undef($pkt);
       $pkt .= $line;
   } else {
       $pkt .= $line;
       &theend if $pkt =~ /x/; # debug
   }
   nanosleep(1); # would not work without this!
}

# all debug below
sub theend {
   close(EQT);
   print "$i loops\ndeque ended\n";
   exit;
}

This generated a cap file that looks just fine in Wireshark.

mysql -Br -D cpc -e "select uncompress(packet) from cpc;" | text2pcap -
m_cap.cap

Hope you find this of interest.

Jon Polacheck




___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev@xxxxxxxxxxxxx>
Archives:    http://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
            mailto:wireshark-dev-request@xxxxxxxxxxxxx?subject=unsubscribe