Home page logo
/

wireshark logo Wireshark mailing list archives

Re: Running tshark on large pcap files
From: Sake Blok <sake () euronet nl>
Date: Wed, 12 Jun 2013 10:07:17 +0200

If you want to search for packets over tcp port 1234 with one mpls label, you would use:

mpls and tcp port 80

If you want to search for packets over tcp port 1234 with two mpls labels, you would use:

mpls and mpls and tcp port 80

But if you want to look for tcp port 80 packets that have zero, one or two mpls labels, you can filter with the 
following filter:

tcp port 80 or (mpls and tcp port 80) or (mpls and tcp port 80)

Beware, each time the word mpls is encountered by the BPF engine, the offsets get increased by 4, but it will never 
reset the offset shift back to 0 so you don't want to add two mpls statements in the last part.

Cheers,
Sake


On 11 jun 2013, at 05:06, Rayne wrote:

Hi, 

I just realized that there is MPLS traffic in my pcap files, and tcpdump doesn't seem to extract those packets with 
the matching ports that are tunneled over MPLS, while tshark does. Is there anyway to tell tshark not to keep state?

Thank you.

From: Rayne <hjazz6 () ymail com>
To: Sake Blok <sake () euronet nl>; Community support list for Wireshark <wireshark-users () wireshark org> 
Sent: Tuesday, June 11, 2013 10:40 AM
Subject: Re: [Wireshark-users] Running tshark on large pcap files

Thanks! I'll try using tcpdump instead.


From: Sake Blok <sake () euronet nl>
To: Rayne <hjazz6 () ymail com>; Community support list for Wireshark <wireshark-users () wireshark org> 
Sent: Monday, June 10, 2013 3:53 PM
Subject: Re: [Wireshark-users] Running tshark on large pcap files

On 10 jun 2013, at 09:14, Rayne wrote:

I'm running tshark on a few large pcap files (each over 100GB in size) to extract packets belonging to a particular 
TCP/UDP port and write them to a file.

I noticed that when tshark first starts, it uses about 90-100% of the CPU, and the processing is pretty fast. 
However, as it continues, it uses more and more of the memory (the server has ~8GB of RAM) and eventually, the CPU 
load is down to 1% or less when it's using almost all of the memory. And it takes days to process one pcap file. I 
had to stop the processing because it was taking too long.

Does this behavior have anything to do with how tshark works on pcap files? Does tshark try to load the pcap into 
memory, and when memory runs out, it slows to a crawl? Is there any way I can make tshark run faster?

Tshark won't load the whole file, but it will keep state of sessions it has seen. So it's memory consumption will 
grow over time. On a 100GB tracefile, I suspect it will run out of physical memory and going to use swap, hence the 
slowdown.

If all you need is TCP/UDP port filtering, you are better of with tcpdump, it does not keep state and the BPF filter 
engine is pretty fast in filtering. You could use:

tcpdump -r infile.pcap -w outfile.pcap "tcp port 80 or tcp port 8080 or udp port 53"

Or something similar to your needs.

Cheers,
Sake


___________________________________________________________________________
Sent via:    Wireshark-users mailing list <wireshark-users () wireshark org>
Archives:    http://www.wireshark.org/lists/wireshark-users
Unsubscribe: https://wireshark.org/mailman/options/wireshark-users
            mailto:wireshark-users-request () wireshark org?subject=unsubscribe


___________________________________________________________________________
Sent via:    Wireshark-users mailing list <wireshark-users () wireshark org>
Archives:    http://www.wireshark.org/lists/wireshark-users
Unsubscribe: https://wireshark.org/mailman/options/wireshark-users
             mailto:wireshark-users-request () wireshark org?subject=unsubscribe


  By Date           By Thread  

Current thread:
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]