Monday, March 9, 2009

Episode #8: Netstat Protocol Stats

Ed Says:

On Windows, the netstat command has really got a lot of features that are useful for analyzing network behavior. Even without installing a sniffer, you can learn a good deal about network traffic with a stock Windows machine by running netstat with the -s flag to see statistics for all supported protocols. You can select an individual protocol's stats with -p [proto], with TCP, UDP, ICMP, and IP supported. On Vista, they also added IPv6, ICMPv6, TCPv6, and UDPv6.
C:\> netstat -s -p ip

That'll show you IPv4 stats including packets received and fragments created.
C:\> netstat -s -p tcp

This one shows the number of active opens and reset connections, among other things. Those stats are useful if you suspect some kinds of denial of service attacks.

Hal Comments:

The Linux "netstat -s" command will also dump statistics:
$ netstat -s
Ip:
115851638 total packets received
237 with invalid headers
0 forwarded
0 incoming packets discarded
115825742 incoming packets delivered
72675668 requests sent out
2914 reassemblies required
1457 packets reassembled ok
Icmp:
34672 ICMP messages received
18 input ICMP message failed.
...

Unfortunately, while there are command line options to dump just the TCP or just the UDP statistics, they don't work consistently across different Linux distributions. In some cases they even include other protocol statistics, like IP and ICMP stats, along with the TCP or UDP stats.

I did come up with a fairly gross hack for pulling out sections of the report for a specific protocol:
$ netstat -s | awk '/:/ { p = $1 }; (p ~ /^Tcp/) { print }'
Tcp:
64684 active connections openings
25587 passive connection openings
1043 failed connection attempts
236 connection resets received
15 connections established
114808177 segments received
71655514 segments send out
24271 segments retransmited
11 bad segments received.
2906 resets sent
TcpExt:
1640 invalid SYN cookies received
15 ICMP packets dropped because they were out-of-window
57520 TCP sockets finished time wait in fast timer
...

The first part of the awk expression matches on the ":" character in the "header" line of each protocol section and sets our magic "p" variable to the current protocol name. That value remains in "p" until we reach the next header, and so on. The second part of the awk expression does a regular expression match against the current value of "p" and prints the current line as long as "p" matches the protocol we're looking for. That gets us the header line itself, plus all of the following lines of output up until the next header.

Why is this so clunky? Basically, Unix commands are generally poor at "remembering context" across multiple lines, so you often end up with these sorts of hacked solutions.

Paul Says:

As byte_bucket mentioned, things work a bit differently on OS X. Hal's command above needs to have a lower case "tcp" in order to work in OS X:

$ netstat -s | awk '/:/ { p = $1 }; (p ~ /^tcp/) { print }'


Also the following command:

$ netstat -s -p tcp


Works great on both Linux and OS X. I find these commands very useful for network troubleshooting, especially given slow performance or high error counts on the network switch.