Archive

Posts Tagged ‘netstat’

Example Unix and Windows/Powershell Network Socket Table Commands

October 7th, 2016 Comments off

1. What tcp sockets are open?

Unix
netstat -an | egrep -I “^tcp.*LISTEN”
netstat -an | egrep -I “^tcp.*LISTEN” | awk ‘{split($4,a,”:”);print a[2]}’

Powershell
netstat -an | select-string “LISTEN”
netstat -an | select-string “LISTEN” | %{$i=%{$_.Line.Split(“:”)};$j=$i[1] -replace ‘\s+’,’ ‘;$k=$j.Split(” “);$k[0]} | where {$_ -ne “”}

2. What TCP ports are in an ESTABLISHED/LISTEN state

Unix
netstat -an | egrep ESTABLISH
netstat -an | egrep ESTABLISH | awk ‘{split($4,a,”:”);print a[2]}’

Powershell
netstat -an | select-string “ESTABLISHED”
netstat -an | select-string “ESTABLISHED” | %{$i=%{$_.Line.Split(“:”)};$j=$i[1] -replace ‘\s+’,’ ‘;$k=$j.Split(” “);$k[0]} | where {$_ -ne “”}

3. What connections are in some sort of Wait State?

Unix
netstat -an | egrep WAIT
Powershell
netstat -an | select-string “WAIT”

4. How many connections are in some sort of WAIT state?

Unix

netstat –an | egrep WAIT | wc –l
Powershell
netstat -an | select-string “WAIT” | where {$_ -ne “”} | Measure-Object –Line

5. What are the IP address and ports of the remote machines that are in some sort of WAIT state to this machine?

Unix

nestat -an | egrep WAIT | awk ‘{print $4}’
Powershell
netstat -an | select-string “WAIT” | %{$_ -replace ‘\s+’,’ ‘ } | %{$i=$_.Split(” “);$i[3]}

6. What are the IP address (no ports) of the remote machines that are in some sort of WAIT state to this machine?

Unix

netstat -an | egrep WAIT | awk ‘{print($4,a,”:”);print a[1]}’
Powershell
netstat -an | select-string “WAIT” | %{$_ -replace ‘\s+’,’ ‘ } | %{$i=$_.Split(” “);$i[3]} | %{$i=$_.Split(“:”);$i[0]}

7. What are the unique IP address of remote connections in some sort of WAIT state?

Unix
netstat -an | egrep WAIT | awk ‘{split($4,a,”:”);print a[1]}’ | sort -u
Powershell
netstat -an | select-string “WAIT” | %{$_ -replace ‘\s+’,’ ‘ } | %{$i=$_.Split(” “);$i[3]} | %{$i=$_.Split(“:”);$i[0]} | Sort-Object | Get-Unique

Network Socket Table

June 2nd, 2016 No comments

 

Preface

The purpose of this document is to give a list of commands that can be executed from the command line during networking or system problem determination. These commands focus on parsing the network socket table (namely: netstat -a). The socket table is arguably a common denominator for any system (regardless of operating system) that participates in the TCP/IP protocol.

Both UNIX/LINUX and Windows (Powershell) commands are given. It is important to understand the context of the command before entering it; more so than constructing the command line syntax. Once you understand the context, you can simply cut and paste the command.

The ability to quickly parse the socket table allows the administrator to be responsive during times of crisis.

What follows is a scenario that shows:

· the value of the quick-use of the socket table

· insight into the left-side and the right-side of the table

· a summary of the context, and finally

· a list of commands in both UNIX/LINUX and Powershell.

Click Here for and Index of UNIX and Powershell Commands


Scenario

It was my first day on the job. I just moved from Austin, Texas to Greensboro, North Carolina to work for a large financial institution. I was fresh from a stable IBM support environment. I was excited to tackle real world issues again.

The environment was a distributed computing environment featuring different operating systems: OS400, AIX, Linux and Windows. A few dedicated AIX systems were also used as part of the network infrastructure… different from the traditional roles for AIX systems: like DB2, Oracle, WebSphere, etc. This was before F5/BigIP devices…. dedicated to load balancing, Data Power devices that bridge multi-level protocols at wire speed, or Cisco’s defining support of vlans in VMware environments. In other words, using AIX or LINUX operating systems to perform functions that have now migrated to “network devices”.

There were 2-twin AIX systems that were used as proxy servers (IBM-B50, 2U-rack mounted units). The network engineer said they were load-balanced.

So…I logged into both machines to for look and after a few minutes I said “well, I don’t think they are load balanced”. The network engineer asked, “what do you mean?”. I said, “well, I only see 3 connections on this partner, and about 1000 connections on the other partner.” These load-balanced proxy servers had been running for about two years.

I simply logged on to both systems and did

netstat -an | egrep <PROXY-SERVER-PORT> | wc -l

Same company, about a week later…this time there was a distributed application hanging on the AIX side. I traced the problem down to a connection to an OS400 machine (via the socket table). I asked the Application team to check the application processes on the OS400 system and they told me that the OS400 machine was not relevant. It wasn’t until I was in a room with the application folks explaining the socket table that they finally looked at the OS400 and found the issue. I pointed to the IP address of the OS400 appearing in the socket table.

So, I love the socket table. It’s one of the first areas I consider whenever I walk into a new environment. It’s a system administrator’s quick insight into applications in a distributed environment. Any host system that participates in the TCP/IP protocol is going to have this table.


The Left-side versus the Right-Side

I think of the socket table as having a left side and a right side. I frequently “slice” (using the Unix cut or awk command) the table into two and then parse the output. The left side will show the local IP addresses of the system (as well as the loop back). The right side shows the IP addresses of machines connecting to this machine.

The following diagram shows a search for ESTABLISHED connections…

image

The following shows “slicing” the table to evaluate remote connection information.

image

In this case our local host has 10 connections to port 5201 on host 192.168.1.101.   I ran the iperf tool in server mode on host 192.168.1.101 and started the client on the local host requesting 10 connections (the iperf tool is used to test network bandwidth).   We also have one connection to port 22 on 192.168.1.101.  Port 22 is the ssh server. This means I have ssh’d into machine 192.168.1.101 from the local host.

 

Context

I always parse the socket table to get a sense of what a server is doing anytime I am called into any system issue, especially when I am looking at a server for the first time. If it’s an issue with an application that works over the network, I always track down the port associated with the application. Here is a partial list of things I look for when parsing the socket table and the context:

Item Context
Ports in LISTEN state This gives an idea of how many server types of applications on the host.
Ports not in LISTEN state This gives an idea of current connection from remote hosts.
Ports in some sort of CLOSE state for a long time. Repetitive use of this command and counting them will show if some connections are hanging in close state.
Ports in SYN state for a long time Repetitive use of this command and counting them will show if some connections are hanging in SYN state.
Ports in WAIT states for a long time Repetitive use of this command and counting them will show if some connections are hanging in WAIT state.
Number of Ports accessing an application Server Port Repetitive use of this command and filtering about a specific port will show the application server load network-wise.
Number of unique hosts connecting to this server Cutting the table in half (vertically, column-wise) and looking at the right side of the socket table reveals this information.  It helps to pipe this output to sort and a count tool.
A list of connections that don’t have 0 in send and receive queues (UNIX only). Repetitive use of this command and observing if the send/receive queues are non-zero can indicate an application is not processing.

 

The Commands


1. What tcp sockets are open?

UNIX

netstat -an | egrep -I “^tcp.*LISTEN”

1a

netstat -an | egrep -I “^tcp.*LISTEN” | awk ‘{split($4,a,”:”);print a[2]}’

1b


Windows (Powershell)

netstat -an | select-string “LISTEN”

image

netstat -an | select-string “LISTEN” | %{$i=%{$_.Line.Split(“:”)};$j=$i[1] -replace ‘\s+’,’ ‘;$k=$j.Split(” “);$k[0]} | where {$_ -ne “”}

image


2. What TCP ports are in an ESTABLISHED state

UNIX

netstat -an | egrep ESTABLISH

2a

netstat -an | egrep ESTABLISH | awk ‘{split($4,a,”:”);print a[2]}’

2b

Windows (Powershell)

netstat -an | select-string “ESTABLISHED” | %{$i=%{$_.Line.Split(“:”)};$j=$i[1] -replace ‘\s+’,’ ‘;$k=$j.Split(” “);$k[0]} | where {$_ -ne “”}

image


3. What TCP ports are in some sort of Wait State?

UNIX

nestat –an | egrep WAIT

3a

 

Windows (Powershell)

netstat -an | select-string “WAIT”

image


4. How many connections are in some sort of WAIT state?

UNIX

netstat -an | egrep WAIT | wc -l

4a

Windows (Powershell)

netstat -an | select-string “WAIT” | where {$_ -ne “”} | Measure-Object –Line

image


5. What are the IP address and ports of the remote machines that are in some sort of WAIT state to this machine?

UNIX

nestat -an | egrep WAIT | awk ‘{print $4}’

5a

Windows (Powershell)

netstat -an | select-string “WAIT” | %{$_ -replace ‘\s+’,’ ‘ } | %{$i=$_.Split(” “);$i[3]}

image


6. What are the IP address (no ports) of the remote machines that are in some sort of WAIT state to this machine?

UNIX

netstat -an | egrep WAIT | awk ‘{print($4,a,”:”);print a[1]}’

6a

Windows (Powershell)

netstat -an | select-string “WAIT” | %{$_ -replace ‘\s+’,’ ‘ } | %{$i=$_.Split(” “);$i[3]} | %{$i=$_.Split(“:”);$i[0]}

image


7. What are the unique IP address of remote connections in some sort of WAIT state?

UNIX

netstat -an | egrep  WAIT | awk ‘{split($4,a,”:”);print a[1]}’ | sort -u

7a

Windows (Powershell)

netstat -an | select-string “WAIT” | %{$_ -replace ‘\s+’,’ ‘ } | %{$i=$_.Split(” “);$i[3]} | %{$i=$_.Split(“:”);$i[0]} | Sort-Object | Get-Unique

image