Archive

Posts Tagged ‘unix’

Auto-Discovery with NMAP

June 26th, 2017 Comments off

Discover

I created this script in order to use it as a feed for enterprise management tools such as Nagios.   It’s a bit difficult to get a handle on auto-discovery tools within an enterprise management tool when it discovers what can be an overwhelming number of hosts.  NMAP is smart enough to translate the MAC address into a vendor if it can.

This script called discover uses the NMAP tool on Linux (tested on Centos).


#! /bin/bash
#
# Name: discover
#
# Garland Joseph, garland.joseph@gmail.com
# Date: June 2017
#
# Auto-discover on subnet using nmap, can be fed into something like nagios
# as a seed file after proper formatting.
#
# ----

if [[ -z ${1} ]]
then
cat <<EOD
$0 <subnet>
where
subnet, by example, is something like 192.168.1.0/24
EOD
exit
fi

nmap -sn ${1} | awk '
BEGIN{ printf("%-16s| %-18s| %-35s| %-30s\n","IP","MAC","NAME","VENDOR") }
/^Nmap scan report/ {
NAME=$5
x=$NF
gsub("[()]","",x)
IP=x
}

/^MAC Address/ {
MAC=$3
split($0,a,"(")
split(a[2],b,")")
VENDOR=b[1]
printf("%-16s| %-18s| %-35s| %-30s\n",IP,MAC,NAME,VENDOR)
}'

Here is an example of the output

[root@localhost ~]# ./discover 192.168.1.0/24
IP            | MAC               | NAME                               | VENDOR
192.168.1.1   | C8:D7:19:DE:54:2E | NyaRaePrimary                      | Cisco Consumer Products
192.168.1.100 | B8:27:EB:72:2A:4A | kodi1.grandenetworks.net           | Raspberry Pi Foundation
192.168.1.101 | 00:1F:3B:75:7F:EB | 192.168.1.101                      | Intel Corporate
192.168.1.102 | 6C:3B:E5:76:96:A5 | HP-Printer.grandenetworks.net      | Hewlett Packard
192.168.1.105 | A8:47:4A:AC:8F:89 | 192.168.1.105                      | Unknown
192.168.1.106 | F0:7D:68:0A:7C:8A | OllieMaeJoseph.grandenetworks.net  | D-Link
192.168.1.107 | 58:82:A8:81:C3:A6 | XboxOne                            | Unknown
192.168.1.109 | 28:56:5A:39:ED:FD | BRW28565A39EDFD.grandenetworks.net | Unknown
192.168.1.110 | 64:20:0C:90:24:D9 | Garlands-iPad.grandenetworks.net   | Apple
192.168.1.112 | 7C:D1:C3:17:0C:58 | Apple-TV.grandenetworks.net        | Apple
192.168.1.115 | B8:27:EB:12:DA:AC | kodi2.grandenetworks.net           | Raspberry Pi Foundation
192.168.1.118 | A4:77:33:8E:CE:C2 | Chromecast.grandenetworks.net      | Google
192.168.1.119 | 6C:AD:F8:5D:3A:D6 | 192.168.1.119                      | Azurewave Technologies
192.168.1.131 | F0:7D:68:0A:7A:D5 | EmmaEdwards.grandenetworks.net     | D-Link
192.168.1.135 | 28:10:7B:0C:3A:71 | EarlEdwards.grandenetworks.net     | D-Link International
192.168.1.136 | 00:09:B0:D6:A8:2A | 192.168.1.136                      | Onkyo
192.168.1.138 | 28:10:7B:0C:3A:74 | LeeJoseph.grandenetworks.net       | D-Link International
192.168.1.145 | D4:3D:7E:EF:93:99 | obama                              | Micro-Star Int'l Co
192.168.1.147 | E4:3E:D7:44:21:8F | LGwebOSTV.grandenetworks.net       | Unknown

Here is a Diagram of my home LAN as revealed by the Network Status Map in Nagios

Categories: Networking Tags: , , , , , ,

u2ucap: circular trace, stop when event detected

May 1st, 2017 Comments off

This script is part of a series of scripts that perform packet capture between two endpoints.  In this case the endpoints are two UNIX/LINUX machines.  The script was tested with the “source endpoint” as a Redhat machine and a “target endpoint” as an Ubuntu machine.  The circular traces are started on each machine and stopped whenever an event is detected.  In this case the event is to monitor a file for a particular string.

Requirements: tcpdump

# Author: Garland R. Joseph, garland.joseph@gmail.com
#   Date: May 2017
#
# This script is offered as is.  It is designed to
# run a circular trace using tcpdump on UNIX systems.
#
# You will either have to manually enter the password
# for the root account on the remote system or setup
# ssh keys from prompt-less access.
#
# Default file size is 2 x 1 GB per endpoint.
#
# The traces will stop once a key string SEARCH_STRING is
# found in LOG_FILE.
#
# Note:   Some UNIX systems line LINUX Fedora will
#         result in permsission denied when using
#         tcpdump  -W and -C options and writing to / or /root.
#
#         Tested on Source endpoint Redhat and Target endpoint Ubuntu.
# -----

#
# Defaults
#

USAGE="u2ucap [-v]  [ -c capture_file ] [ -w secs ] -h remote_host -l log_file -s search_string"
DEBUG=false
CAPTURE_FILE="/tmp/capture"
SLEEP_TIME="5"   #seconds
SEARCH_STRING=""
LOG_FILE=""
REMOTE_HOST=""
TCPDUMPCMD="tcpdump -C 1 -W 2 -w ${CAPTURE_FILE}"

#
# Process command line arguments
#

while getopts ":vc:w:l:s:h:" opts
do
 case ${opts} in
   v) DEBUG=true ;;
   c) CAPTURE_FILE=${OPTARG} ;;
   w) SLEEP_TIME=${OPTARG} ;;
   s) SEARCH_STRING=${OPTARG}  ;;
   l) LOG_FILE=${OPTARG} ;;
   h) REMOTE_HOST=${OPTARG} ;;
   ":") echo "Please specify a value for ${OPTARG}" ; exit ;;
  \?) echo "${OPTARG} is not a valid switch" ; echo "${USAGE}" ; exit;;
 esac
done

#
# Insure required values have been specified, check for existence of
# log file, getops should handle case of no values for -l and -s.
# A sanity check in the event getopts varies per unix
#

if [[ -z ${SEARCH_STRING} || -z ${LOG_FILE} || -z ${REMOTE_HOST} ]]
then
 echo ${USAGE}
 exit
fi

if ! [[ -f ${LOG_FILE} ]]
then
 echo "File ${LOG_FILE} does not exist"
 exit
fi

#
# Start trace on remote host
#

ssh root@${REMOTE_HOST} ${TCPDUMPCMD} 2>/dev/null 1>/dev/null &
REMOTE_PID=`ssh root@${REMOTE_HOST} ps -aef | egrep tcpdump | egrep -v grep | awk '{print $2}'`
${DEBUG} && echo "${0}-I-REMOTE_PID, remote pid is ${REMOTE_PID}."
#
# Start trace on this host
#
#${TCPDUMPCMD} 2>/dev/null 1>/dev/null &
exec ${TCPDUMPCMD} 2>/dev/null 1>/dev/null &
LOCAL_PID=$!
${DEBUG} && echo "${0}-I-LOCAL_PID, local pid is ${LOCAL_PID}."

#
# Monitor log file
#

old_count=`grep -c ${SEARCH_STRING} ${LOG_FILE}`
(( new_count=old_count ))
(( i = 0 ))
while (( old_count == new_count ))
do
   (( i++ ))
   ${DEBUG} && echo "${0}-F-SLEEP, sleeping ${SLEEP_TIME}, iteration ${i}."
   sleep ${SLEEP_TIME}
   new_count=`grep -c ${SEARCH_STRING} ${LOG_FILE}`
done

#
#  At this point, search string has been found, stop traces
#

###${DEBUG} && set -x
kill ${LOCAL_PID}
ssh root@${REMOTE_HOST} kill ${REMOTE_PID}
####${DEBUG} && set +x

#
# Verbose only reminders
#

####${DEBUG} && echo "Consult files ${CAPTURE_FILE} on local and remote host."

exit

Example Unix and Windows/Powershell Network Socket Table Commands

October 7th, 2016 Comments off

1. What tcp sockets are open?

Unix
netstat -an | egrep -I “^tcp.*LISTEN”
netstat -an | egrep -I “^tcp.*LISTEN” | awk ‘{split($4,a,”:”);print a[2]}’

Powershell
netstat -an | select-string “LISTEN”
netstat -an | select-string “LISTEN” | %{$i=%{$_.Line.Split(“:”)};$j=$i[1] -replace ‘\s+’,’ ‘;$k=$j.Split(” “);$k[0]} | where {$_ -ne “”}

2. What TCP ports are in an ESTABLISHED/LISTEN state

Unix
netstat -an | egrep ESTABLISH
netstat -an | egrep ESTABLISH | awk ‘{split($4,a,”:”);print a[2]}’

Powershell
netstat -an | select-string “ESTABLISHED”
netstat -an | select-string “ESTABLISHED” | %{$i=%{$_.Line.Split(“:”)};$j=$i[1] -replace ‘\s+’,’ ‘;$k=$j.Split(” “);$k[0]} | where {$_ -ne “”}

3. What connections are in some sort of Wait State?

Unix
netstat -an | egrep WAIT
Powershell
netstat -an | select-string “WAIT”

4. How many connections are in some sort of WAIT state?

Unix

netstat –an | egrep WAIT | wc –l
Powershell
netstat -an | select-string “WAIT” | where {$_ -ne “”} | Measure-Object –Line

5. What are the IP address and ports of the remote machines that are in some sort of WAIT state to this machine?

Unix

nestat -an | egrep WAIT | awk ‘{print $4}’
Powershell
netstat -an | select-string “WAIT” | %{$_ -replace ‘\s+’,’ ‘ } | %{$i=$_.Split(” “);$i[3]}

6. What are the IP address (no ports) of the remote machines that are in some sort of WAIT state to this machine?

Unix

netstat -an | egrep WAIT | awk ‘{print($4,a,”:”);print a[1]}’
Powershell
netstat -an | select-string “WAIT” | %{$_ -replace ‘\s+’,’ ‘ } | %{$i=$_.Split(” “);$i[3]} | %{$i=$_.Split(“:”);$i[0]}

7. What are the unique IP address of remote connections in some sort of WAIT state?

Unix
netstat -an | egrep WAIT | awk ‘{split($4,a,”:”);print a[1]}’ | sort -u
Powershell
netstat -an | select-string “WAIT” | %{$_ -replace ‘\s+’,’ ‘ } | %{$i=$_.Split(” “);$i[3]} | %{$i=$_.Split(“:”);$i[0]} | Sort-Object | Get-Unique

Network Socket Table

June 2nd, 2016 No comments

 

Preface

The purpose of this document is to give a list of commands that can be executed from the command line during networking or system problem determination. These commands focus on parsing the network socket table (namely: netstat -a). The socket table is arguably a common denominator for any system (regardless of operating system) that participates in the TCP/IP protocol.

Both UNIX/LINUX and Windows (Powershell) commands are given. It is important to understand the context of the command before entering it; more so than constructing the command line syntax. Once you understand the context, you can simply cut and paste the command.

The ability to quickly parse the socket table allows the administrator to be responsive during times of crisis.

What follows is a scenario that shows:

· the value of the quick-use of the socket table

· insight into the left-side and the right-side of the table

· a summary of the context, and finally

· a list of commands in both UNIX/LINUX and Powershell.

Click Here for and Index of UNIX and Powershell Commands


Scenario

It was my first day on the job. I just moved from Austin, Texas to Greensboro, North Carolina to work for a large financial institution. I was fresh from a stable IBM support environment. I was excited to tackle real world issues again.

The environment was a distributed computing environment featuring different operating systems: OS400, AIX, Linux and Windows. A few dedicated AIX systems were also used as part of the network infrastructure… different from the traditional roles for AIX systems: like DB2, Oracle, WebSphere, etc. This was before F5/BigIP devices…. dedicated to load balancing, Data Power devices that bridge multi-level protocols at wire speed, or Cisco’s defining support of vlans in VMware environments. In other words, using AIX or LINUX operating systems to perform functions that have now migrated to “network devices”.

There were 2-twin AIX systems that were used as proxy servers (IBM-B50, 2U-rack mounted units). The network engineer said they were load-balanced.

So…I logged into both machines to for look and after a few minutes I said “well, I don’t think they are load balanced”. The network engineer asked, “what do you mean?”. I said, “well, I only see 3 connections on this partner, and about 1000 connections on the other partner.” These load-balanced proxy servers had been running for about two years.

I simply logged on to both systems and did

netstat -an | egrep <PROXY-SERVER-PORT> | wc -l

Same company, about a week later…this time there was a distributed application hanging on the AIX side. I traced the problem down to a connection to an OS400 machine (via the socket table). I asked the Application team to check the application processes on the OS400 system and they told me that the OS400 machine was not relevant. It wasn’t until I was in a room with the application folks explaining the socket table that they finally looked at the OS400 and found the issue. I pointed to the IP address of the OS400 appearing in the socket table.

So, I love the socket table. It’s one of the first areas I consider whenever I walk into a new environment. It’s a system administrator’s quick insight into applications in a distributed environment. Any host system that participates in the TCP/IP protocol is going to have this table.


The Left-side versus the Right-Side

I think of the socket table as having a left side and a right side. I frequently “slice” (using the Unix cut or awk command) the table into two and then parse the output. The left side will show the local IP addresses of the system (as well as the loop back). The right side shows the IP addresses of machines connecting to this machine.

The following diagram shows a search for ESTABLISHED connections…

image

The following shows “slicing” the table to evaluate remote connection information.

image

In this case our local host has 10 connections to port 5201 on host 192.168.1.101.   I ran the iperf tool in server mode on host 192.168.1.101 and started the client on the local host requesting 10 connections (the iperf tool is used to test network bandwidth).   We also have one connection to port 22 on 192.168.1.101.  Port 22 is the ssh server. This means I have ssh’d into machine 192.168.1.101 from the local host.

 

Context

I always parse the socket table to get a sense of what a server is doing anytime I am called into any system issue, especially when I am looking at a server for the first time. If it’s an issue with an application that works over the network, I always track down the port associated with the application. Here is a partial list of things I look for when parsing the socket table and the context:

Item Context
Ports in LISTEN state This gives an idea of how many server types of applications on the host.
Ports not in LISTEN state This gives an idea of current connection from remote hosts.
Ports in some sort of CLOSE state for a long time. Repetitive use of this command and counting them will show if some connections are hanging in close state.
Ports in SYN state for a long time Repetitive use of this command and counting them will show if some connections are hanging in SYN state.
Ports in WAIT states for a long time Repetitive use of this command and counting them will show if some connections are hanging in WAIT state.
Number of Ports accessing an application Server Port Repetitive use of this command and filtering about a specific port will show the application server load network-wise.
Number of unique hosts connecting to this server Cutting the table in half (vertically, column-wise) and looking at the right side of the socket table reveals this information.  It helps to pipe this output to sort and a count tool.
A list of connections that don’t have 0 in send and receive queues (UNIX only). Repetitive use of this command and observing if the send/receive queues are non-zero can indicate an application is not processing.

 

The Commands


1. What tcp sockets are open?

UNIX

netstat -an | egrep -I “^tcp.*LISTEN”

1a

netstat -an | egrep -I “^tcp.*LISTEN” | awk ‘{split($4,a,”:”);print a[2]}’

1b


Windows (Powershell)

netstat -an | select-string “LISTEN”

image

netstat -an | select-string “LISTEN” | %{$i=%{$_.Line.Split(“:”)};$j=$i[1] -replace ‘\s+’,’ ‘;$k=$j.Split(” “);$k[0]} | where {$_ -ne “”}

image


2. What TCP ports are in an ESTABLISHED state

UNIX

netstat -an | egrep ESTABLISH

2a

netstat -an | egrep ESTABLISH | awk ‘{split($4,a,”:”);print a[2]}’

2b

Windows (Powershell)

netstat -an | select-string “ESTABLISHED” | %{$i=%{$_.Line.Split(“:”)};$j=$i[1] -replace ‘\s+’,’ ‘;$k=$j.Split(” “);$k[0]} | where {$_ -ne “”}

image


3. What TCP ports are in some sort of Wait State?

UNIX

nestat –an | egrep WAIT

3a

 

Windows (Powershell)

netstat -an | select-string “WAIT”

image


4. How many connections are in some sort of WAIT state?

UNIX

netstat -an | egrep WAIT | wc -l

4a

Windows (Powershell)

netstat -an | select-string “WAIT” | where {$_ -ne “”} | Measure-Object –Line

image


5. What are the IP address and ports of the remote machines that are in some sort of WAIT state to this machine?

UNIX

nestat -an | egrep WAIT | awk ‘{print $4}’

5a

Windows (Powershell)

netstat -an | select-string “WAIT” | %{$_ -replace ‘\s+’,’ ‘ } | %{$i=$_.Split(” “);$i[3]}

image


6. What are the IP address (no ports) of the remote machines that are in some sort of WAIT state to this machine?

UNIX

netstat -an | egrep WAIT | awk ‘{print($4,a,”:”);print a[1]}’

6a

Windows (Powershell)

netstat -an | select-string “WAIT” | %{$_ -replace ‘\s+’,’ ‘ } | %{$i=$_.Split(” “);$i[3]} | %{$i=$_.Split(“:”);$i[0]}

image


7. What are the unique IP address of remote connections in some sort of WAIT state?

UNIX

netstat -an | egrep  WAIT | awk ‘{split($4,a,”:”);print a[1]}’ | sort -u

7a

Windows (Powershell)

netstat -an | select-string “WAIT” | %{$_ -replace ‘\s+’,’ ‘ } | %{$i=$_.Split(” “);$i[3]} | %{$i=$_.Split(“:”);$i[0]} | Sort-Object | Get-Unique

image