Archive

Posts Tagged ‘garland joseph’

w2wcap: circular trace, stop when event detected

June 13th, 2017 No comments

This script is part of a series of scripts that perform packet capture between two endpoints.  In this case, the endpoints are two windows machines.  This script was tested with the “source endpoint” as a Windows 10 machine and the “target endpoint” a Windows 2016 Server machine.   The circular traces are started on each machine and stopped whenever an event is detected.  In this case the event is to monitor a file for a particular string.

Requirements: Wireshark installed on Windows
Caveats: Target endpoint must be Windows Server (Source endpoint can be windows client).


<#
################################################################################################

.SYNOPSIS
Author: Garland R.Joseph, garland.joseph@gmail.com, July 2016

Simple Packet capture from windows to windows based on monitoring a file.

.DESCRIPTION

Side-effects
Uses an interim x.x and local_trace.ps1 file, but removes them at end.

.EXAMPLE

w2wcap [ -verbose | -v ] [ -c "capture-file" ] [ -s "seconds" ] [ -u "remote user" ] [ -p "password" ]

-r "remote host" -l "log file" -t "text string" -a "local interface" -b "remote interface"

.PARAMETER v | verbose
Optional.  This option prints additional informational messages during processing.

.PARAMETER c
Optional. This options specifies the name of the capture file.  The default is set 
to capture.  The script is set to only create 2 files per endpoint and the default
size is 512 MB. So the 2 files total will be less than a 1 GB.  You can modify this
behavior by changing the FILESIZE and FILECOUNT variables below.  The filesname will
look similar to...
capture_00049_20170612214134
capture_00050_20170612215042

.PARAMETER s
Optional. Range is 1 second to 18000 seconds (5 hours) for parsing the log file. The default
is 5 seconds. Please examine the process to gauge impact on system resources.
The lower the number, the more system resources are consumed.

.PARAMETER u
Optional. This is the username for the remote user on the target endpoint. You can hard 
code the value in the script (see the $u=Wireshark line below) or 
optionally specify the name on the command line.  The account should have 
sufficient privilege to run the tshark command tool for the Wireshark facility. 
This parameter is optional, but recommended so you don't store the username in a file.

.PARAMETER p
Optional.  This is the password for the remote user.  This parameter is optional, but
recommended so you don't store the password in a file.

.PARAMETER r
Required.  This is the hostname or ip address of the target endpoint.

.PARAMETER l
Required.  This is the log file that will be parsed for the string as specified
by parameter t.  The traces will stop once a new string specified by parameter t
is found in the log file.  The script parses for an initial count so the log file
does not have to be "zeroed-out".

.PARAMETER t
Required.  This the text string used in parsing the log file.

.PARAMETER a
Required.  This is the ip address of the local interface on the source
endpoint that "talks" to the target endpoint.

.PARAMETER b
Required.  This is the ip address of the remote interface on the target
endpoint.
###############################################################################################
#>

#
# Setup Command Line Options
#

[CmdletBinding()]

Param (

[string]$c = "capture",

[ValidateRange(1,18000)]
[int]$s = 5,

[string]$u = "Wireshark",
[string]$p = "Pr1nceH1ll",


[Parameter(Mandatory=$True)]
[string]$r,

[Parameter(Mandatory=$True)]
[string]$l,

[Parameter(Mandatory=$True)]
[string]$t,

[Parameter(Mandatory=$True)]
[int]$a,

[Parameter(Mandatory=$False)]
[int]$b
)

If (-Not (Test-Path $l ) ) {
Write-Host "Log file $l does not exist"
exit
}

#
# Default options, can be changed to increase capture file size, count, etc
#


#$FILESIZE=1000 #units or kB, so this means 1 Meg
$FILESIZE=500000 #512 Meg
#$FILESIZE=1000000 #units or kB, so this means 1 Gig

$FILECOUNT=2 #creates a count of FILECOUNT of trace files at most of size FILESIZE

$TSHARK_LOCATION_REMOTE="c:\progra~1\wireshark\tshark"
$TSHARK_LOCATION_LOCAL="c:\progra~1\wireshark\tshark"
$TRACECMD_REMOTE="$TSHARK_LOCATION_REMOTE -b filesize:$FILESIZE -b files:$FILECOUNT -w $c -i $b"
$TRACECMD_LOCAL="$TSHARK_LOCATION_REMOTE -b filesize:$FILESIZE -b files:$FILECOUNT -w $c -i $a"

#
# Set up script block in order to evaluate parameters for remote command
# We use an interim file for asynchronously running the local trace
#

$sb_remote = {
param ($p1,$p2,$p3,$p4,$p5,$p6)
winrs /r:$p1 /u:$p2 /p:$p3 $p4
}

$TRACECMD_LOCAL &gt; ./local_trace.ps1

#
# Start trace on remote host, then on local host
#

Write-Verbose "Starting remote trace with ${TRACECMD_REMOTE}"
Write-Verbose "(winrs /r:$r /u:$u /p:$p $TRACECMD_REMOTE)"
Start-Job -Scriptblock $sb_remote -ArgumentList $r,$u,$p,$TRACECMD_REMOTE

Write-Verbose "Starting local trace with ${TRACECMD_LOCAL}"
Start-Process powershell.exe -ArgumentList "-file ./local_trace.ps1"


#
# Monitor log file
#
#
Write-Verbose "Start of monitoring file $l ever $s seconds for string $t..."
#init counters
Select-string -path "$l" -pattern $t | Measure-object -line | ft -hidetableheaders &gt; x.x ; $old_count = cat x.x | where {$_ -ne ""} | %{$_ -replace '\s+','' }
$new_count=$old_count
#loop until match found
$i=0
while ($new_count -eq $old_count) {
$i++
write-verbose "Iteration: $i, sleeping $s seconds..."
start-sleep -s $s
Write-verbose "Searching $l for $t..."
Select-string -path "$l" -pattern $t | Measure-object -line | ft -hidetableheaders &gt; x.x ; $new_count = cat x.x | where {$_ -ne ""} | %{$_ -replace '\s+','' }
}

#
# At this point, search string has been found, stop traces
#

#remote
taskkill /f /s $r /u $u /p $p /fi "imagename eq tshark*"

#local
taskkill /f /fi "imagename eq tshark*"

Write-Verbose "Traces completed after $i iterations. Examine the set of $FILECOUNT files on each endpoint with file name: $l on both endpoints."

#
# Clean up
#

if (Test-Path "./x.x") { rm "./x.x" }
if (Test-Path "./local_trace.ps1") { rm ./local_trace.ps1 }

exit

 

 

 

 

u2ucap: circular trace, stop when event detected

May 1st, 2017 Comments off

This script is part of a series of scripts that perform packet capture between two endpoints.  In this case the endpoints are two UNIX/LINUX machines.  The script was tested with the “source endpoint” as a Redhat machine and a “target endpoint” as an Ubuntu machine.  The circular traces are started on each machine and stopped whenever an event is detected.  In this case the event is to monitor a file for a particular string.

Requirements: tcpdump

# Author: Garland R. Joseph, garland.joseph@gmail.com
#   Date: May 2017
#
# This script is offered as is.  It is designed to
# run a circular trace using tcpdump on UNIX systems.
#
# You will either have to manually enter the password
# for the root account on the remote system or setup
# ssh keys from prompt-less access.
#
# Default file size is 2 x 1 GB per endpoint.
#
# The traces will stop once a key string SEARCH_STRING is
# found in LOG_FILE.
#
# Note:   Some UNIX systems line LINUX Fedora will
#         result in permsission denied when using
#         tcpdump  -W and -C options and writing to / or /root.
#
#         Tested on Source endpoint Redhat and Target endpoint Ubuntu.
# -----

#
# Defaults
#

USAGE="u2ucap [-v]  [ -c capture_file ] [ -w secs ] -h remote_host -l log_file -s search_string"
DEBUG=false
CAPTURE_FILE="/tmp/capture"
SLEEP_TIME="5"   #seconds
SEARCH_STRING=""
LOG_FILE=""
REMOTE_HOST=""
TCPDUMPCMD="tcpdump -C 1 -W 2 -w ${CAPTURE_FILE}"

#
# Process command line arguments
#

while getopts ":vc:w:l:s:h:" opts
do
 case ${opts} in
   v) DEBUG=true ;;
   c) CAPTURE_FILE=${OPTARG} ;;
   w) SLEEP_TIME=${OPTARG} ;;
   s) SEARCH_STRING=${OPTARG}  ;;
   l) LOG_FILE=${OPTARG} ;;
   h) REMOTE_HOST=${OPTARG} ;;
   ":") echo "Please specify a value for ${OPTARG}" ; exit ;;
  \?) echo "${OPTARG} is not a valid switch" ; echo "${USAGE}" ; exit;;
 esac
done

#
# Insure required values have been specified, check for existence of
# log file, getops should handle case of no values for -l and -s.
# A sanity check in the event getopts varies per unix
#

if [[ -z ${SEARCH_STRING} || -z ${LOG_FILE} || -z ${REMOTE_HOST} ]]
then
 echo ${USAGE}
 exit
fi

if ! [[ -f ${LOG_FILE} ]]
then
 echo "File ${LOG_FILE} does not exist"
 exit
fi

#
# Start trace on remote host
#

ssh root@${REMOTE_HOST} ${TCPDUMPCMD} 2>/dev/null 1>/dev/null &
REMOTE_PID=`ssh root@${REMOTE_HOST} ps -aef | egrep tcpdump | egrep -v grep | awk '{print $2}'`
${DEBUG} && echo "${0}-I-REMOTE_PID, remote pid is ${REMOTE_PID}."
#
# Start trace on this host
#
#${TCPDUMPCMD} 2>/dev/null 1>/dev/null &
exec ${TCPDUMPCMD} 2>/dev/null 1>/dev/null &
LOCAL_PID=$!
${DEBUG} && echo "${0}-I-LOCAL_PID, local pid is ${LOCAL_PID}."

#
# Monitor log file
#

old_count=`grep -c ${SEARCH_STRING} ${LOG_FILE}`
(( new_count=old_count ))
(( i = 0 ))
while (( old_count == new_count ))
do
   (( i++ ))
   ${DEBUG} && echo "${0}-F-SLEEP, sleeping ${SLEEP_TIME}, iteration ${i}."
   sleep ${SLEEP_TIME}
   new_count=`grep -c ${SEARCH_STRING} ${LOG_FILE}`
done

#
#  At this point, search string has been found, stop traces
#

###${DEBUG} && set -x
kill ${LOCAL_PID}
ssh root@${REMOTE_HOST} kill ${REMOTE_PID}
####${DEBUG} && set +x

#
# Verbose only reminders
#

####${DEBUG} && echo "Consult files ${CAPTURE_FILE} on local and remote host."

exit

Windows SNMP

October 25th, 2016 Comments off

Utilities

FilenameDescription
eventcmd.execommand line tool to configure the SNMP event translator
eventwin.exeapplication / user interface for configuring SNMP event translator
mibcc.exeSNMP MIB Compiler
snmputil.exeSample SNMP Manager console application

Essential  SNMP Files

FilenameDescription
dhcpmib.dllDCHP MIB. Only used on DHCP servers
eventagnt.dllSNMP Event Translator. Translates event logs into SNMP traps.
hostmib.dllImplements the Host Resources MIB
lmmib2.dllImplements LAN Manger MIB-II
mgmtapi.dllMicrosoft SNMP manager library. API allows snmp manager applications to listen for snmp manager requests, send requests, and receive responses from SNMP agents.
mib.bincompiled MIB information for mgmtapi.dll
snmp.exeA Service. The master agent that receives SNMP requests and delivers to the appropirate DLL.
snmpapi.dllsnmp utilities used by snmp extension agents and manager applications. Contains framework for developing extension agent DLLs.
snmpsnap.dllSNMP trap services. Receives SNMP traps and forwards them to SNMP managers
winsmib.dllMicrosoft-defined WINS mib. Installed only on WINS servers.
wsnmp32.dllThis API allows SNMP manager applications to "listen" for SNMP manager requests, and send requests to and receive responses from SNMP agents.

Related Links

Configure SNMP

Categories: Networking Tags: , ,

Example Unix and Windows/Powershell Network Socket Table Commands

October 7th, 2016 Comments off

1. What tcp sockets are open?

Unix
netstat -an | egrep -I “^tcp.*LISTEN”
netstat -an | egrep -I “^tcp.*LISTEN” | awk ‘{split($4,a,”:”);print a[2]}’

Powershell
netstat -an | select-string “LISTEN”
netstat -an | select-string “LISTEN” | %{$i=%{$_.Line.Split(“:”)};$j=$i[1] -replace ‘\s+’,’ ‘;$k=$j.Split(” “);$k[0]} | where {$_ -ne “”}

2. What TCP ports are in an ESTABLISHED/LISTEN state

Unix
netstat -an | egrep ESTABLISH
netstat -an | egrep ESTABLISH | awk ‘{split($4,a,”:”);print a[2]}’

Powershell
netstat -an | select-string “ESTABLISHED”
netstat -an | select-string “ESTABLISHED” | %{$i=%{$_.Line.Split(“:”)};$j=$i[1] -replace ‘\s+’,’ ‘;$k=$j.Split(” “);$k[0]} | where {$_ -ne “”}

3. What connections are in some sort of Wait State?

Unix
netstat -an | egrep WAIT
Powershell
netstat -an | select-string “WAIT”

4. How many connections are in some sort of WAIT state?

Unix

netstat –an | egrep WAIT | wc –l
Powershell
netstat -an | select-string “WAIT” | where {$_ -ne “”} | Measure-Object –Line

5. What are the IP address and ports of the remote machines that are in some sort of WAIT state to this machine?

Unix

nestat -an | egrep WAIT | awk ‘{print $4}’
Powershell
netstat -an | select-string “WAIT” | %{$_ -replace ‘\s+’,’ ‘ } | %{$i=$_.Split(” “);$i[3]}

6. What are the IP address (no ports) of the remote machines that are in some sort of WAIT state to this machine?

Unix

netstat -an | egrep WAIT | awk ‘{print($4,a,”:”);print a[1]}’
Powershell
netstat -an | select-string “WAIT” | %{$_ -replace ‘\s+’,’ ‘ } | %{$i=$_.Split(” “);$i[3]} | %{$i=$_.Split(“:”);$i[0]}

7. What are the unique IP address of remote connections in some sort of WAIT state?

Unix
netstat -an | egrep WAIT | awk ‘{split($4,a,”:”);print a[1]}’ | sort -u
Powershell
netstat -an | select-string “WAIT” | %{$_ -replace ‘\s+’,’ ‘ } | %{$i=$_.Split(” “);$i[3]} | %{$i=$_.Split(“:”);$i[0]} | Sort-Object | Get-Unique

Network Socket Table

June 2nd, 2016 No comments

 

Preface

The purpose of this document is to give a list of commands that can be executed from the command line during networking or system problem determination. These commands focus on parsing the network socket table (namely: netstat -a). The socket table is arguably a common denominator for any system (regardless of operating system) that participates in the TCP/IP protocol.

Both UNIX/LINUX and Windows (Powershell) commands are given. It is important to understand the context of the command before entering it; more so than constructing the command line syntax. Once you understand the context, you can simply cut and paste the command.

The ability to quickly parse the socket table allows the administrator to be responsive during times of crisis.

What follows is a scenario that shows:

· the value of the quick-use of the socket table

· insight into the left-side and the right-side of the table

· a summary of the context, and finally

· a list of commands in both UNIX/LINUX and Powershell.

Click Here for and Index of UNIX and Powershell Commands


Scenario

It was my first day on the job. I just moved from Austin, Texas to Greensboro, North Carolina to work for a large financial institution. I was fresh from a stable IBM support environment. I was excited to tackle real world issues again.

The environment was a distributed computing environment featuring different operating systems: OS400, AIX, Linux and Windows. A few dedicated AIX systems were also used as part of the network infrastructure… different from the traditional roles for AIX systems: like DB2, Oracle, WebSphere, etc. This was before F5/BigIP devices…. dedicated to load balancing, Data Power devices that bridge multi-level protocols at wire speed, or Cisco’s defining support of vlans in VMware environments. In other words, using AIX or LINUX operating systems to perform functions that have now migrated to “network devices”.

There were 2-twin AIX systems that were used as proxy servers (IBM-B50, 2U-rack mounted units). The network engineer said they were load-balanced.

So…I logged into both machines to for look and after a few minutes I said “well, I don’t think they are load balanced”. The network engineer asked, “what do you mean?”. I said, “well, I only see 3 connections on this partner, and about 1000 connections on the other partner.” These load-balanced proxy servers had been running for about two years.

I simply logged on to both systems and did

netstat -an | egrep <PROXY-SERVER-PORT> | wc -l

Same company, about a week later…this time there was a distributed application hanging on the AIX side. I traced the problem down to a connection to an OS400 machine (via the socket table). I asked the Application team to check the application processes on the OS400 system and they told me that the OS400 machine was not relevant. It wasn’t until I was in a room with the application folks explaining the socket table that they finally looked at the OS400 and found the issue. I pointed to the IP address of the OS400 appearing in the socket table.

So, I love the socket table. It’s one of the first areas I consider whenever I walk into a new environment. It’s a system administrator’s quick insight into applications in a distributed environment. Any host system that participates in the TCP/IP protocol is going to have this table.


The Left-side versus the Right-Side

I think of the socket table as having a left side and a right side. I frequently “slice” (using the Unix cut or awk command) the table into two and then parse the output. The left side will show the local IP addresses of the system (as well as the loop back). The right side shows the IP addresses of machines connecting to this machine.

The following diagram shows a search for ESTABLISHED connections…

image

The following shows “slicing” the table to evaluate remote connection information.

image

In this case our local host has 10 connections to port 5201 on host 192.168.1.101.   I ran the iperf tool in server mode on host 192.168.1.101 and started the client on the local host requesting 10 connections (the iperf tool is used to test network bandwidth).   We also have one connection to port 22 on 192.168.1.101.  Port 22 is the ssh server. This means I have ssh’d into machine 192.168.1.101 from the local host.

 

Context

I always parse the socket table to get a sense of what a server is doing anytime I am called into any system issue, especially when I am looking at a server for the first time. If it’s an issue with an application that works over the network, I always track down the port associated with the application. Here is a partial list of things I look for when parsing the socket table and the context:

Item Context
Ports in LISTEN state This gives an idea of how many server types of applications on the host.
Ports not in LISTEN state This gives an idea of current connection from remote hosts.
Ports in some sort of CLOSE state for a long time. Repetitive use of this command and counting them will show if some connections are hanging in close state.
Ports in SYN state for a long time Repetitive use of this command and counting them will show if some connections are hanging in SYN state.
Ports in WAIT states for a long time Repetitive use of this command and counting them will show if some connections are hanging in WAIT state.
Number of Ports accessing an application Server Port Repetitive use of this command and filtering about a specific port will show the application server load network-wise.
Number of unique hosts connecting to this server Cutting the table in half (vertically, column-wise) and looking at the right side of the socket table reveals this information.  It helps to pipe this output to sort and a count tool.
A list of connections that don’t have 0 in send and receive queues (UNIX only). Repetitive use of this command and observing if the send/receive queues are non-zero can indicate an application is not processing.

 

The Commands


1. What tcp sockets are open?

UNIX

netstat -an | egrep -I “^tcp.*LISTEN”

1a

netstat -an | egrep -I “^tcp.*LISTEN” | awk ‘{split($4,a,”:”);print a[2]}’

1b


Windows (Powershell)

netstat -an | select-string “LISTEN”

image

netstat -an | select-string “LISTEN” | %{$i=%{$_.Line.Split(“:”)};$j=$i[1] -replace ‘\s+’,’ ‘;$k=$j.Split(” “);$k[0]} | where {$_ -ne “”}

image


2. What TCP ports are in an ESTABLISHED state

UNIX

netstat -an | egrep ESTABLISH

2a

netstat -an | egrep ESTABLISH | awk ‘{split($4,a,”:”);print a[2]}’

2b

Windows (Powershell)

netstat -an | select-string “ESTABLISHED” | %{$i=%{$_.Line.Split(“:”)};$j=$i[1] -replace ‘\s+’,’ ‘;$k=$j.Split(” “);$k[0]} | where {$_ -ne “”}

image


3. What TCP ports are in some sort of Wait State?

UNIX

nestat –an | egrep WAIT

3a

 

Windows (Powershell)

netstat -an | select-string “WAIT”

image


4. How many connections are in some sort of WAIT state?

UNIX

netstat -an | egrep WAIT | wc -l

4a

Windows (Powershell)

netstat -an | select-string “WAIT” | where {$_ -ne “”} | Measure-Object –Line

image


5. What are the IP address and ports of the remote machines that are in some sort of WAIT state to this machine?

UNIX

nestat -an | egrep WAIT | awk ‘{print $4}’

5a

Windows (Powershell)

netstat -an | select-string “WAIT” | %{$_ -replace ‘\s+’,’ ‘ } | %{$i=$_.Split(” “);$i[3]}

image


6. What are the IP address (no ports) of the remote machines that are in some sort of WAIT state to this machine?

UNIX

netstat -an | egrep WAIT | awk ‘{print($4,a,”:”);print a[1]}’

6a

Windows (Powershell)

netstat -an | select-string “WAIT” | %{$_ -replace ‘\s+’,’ ‘ } | %{$i=$_.Split(” “);$i[3]} | %{$i=$_.Split(“:”);$i[0]}

image


7. What are the unique IP address of remote connections in some sort of WAIT state?

UNIX

netstat -an | egrep  WAIT | awk ‘{split($4,a,”:”);print a[1]}’ | sort -u

7a

Windows (Powershell)

netstat -an | select-string “WAIT” | %{$_ -replace ‘\s+’,’ ‘ } | %{$i=$_.Split(” “);$i[3]} | %{$i=$_.Split(“:”);$i[0]} | Sort-Object | Get-Unique

image