In this article, I will show you guys how to combine the usage of these commands (grep, awk, sed, sort, uniq, seq) to run one single command line and generate tons of output, yet formatted into a very verbose but informative human readable format.
Assumptions
- We have a job server farm consisting of server names job7201 to job7955 (continuous consecutive)- These job servers run identical processes in production and serve one common purpose
- There is an administrative server called "adm4125" from which we run all commands
- Passwordless SSH has been setup so that you can do passwordless SSH from adm4125 to ALL job servers
Objectives
# Current Objective- Generate an inventory listing of all hardware, Serial Numbers, OS, CPU/RAM
# Future updates to this article will include
# - Perform remote health checks on the servers and their applications
# - Find out which server(s) participated in a certain job using a jobID
Preparations
Prepare the server names list[blogger@adm4125:~] $ seq 7201 7955
7201
7202
7203
....
7954
7955
Prepare Commands to Extract Data
We are using server job7201 as a test to extract sample data and observing from which, we can decide how to formulate our commands create the report
[blogger@adm4125:~] $ ssh job7201 "sudo /usr/sbin/dmidecode | grep -A6 'System Information'" # Get the hardware information
System Information
Manufacturer: HP
Product Name: ProLiant DL385 G2
Version: Not Specified
Serial Number: ************
UUID: 34313431-3039-5553-4537-31364E324248
Wake-up Type: Power Switch
[blogger@adm4125:~] $ ssh job7201 "uname -a" # Get the kernel version
Linux job7201.sentiblue.com 2.6.18-128.el5 #1 SMP Wed Dec 17 11:42:39 EST 2008 i686 i686 i386 GNU/Linux[blogger@adm4125:~] $ ssh job7201 "cat /etc/redhat-release" # Get the OS version
Red Hat Enterprise Linux Server release 5.3 (Tikanga)
[blogger@adm4125:~] $ MEM=`ssh job7201 "head -1 /proc/meminfo" | awk '{ print $2 }'` # Get the memory amount (in KB)
[blogger@adm4125:~] $ MEM=$(($MEM/1024/1024)) # Calculate out to GB
[blogger@adm4125:~] $ echo ${MEM}GB
125GB
[blogger@adm4125:~] $ ssh job7201 "grep -c Processor /proc/cpuinfo" # Count CPU Cores
48
Generate Hardware/OS/Resource Inventory Report
$ for SEQ in `seq 7201 7955 | xargs`; do> SERVER="job$SEQ"
> echo -n "$SERVER: "
> ssh $SERVER "sudo /usr/sbin/dmidecode | grep -A6 'System Info'
> | grep -e Manufacturer -e 'Product Name' -e Serial"
> done
job7201: Manufacturer: HP
Product Name: ProLiant DL385 G2
Serial Number: SF349827
job7202: Manufacturer: HP
Product Name: ProLiant DL360 G5
Serial Number: F39ERIU2
...
...
job7955: Manufacturer: HP
Product Name: ProLiant DL585 G7
Serial Number: 2DF20D8R
That doesn't look pretty! Let's fine tune the command some more;
- We want to get rid of the headers like "Manufacturer: ", "Product Name: ", "Serial Number: "
- We also want all data pertaining to each server to show in one single line
The colors of each fine tune will match below between objective and syntax.
$ for SEQ in `seq 7201 7955 | xargs`; do
> SERVER="job$SEQ"
> echo -n "$SERVER: "
> ssh $SERVER "sudo /usr/sbin/dmidecode | grep -A6 'System Info'
> | grep -e Manufacturer -e 'Product Name' -e Serial"
> | sed 's/ Manufacturer: //g'
> | sed 's/ Product Name: //g'
> | sed 's/ Serial Number: //g'
> | xargs
> done
job7201: HP ProLiant DL385 G2 SF349827
job7202: HP ProLiant DL360 G5 F39ERIU2
...
...
job7955: HP ProLiant DL585 G7 2DF20D8R
MUCH MUCH MUCH BETTER!
But so far we've only gathered Manufacturer, Model and Serial Numbers. Let's get some additional information; like the OS version and kernel architecture.
$ for SEQ in `seq 7201 7955 | xargs`; do
> SERVER="job$SEQ"
> echo -n "$SERVER: "
> ssh $SERVER "sudo /usr/sbin/dmidecode | grep -A6 'System Info'
> | grep -e Manufacturer -e 'Product Name' -e Serial"
> | sed 's/ Manufacturer: //g'
> | sed 's/ Product Name: //g'
> | sed 's/ Serial Number: //g'
> | xargs
> ( cat /etc/redhat-release | awk '{ print \$NF }'; uname -i ) | xargs
> done
job7201: HP ProLiant DL385 G2 SF349827 (Tikanga) x86_64
job7202: HP ProLiant DL360 G5 F39ERIU2 (Tikanga) x86_64
...
...
job7955: HP ProLiant DL585 G7 2DF20D8R (Tikanga) x86_64
There was more to the Original Objectives. For now I will end the article here leaving Objective #1 hanging a little loose.
I'd like those who read this article to fine tune the above command to include the CPU/RAM information into the report so that it looks like this
job7201: HP ProLiant DL385 G2 SF349827 (Tikanga) x86_64 24 Cores 196GB
job7202: HP ProLiant DL360 G5 F39ERIU2 (Tikanga) x86_64 48 Cores 64GB
...
...
job7955: HP ProLiant DL585 G7 2DF20D8R (Tikanga) x86_64 96 Cores 128GB
Don't let me down!!
This article will be revised in the future to meet Objectives #2 and #3
- Perform remote health checks on the servers and their applications
- Find out which server(s) participated in a certain job using a jobID
No comments:
Post a Comment
Help a friend, share your knowledge