use netstat to monitor receive queue Recv-Q

Posted on by
Reply 
# i=0; while true; do i=$(($i+1)); echo $i ==============================; netstat -natlp | grep ^tcp | sort -nk1 | awk '{ if($2 != 0) {print}}' ; sleep 1;  done;
1 ==============================
2 ==============================
3 ==============================
4 ==============================
5 ==============================
tcp      100      0 10.0.3.167:22           198.21.8.23:53477       ESTABLISHED 99304/sshd: fordodone
6 ==============================
7 ==============================
8 ==============================
9 ==============================
tcp    43520      0 10.0.3.167:53877        10.0.9.55:3306          ESTABLISHED 119789/mysqldump
10 ==============================
11 ==============================
12 ==============================
13 ==============================
14 ==============================
15 ==============================
16 ==============================
tcp6       1      0 10.0.3.167:80           198.21.8.23:65114       CLOSE_WAIT  3880/apache2    
17 ==============================
18 ==============================

use tee to send output to STDOUT and a pipe

Posted on by
Reply

Generally, tee is used to send output from a command to both STDOUT and to a file. You can also use tee to send output to STDOUT and pass the output on to another command like you would normally using a pipe. This uses a process substitution syntax.

~$ echo "some error message" | tee >(logger -t test_error_msg)
some error message
~$ grep test_error_msg /var/log/syslog
Feb 24 11:37:54 foohost test_error_msg: some error message
~$

The error message is output to STDOUT and also sent to the logger command (which writes it to syslog). This can be usefull when you have silent cron jobs (>/dev/null 2>&1) that log errors to syslog, but at the same time if you run the shell script by hand you can see the error output.

start screen session with 4-way split screen

Posted on by
Reply

There are several terminals that allow splitting the screen to accommodate multiple regions in a single window. When logged into a non-desktop/server environment Linux screen is great for this. It has support for splitting the screen vertically and/or horizontally. You can use ctrl + a + S or ctrl + a + | to split regions horizontally or vertically. Here’s an excerpt from a .screenrc file to split the screen into 4 regions, and start ssh sessions to four separate servers in each of those regions.


split
split -v
focus down
split -v

screen -t bash /bin/bash
screen -t deploy1 /usr/bin/ssh deploy1
screen -t deploy2 /usr/bin/ssh deploy2
screen -t deploy3 /usr/bin/ssh deploy3
screen -t deploy4 /usr/bin/ssh deploy4

focus up
focus left
select 1
focus right
select 2
focus left
focus down
select 3
focus right
select 4

Now I can start the screen session…

# screen -c .screenrc-multiwindow

and automatically get this:

4-way-screen-split

convert dmesg timestamp to date time

Posted on by
3

If you have messages in dmesg that log the event time as “seconds since last boot”, it can be difficult to tell when they happened. Here’s an example of one of these messages:

# tail -1 /var/log/dmesg
[8969653.483175] poorcoding.php[14798]: segfault at 7f2efca36ed0 ip 00007f2efca36ed0 sp 00007f2efaf0be98 error 14
#

You could use something like this to parse out the timestamp and convert it to a date:

# date --date=@$((`date +%s --date="\`who -b | awk '{print $3" "$4}'\`"` + `dmesg | tail -1 | awk '{print $1}' | sed -e 's/\[//g' -e 's/\..*//g'`))
Thu Nov  5 01:21:53 PDT 2015
#

If your application logs to kern.log, it uses timestamps as well as seconds since uptime, so the error will already have a timestamp on it

Aug 14 09:57:59 myhostname kernel: [8969653.483177] poorcoding.php[14800]: segfault at 7f2efca36ec8 ip 00007f2efca36ec8 sp 00007f2ef10c4e98 error 14

TODO: make a quick function; convert all the messages not just a tailed or greped one;