use netstat to monitor receive queue Recv-Q

# i=0; while true; do i=$(($i+1)); echo $i ==============================; netstat -natlp | grep ^tcp | sort -nk1 | awk '{ if($2 != 0) {print}}' ; sleep 1;  done;
1 ==============================
2 ==============================
3 ==============================
4 ==============================
5 ==============================
tcp      100      0 10.0.3.167:22           198.21.8.23:53477       ESTABLISHED 99304/sshd: fordodone
6 ==============================
7 ==============================
8 ==============================
9 ==============================
tcp    43520      0 10.0.3.167:53877        10.0.9.55:3306          ESTABLISHED 119789/mysqldump
10 ==============================
11 ==============================
12 ==============================
13 ==============================
14 ==============================
15 ==============================
16 ==============================
tcp6       1      0 10.0.3.167:80           198.21.8.23:65114       CLOSE_WAIT  3880/apache2    
17 ==============================
18 ==============================
share:

convert dmesg timestamp to date time

If you have messages in dmesg that log the event time as “seconds since last boot”, it can be difficult to tell when they happened. Here’s an example of one of these messages:

# tail -1 /var/log/dmesg
[8969653.483175] poorcoding.php[14798]: segfault at 7f2efca36ed0 ip 00007f2efca36ed0 sp 00007f2efaf0be98 error 14
#

You could use something like this to parse out the timestamp and convert it to a date:

# date --date=@$((`date +%s --date="\`who -b | awk '{print $3" "$4}'\`"` + `dmesg | tail -1 | awk '{print $1}' | sed -e 's/\[//g' -e 's/\..*//g'`))
Thu Nov  5 01:21:53 PDT 2015
#

If your application logs to kern.log, it uses timestamps as well as seconds since uptime, so the error will already have a timestamp on it

Aug 14 09:57:59 myhostname kernel: [8969653.483177] poorcoding.php[14800]: segfault at 7f2efca36ec8 ip 00007f2efca36ec8 sp 00007f2ef10c4e98 error 14

TODO: make a quick function; convert all the messages not just a tailed or greped one;

share:

docker get list of tags in repository

The native docker command has an excellent way to search the docker hub repository for an image. Just use docker search <search string> to look in their registry.

# docker search debian
NAME                          DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
ubuntu                        Ubuntu is a Debian-based Linux operating s...   2338      [OK]       
debian                        Debian is a Linux distribution that's comp...   763       [OK]       
google/debian                                                                 47                   [OK]
neurodebian                   NeuroDebian provides neuroscience research...   12        [OK]       
jesselang/debian-vagrant      Stock Debian Images made Vagrant-friendly ...   4                    [OK]
eboraas/debian                Debian base images, for all currently-avai...   3                    [OK]
armbuild/debian               ARMHF port of debian                            3                    [OK]
mschuerig/debian-subsonic     Subsonic 5.1 on Debian/wheezy.                  3                    [OK]
fike/debian-postgresql        PostgreSQL 9.4 until 9.0 version running D...   2                    [OK]
maxexcloo/debian              Docker base image built on Debian with Sup...   1                    [OK]
kalabox/debian                                                                1                    [OK]
takeshi81/debian-wheezy-php   Debian wheezy based PHP repo.                   1                    [OK]
webhippie/debian              Docker images for debian                        1                    [OK]
eeacms/debian                 Docker image for Debian to be used with EE...   1                    [OK]
reinblau/debian               Debian with usefully default packages for ...   1                    [OK]
mariorez/debian               Debian Containers for PHP Projects              0                    [OK]
opennsm/debian                Lightly modified Debian images for OpenNSM      0                    [OK]
konstruktoid/debian           Debian base image                               0                    [OK]
visono/debian                 Docker base image of debian 7 with tools i...   0                    [OK]
nimmis/debian                 This is different version of Debian with a...   0                    [OK]
pl31/debian                   Basic debian image                              0                    [OK]
idcu/debian                   mini debian os                                  0                    [OK]
sassmann/debian-chromium      Chromium browser based on debian                0                    [OK]
sassmann/debian-firefox       Firefox browser based on debian                 0                    [OK]
cloudrunnerio/debian                                                          0                    [OK]

We can see the official debian repository right at the top. Unfortunately there’s no way to see what tags and images are available for us to pull down and deploy. However, there is a way to query the registry for all the tags in a repository, returned in JSON format. You can use a higher level programming language to get the list and parse the JSON for you. Or you can just use a simple one-liner:

# wget -q https://registry.hub.docker.com/v1/repositories/debian/tags -O -  | sed -e 's/[][]//g' -e 's/"//g' -e 's/ //g' | tr '}' '\n'  | awk -F: '{print $3}'
latest
6
6.0
6.0.10
6.0.8
6.0.9
7
7.3
7.4
7.5
7.6
7.7
7.8
7.9
8
8.0
8.1
8.2
experimental
jessie
jessie-backports
oldstable
oldstable-backports
rc-buggy
sid
squeeze
stable
stable-backports
stretch
testing
unstable
wheezy
wheezy-backports

Wrap that in a little bash script and you have an easy way to list the tags of a repository. Since a tag is just a pointer to a image commit multiple tags can point to the same image. Get fancy:

# wget -q https://registry.hub.docker.com/v1/repositories/debian/tags -O -  | sed -e 's/[][]//g' -e 's/"//g' -e 's/ //g' | tr '}' '\n' | sed -e 's/^,//' | sort -t: -k2 | awk -F[:,] 'BEGIN {i="image";j="tags"}{if(i!=$2){print i" : "j; i=$2;j=$4}else{j=$4" | "j} }END{print i" : "j}'
image : tags
06af7ad6 : 7.5
19de96c1 : wheezy | 7.9 | 7
1aa59f81 : experimental
20096d5a : rc-buggy
315baabd : stable
37cbf6c3 : testing
47921512 : 7.7
4a5e6db8 : 8.1
4fbc238a : oldstable-backports
52cb7765 : wheezy-backports
84bd6e50 : unstable
88dc7f13 : jessie-backports
8c00acfb : latest | jessie | 8.2 | 8
91238ddc : stretch
b2477d24 : stable-backports
b5fe16f2 : 7.3
bbe78c1a : 7.8
bd4b66c4 : oldstable
c952ddeb : squeeze | 6.0.10 | 6.0 | 6
d56191e1 : 6.0.8
df2a0347 : 8.0
e565fbbc : 7.4
e7d52d7d : sid
feb75584 : 7.6
fee2ea4e : 6.0.9

share:

add auto statements for interface aliases to /etc/network/interfaces

someone added a bunch of iface statements for configuration but forgot the auto part…

# sed -i 's/iface eth0:\([0-9]\{3\}\)/auto eth0:\1\niface eth0:\1/' /etc/network/interfaces
<snip>
auto eth0:196
iface eth0:196 inet static
  address 1.2.3.4
  netmask 255.255.255.0
auto eth0:197
iface eth0:197 inet static
  address 1.2.3.5
  netmask 255.255.255.0
auto eth0:198
iface eth0:198 inet static
  address 1.2.3.6
  netmask 255.255.255.0
</snip>
share:

tcpdump mysql queries

If you have access to the MySQL server and logging is turned on then you have access to the queries as they are logged. Many production databases do not have logging turned on, simply because there are too many queries to handle. Also, there could be hundreds of servers hitting the logs at any given time, making it hard to see activity from a particular client. To take a look at MySQL queries as they leave a webserver you can use tcpdump and massage the output to get you what queries are being sent from that host.

# tcpdump -i eth0 -l -s 0 -w - dst port 3306 | stdbuf -o0 strings| stdbuf -o0 grep "SELECT\|INSERT\|UPDATE|\FROM\|WHERE\|ORDER\|AND\|LIMIT\|FROM\|SET\|COMMIT\|ROLLBACK"

Sometimes the query gets broken up into pieces if WHERE or LIMIT is used, and those pieces wind up on separate lines so we need to grep for them separately. Use stdbuf to force all the pipes to NOT buffer output, i.e. print output in pseudo real time.

share:

count character occurrence rates in filenames

find all the files in a directory. Take out the first dot . put in by find. Remove slashes (can’t be a character in a filename). Use fold -w 1 (–width) the width option limits column output to 1 character, which puts each character on it’s own line. Don’t count spaces (we don’t care about them). Sort the output, count how many occurrences of each character happened. Sort output by least to most occurrences of characters.

find . -type f | sed -e 's/\.//' -e 's/\// /g' | fold -w 1 | grep -v '^ $' | sort | uniq -c | sort -nk1
      1 '
      7 ^
     22 ,
     29 (
     29 )
     40 #
     51 =
     72 ~
    214 @
    312 :
    672 Y
   1141 +
   1217 J
   1497 Z
   2813 G
   3696 U
   3727 H
   5168 O
   5654 N
   5700 X
   5721 K
  10185 R
  10590 W
  11414 F
  12412 A
  13114 E
  13424 C
  13904 z
  15369 Q
  15698 j
  18746 I
  20582 S
  30232 M
  39547 q
  44301 B
  44946 P
  54675 7
  74749 9
  74777 L
  78077 T
  83720 8
  86739 D
  87151 4
  92824 k
  93168 y
  94261 5
  96495 w
 105734 V
 135527 6
 193306 f
 215943 0
 239003 g
 274810 3
 284082 v
 291777 1
 305769 h
 329499 _
 353852 2
 397075 b
 493086 m
 513388 p
 523439 d
 539160 x
 654812 -
 697485 l
 717868 a
 728134 n
 843460 t
 862742 u
 883640 .
1059771 i
1060749 c
1109991 o
1227620 r
1326244 s
1440326 e
share: