monitor Apache memory usage

When looking at a webserver for memory usage, it’s important to consider the VSZ and RSS memory usage.

This little one liner gets the Total and Average VSZ and RSS usage as well as thread count, and prints those statistics every 5 seconds:

# while true; do ps auxfww | grep apache | grep -v -e cronolog -e grep | awk '{ vsum+=$5; rsum+=$6 } END { print "VSZ:", vsum, "(", vsum/NR, ") RSS:", rsum, "(", rsum/NR, ") Procs:", NR }'; sleep 5; done;
VSZ: 9896272 ( 341251 ) RSS: 1716216 ( 59179.9 ) Procs: 29
VSZ: 9547608 ( 340986 ) RSS: 1650100 ( 58932.1 ) Procs: 28
VSZ: 9546328 ( 340940 ) RSS: 1649044 ( 58894.4 ) Procs: 28
VSZ: 9861976 ( 340068 ) RSS: 1687968 ( 58205.8 ) Procs: 29
VSZ: 9868632 ( 340298 ) RSS: 1694496 ( 58430.9 ) Procs: 29
VSZ: 9853272 ( 339768 ) RSS: 1679112 ( 57900.4 ) Procs: 29
VSZ: 9853272 ( 339768 ) RSS: 1679264 ( 57905.7 ) Procs: 29
^C
#

So there are around 29 threads running right now on this server. The threads are using an average of 340MB per thread VSZ, and 59MB per thread RSS. The total of around 1.7GB of RSS looks good, on a machine with 8G physical memory.

get current client IP addresses from web farm

To see what common IPs are connecting to your web farm, ssh to all of the servers and get a list of clients. Then sort it until you see most busy clients.

# for i in `seq 401 436`; do ssh www$i "netstat -natp | grep EST | grep apa | grep ":80 "| awk '{print \$5}' | cut -d : -f1"; done | sort | uniq -c | sort -nk1 | tail
      3 10.0.0.1
      3 10.0.0.10
      3 10.245.34.2
      4 10.29.45.89
      5 10.111.111.111
      5 10.239.234.234
      5 10.1.1.1
      5 10.2.2.2
      6 10.3.3.3
     10 10.100.100.100
#

The list shows the number of connections, and the client IP.

generate SSL Certificate Signing Request

First generate a new 2048 bit key:

# openssl genrsa -out fordodone.com.key 2048

I choose not to encrypt the key, because when reloading 240 apache servers, I don’t want to have to enter the passphrase each time. Now we use our key to generate a Certificat Signing Request.

# openssl req -new -key fordodone.com.key -out fordodone.com.csr

To inspect the CSR:

openssl req -in fordodone.com.csr -noout -text

Now you can sign your own cert, or upload/paste the CSR to a 3rd party provider to issue an official SSL Certificate.