rsync on different ssh port

By default ssh runs on TCP port 22. If you have a ssh configured to listen on a non standard port, you may need a special option to make rsync connect to that port. I was writing a quick backup script for this worpress site, and ran into this issue. In my case I was trying to rsync to a remote server with ssh listening on 4590. You have to give rsync a special ssh option:

# rsync -a --rsh='ssh -p 4590' /srv/www/wp-uploads/ backupsite.com:/backups/wp/wp-uplodas

find recently modified files

To find the 10 newest items in your home directory you can just use ls.

# cd 
# ls -lt | head
total 1197116
-rw-r--r--  1 fordodone     fordodone        5353 2013-04-23 10:42 file1
-rw-r--r--  1 fordodone     fordodone        2945 2013-04-23 10:21 file2
drwxr-xr-x  2 fordodone     fordodone       12288 2013-04-12 08:53 bin
-rw-r--r--  1 fordodone fordodone         0 2013-03-27 08:45 file3
-rw-------  1 fordodone fordodone     90420 2013-03-23 09:03 file4
-rw-r--r--  1 fordodone     fordodone          83 2013-03-19 10:35 file5
-rw-r--r--  1 fordodone     fordodone        8683 2013-03-15 10:26 file6
-rw-r--r--  1 fordodone     fordodone       28628 2013-03-15 09:15 file7
-rw-r--r--  1 fordodone     fordodone       81303 2013-03-15 09:15 file8

You could even get more aggressive by throwing the recursive flag in. Simple, right? But what if you need to recurse a large number of files and directories on a storage system, say 123,000 directories and 37 million files. I think find might be the way to go.

This will find files modified in the last 24 hours:

# find . -type f -mtime -1 -ls

find doesn’t really provide granular control of searching for files with a certain modified time. If you just want to find files that have been modified today (i.e. since 12am) we can use the -newer flag. First touch a temporary file with a timestamp to compare to files you want to find. In this case we make a date string of 04240000, or today at 00:00, and touch a file with that timestamp. Then use find to find files that are newer than the timestamp of the file you just touched.


# touch -t `date +%m%d0000` /tmp/compare
# find . -type f -newer /tmp/compare
(long output)
# rm /tmp/compare

troubleshoot UDP connectivity

When troubleshooting network connectivity issues the best method is to begin with the lower layers of the OSI model and work your way up until you eliminate the problem. It’s similar to the way some people troubleshoot helpdesk issues, where the first question is always “is it plugged in?”

Suppose you have 2 hosts, and you are trying to do an snmp query from one to the other. It doesn’t work.

# snmpwalk -v2c -c public 10.210.0.142 
Timeout: No Response from 10.210.0.142

You try pinging one from the other and you get replies. At this point you know that everything is working through layer 3, but something is not right with the transport layer. If you were troubleshooting a TCP based application, you could just use ol’ telnet to test if the port will open up. To see if a webserver is running you could try to telnet to port 80. UDP is a connectionless protocol, and can’t be tested like TCP. Here is where you can use nmap to see if the UDP port in question is open. In this case we know that by default snmpd listens on UDP 161.

# nmap -sU -v -p161 10.210.0.142

Starting Nmap 5.00 ( http://nmap.org ) at 2013-04-24 11:27 MST
NSE: Loaded 0 scripts for scanning.
Initiating ARP Ping Scan at 11:27
Scanning 10.210.0.142 [1 port]
Completed ARP Ping Scan at 11:27, 0.03s elapsed (1 total hosts)
Initiating UDP Scan at 11:27
Scanning example.fordodone.com (10.210.0.142) [1 port]
Completed UDP Scan at 11:27, 0.07s elapsed (1 total ports)
Host example.fordodone.com (10.210.0.142) is up (0.00100s latency).
Interesting ports on example.fordodone.com (10.210.0.142):
PORT    STATE  SERVICE
161/udp closed snmp
MAC Address: 00:0C:29:A2:BE:4D (VMware)

Read data files from: /usr/share/nmap
Nmap done: 1 IP address (1 host up) scanned in 0.43 seconds
           Raw packets sent: 2 (70B) | Rcvd: 2 (98B)

Well, there you go. The port is closed. No wonder our queries were timing out. Remove the -p161 part to just scan for all open UDP ports. Also, if your host doesn’t allow icmp, you can use -P0 to just skip host discovery, and assume it’s up. Depending whether they are open, filtered, or closed, you can start to diagnose what’s going on, i.e. snmpd crashed, firewall is blocking it, etc.

print last 3 characters of a string

If you need to get the last few characters of a string, you can just use awk.  This works great because you don’t have to know how long or short the string is to begin with. It first gets the length of the string. So in this example the length is 9 characters, and the ‘1’ would be in position 9 of the string. Then it subtracts 2 from position 9, to get position 7 (which is the ‘3’), then it gets 3 characters starting from position 7: ‘371’

# echo server371 | awk '{print substr($0,length($0)-2,3)}'
371

# echo anotherserver892 | awk '{print substr($0,length($0)-2,3)}'
892