Author Archives: ForDoDone

SCP files from one host to another

Posted on by
Reply

Everyone knows how to copy files around using SCP, but it can be a pain if you have to enter passwords for every copy. If you have an administration host with shared ssh keys to every other host, you can just use a quick little one liner to drag files from hostA, through the admin box, over to hostB:

adminbox # ssh hostA "tar cf - /usr/local/sbin/myscript.sh 2>/dev/null" | ssh hostB "cd / && tar xvf - 1>/dev/null"

Using tar, the file is output to STDOUT and piped over ssh, then read from STDIN. It copies /usr/local/sbin/myscript.sh from hostA to hostB. Because the admin box has ssh keys to both hostA and hostB, the process is automatic and does not require password authentication. This means you can use this method in scripts for batch copies, etc. Also, you won’t have to create a temporary copy on the admin host.

Drop it into a simple shell script and it will be even easier:

#!/bin/bash
# FILE: file_dragger.sh
# AUTHOR: fordodone <fordodone at email.com>
# DATE: 2013/07/11
# NOTES: drags a file from one host to another
#

if [ $# -ne 3 ]
then
echo ""
echo "usage: </full/path/to/file> <src> <dst>"
echo ""
exit
fi

ssh $2 "tar cf - $1 2>/dev/null" | ssh $3 "cd / && tar xvf - 1>/dev/null"

To use it for the original copy example do this:

# file_dragger.sh /usr/local/sbin/myscript.sh hostA hostB
#

set NetApp administration hosts

Posted on by
Reply

When creating a volume on a NetApp system, if NFS is licensed an entry in /etc/exports will be created for the new volume. It adds the administration hosts (configured at setup) to have root access to the new volume. If you change admin hosts, or add new ones, you need to update /etc/exports to reflect the change, however, any subsequent volume creations will still be using the old admin hosts list. Use the hidden option ‘admin.hosts’ to see the current admin hosts:

toaster> options admin.hosts
admin.hosts                  10.14.33.141,10.14.22.141

Update the list:

toaster> options admin.hosts 10.14.33.141,172.16.11.23,192.168.1.3
toaster>
toaster> options admin.hosts
admin.hosts 10.14.33.141,172.16.11.23,192.168.1.3
toaster>

delete files with unrecognized characters

Posted on by
Reply

For whatever reason you may find some files with unrecognized or missencoded characters that need to be removed. Because the terminal doesn’t recognize the characters it’s difficult to do anything with them.

# ls -l
-rw-r--r-- 1 www-data www-data 14828193 Nov 26  2008 ?¡?ú?©?ç?}?¤?I?@áÁ?????�?ï?i?r?g?j?j [51] 2008.10.02 ?w?¼?e?Ɋm?F?µ?Ă݂܂·?I?x - HirataTalk +AB Quiz.wma
-rw-r--r-- 1 www-data www-data 14568695 Nov 26  2008 ?V?g?ƈ«???̎·???J?t?F?ւ悤?±?» [01] 2007.08.31 - ?ò?é?݂䂫.wma
-rw-r--r-- 1 www-data www-data 11898139 Nov 26  2008 ?V?g?ƈ«???̎·???J?t?F?ւ悤?±?» [02] 2007.09.07 - kukui.wma
-rw-r--r-- 1 www-data www-data 11642799 Nov 26  2008 ?V?g?ƈ«???̎·???J?t?F?ւ悤?±?» [03] 2007.09.14 - ?ێu???ê?N.wma
#

Use the -i flag with ls to obtain the inode number of the files:

# ls -li
6886578 -rw-r--r-- 1 www-data www-data 14828193 Nov 26  2008 ?¡?ú?©?ç?}?¤?I?@áÁ?????�?ï?i?r?g?j?j [51] 2008.10.02 ?w?¼?e?Ɋm?F?µ?Ă݂܂·?I?x - HirataTalk +AB Quiz.wma
6886580 -rw-r--r-- 1 www-data www-data 14568695 Nov 26  2008 ?V?g?ƈ«???̎·???J?t?F?ւ悤?±?» [01] 2007.08.31 - ?ò?é?݂䂫.wma
6886581 -rw-r--r-- 1 www-data www-data 11898139 Nov 26  2008 ?V?g?ƈ«???̎·???J?t?F?ւ悤?±?» [02] 2007.09.07 - kukui.wma
6886582 -rw-r--r-- 1 www-data www-data 11642799 Nov 26  2008 ?V?g?ƈ«???̎·???J?t?F?ւ悤?±?» [03] 2007.09.14 - ?ێu???ê?N.wma
#

Now use find with the -inum flag to find only the file with a specific inode number. Then delete it:

# find . -inum 6886578 -delete
#

check if a directory is a nfs mount

Posted on by
Reply

Use stat to display the file system status, following links, and output the format as the human readable file system type:

#stat -f -L -c %T /tmp
ext2/ext3
#
# stat -f -L -c %T /mnt/fileserver/volume
nfs
#

As suspected “/tmp” is a regular ext file system, but the path “/mnt/fileserver/volume” is an nfs mount. What happens if we unmount the nfs mount:

# umount /mnt/fileserver/volume
#
# stat -f -L -c %T /mnt/fileserver/volume
ext2/ext3
#

It reports properly that the directory is just an unmounted directory in a regular ext file system.

Alternatively you can use the mountpoint command:

# mountpoint /tmp
/tmp is not a mountpoint
#
# mountpoint /mnt/adfs40/vol1
/mnt/adfs40/vol1 is a mountpoint
#
# umount /mnt/fileserver/volume
# mountpoint /mnt/adfs40/vol1
/mnt/adfs40/vol1 is not a mountpoint
#

monitor host for connectivity

Posted on by
Reply

Sometimes, you want to be notified if a host goes up or down. Usually Nagios is perfect for this, but in this case I had an internet circuit, and all I cared about was knowing when the ISP deactivated it. Use ping in a loop, make 1 request every second, if ping doesn’t get a response, then send a text message (Verizon number) and stop the loop.

while true; do ping -nc 1 -W 1 5.6.7.8 | grep -q icmp; if [ "$?" == "1" ]; then echo "circuit is down" | mail <10-digit phone number no spaces>@vtext.com; break; fi; sleep 1s; done;

I also use the converse of this method when I want to know when a new circuit comes up.

test bandwidth throughput using iperf

Posted on by
Reply

Testing bandwidth between 2 hosts can be helpful to determine maximum transfer rates, and how much bandwidth can be pushed. The iperf utility is a great option for this. Set up one side as a listening server and the other side as a client. In this scenario I want to test an OpenVPN tunnel between a Raspberry Pi and a Vyatta server. The layout looks like this:

Laptop --(wireless)--> RPi --(wireless)--> Linksys router -> Enterasys router -> Cable Modem -> ISP -> INTERNET -> Vyatta -> remote host

Start iperf in server mode on the remote host listening for UDP datagrams:

# iperf -s -u
------------------------------------------------------------
Server listening on UDP port 5001
Receiving 1470 byte datagrams
UDP buffer size:   110 KByte (default)
------------------------------------------------------------

Start up the client. The -c flag tells it what remote host to connect to with iperf, the -u makes it use UDP, the -b tells it what bandwidth to try to achieve, and the -t sets how many seconds to run for:

# iperf -c 10.211.0.141 -u -b 1000K -t30

As the test runs, the server and client will output the rates. Here’s the server side output:

[  3] local 10.211.0.141 port 5001 connected with 10.211.0.6 port 40816
[ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total Datagrams
[  3]  0.0-30.0 sec  3.57 MBytes    998 Kbits/sec  14.738 ms    4/ 2552 (0.16%)

So we were able to push a full 1 Mbps and only lost 4 datagrams, that’s pretty good. I would say this can comfortably push 1Mbps. (Side note: top reported openvpn process on RPi was at 9% CPU utilization) Let’s try for 2Mbps:

# iperf -c 10.211.0.141 -u -b 2000K -t30

And the output:

[  4] local 10.211.0.141 port 5001 connected with 10.211.0.6 port 56707
[  4]  0.0-30.0 sec  6.21 MBytes  1.74 Mbits/sec  7.346 ms  670/ 5103 (13%)

We were able to achieve 1.74 Mbps, but we lost 13% of the traffic. That is unacceptable! I would run again at 1500K and narrow in where exactly the break point is for throughput. To find the bottleneck you can run iperf from every (Linux) host in the chain to pinpoint the limitation. I suspect it could be the upload speed of my home internet connection, or the poor wireless signal of the client side of the RPi.

change SSH listen address

Posted on by
Reply

If you have servers with internal and external interfaces, you may want to disable ssh on the external side. In this case we just get the internal IP address and tell sshd to only listen on that address:

sed -i "s/#ListenAddress 0.0.0.0/ListenAddress `grep address /etc/network/interfaces | grep 10.229 | awk '{print $2}'`/" /etc/ssh/sshd_config

Do it to many hosts:

for i in `seq 313 364`; do ssh ftp$i "sed -i \"s/#ListenAddress 0.0.0.0/ListenAddress \`grep address /etc/network/interfaces | grep 10.229 | awk '{print \$2}'\`/\" /etc/ssh/sshd_config"

And restart SSH:

for i in `seq 313 364`; do ssh ftp$i "service ssh restart"; done;

remove IPv6 address from interface

Posted on by
Reply

If you don’t want to have an IPv6 address on an interface, you can quickly remove it:

# ip addr show dev eth0 
2: eth0:  mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:c2:fc:59 brd ff:ff:ff:ff:ff:ff
    inet 10.210.0.141/16 brd 10.210.255.255 scope global eth0
    inet6 fe80::20c:29ff:fec2:fc59/64 scope link 
       valid_lft forever preferred_lft forever

Only get the address:

# ip addr show dev eth0 | grep inet6 | awk '{print $2}'
fe80::20c:29ff:fec2:fc59/64

Remove it:

# ip -6 addr del `ip addr show dev eth0 | grep inet6 | awk '{print $2}'` dev eth0

Or do it to many hosts:

for i in `seq 201 272`; do ssh www$i "ip -6 addr del \`ip addr show dev eth0 | grep inet6 | awk '{print \$2}'\` dev eth0"; done;