Tag Archives: dd

ftp to dev null to test bandwidth

Posted on by
Reply

When testing bandwidth, and troubleshooting bottlenecks, I prefer to use iperf. If you insist on testing bandwidth with FTP, it’s important NOT to use regular files. If you transfer actual files, the transfer could be limited by disk i/o, due to reads and writes. To eliminate this you can FTP from /dev/zero to /dev/null. It sounds super easy, but you have to use FTP in a special way to get it to read and write to special devices.

Here’s a little script. Be sure to replace the destination IP address, username and password with actual values:

# cat ftp_dev_null.sh
#!/bin/bash
/usr/bin/ftp -n <IP address of machine> <<END
verbose on
user <usernanme> <password>
bin
put "|dd if=/dev/zero bs=32k" /dev/null
bye
END

# ftp_dev_null.sh
Verbose mode on.
331 Password required for fordodone
230 User fordodone logged in
Remote system type is UNIX.
Using binary mode to transfer files.
200 Type set to I
local: |dd if=/dev/zero bs=32k remote: /dev/null
200 PORT command successful
150 Opening BINARY mode data connection for /dev/null
^C
send aborted
waiting for remote to finish abort
129188+0 records in
129187+0 records out
4233199616 bytes (4.2 GB) copied, 145.851 s, 29.0 MB/s
226 Transfer complete
4233142272 bytes sent in 145.82 secs (28350.0 kB/s)
221 Goodbye.
#

In this case I was getting around 230Mbits per second (over an IPSec tunnel) between my client and the FTP server. Not too bad.

show progress for dd

Posted on by
Reply

By default dd is silent. It just copies whatever blocks you want from in to out. In order to see progress, send it a USR1 signal using kill.

Start a useless dd:

# dd if=/dev/zero of=/dev/null

In another terminal find the pid:

# ps aux | grep dd | grep -v grep
root      7784 90.5  0.0   2884   560 pts/9    R+   10:01   0:06 dd if /dev/zero of /dev/null
#
# kill -USR1 7784

The original window will now show this:

# dd if=/dev/zero of=/dev/null
14501614+0 records in
14501614+0 records out
7424826368 bytes (7.4 GB) copied, 16.2149 seconds, 458 MB/s

Then you can ctrl+c it to get the final output:

# dd if=/dev/zero of=/dev/null
14501614+0 records in
14501614+0 records out
7424826368 bytes (7.4 GB) copied, 16.2149 seconds, 458 MB/s
16888077+0 records in
16888076+0 records out
8646694912 bytes (8.6 GB) copied, 19.3507 seconds, 447 MB/s

This one liner will start your dd, then monitor it and output progress every 20 seconds. Once the dd is finished it will stop and give your shell back.

dd if=/dev/zero of=/dev/null & pid=$! && sleep 20s && while true; do i=`ps aux | awk '{print $2}' | grep ^$pid$`; if [ "${i:-a}" !=  "$pid" ]; then break; fi; kill -USR1 $pid; sleep 20s; done;

mount windows disk image in linux

Posted on by
Reply

I was recently converting a Windows installation from a physical desktop to a virtual machine. Because the installation was an OEM installation the P2V conversion failed the “hardware” check and the target VM was unregistered. The only way to fix was to do a repair install changing the installation from an OEM to a Retail version of Windows. This needed to be done on the physical source desktop. There’s no way I was going to attempt this without a total backup of the system. I could have just copied important files, but I decided a disk clone was more appropriate. I booted off of a Debian rescue cd, mounted a NFS share, and dd’d a copy of the entire drive to a flat file.

# mkdir /mnt/nfsserver/diskclone
# mount nfsserver:/vol/diskclone /mnt/nfsserver/diskclone
# cd /mnt/nfsserver/diskclone
#
# dd if=/dev/sda of=desktop.img
156250000+0 records in
156250000+0 records out
80000000000 bytes (80 GB) copied, 7079.64601 seconds, 11.3 MB/s
#
# ls -l
total 78278496
-rw-r--r-- 1 root root 80000000000 2013-05-20 15:47 desktop.img

I then needed to mount the image and take a look at the contents. A flat disk image file is just a block for block copy of the actual disk, so the first step is to look at the partition table listed in sectors and find the offset. The offset will tell mount where the beginning of the file system is.

# losetup /dev/loop0 desktop.img
#
# fdisk -l -u  /dev/loop0

Disk /dev/loop0: 80.0 GB, 80000000000 bytes
255 heads, 63 sectors/track, 9726 cylinders, total 156250000 sectors
Units = sectors of 1 * 512 = 512 bytes 

      Device Boot      Start         End      Blocks   Id  System
/dev/loop0p1   *          63   156232124    78116031    7  HPFS/NTFS
#
#

Find the offset by multiplying the 63 (start sector) by 512 bytes ( 63*512 ) and the offset is 32256. Now we can tell mount where the start is. 

# mkdir /mnt/diskimage
# mount -t ntfs -o offset=32256,ro desktop.img /mnt/diskimage
#

I mounted readonly so that we don’t mess anything up. You may need to install ntfsprogs to be able to read an NTFS file system (apt-get install ntfsprogs) if you don’t have it installed already