Tag Archives: cut

monitor NetApp SnapMirror transfer speed

Posted on by
Reply

You may want to monitor the speed of a current snapmirror to get an idea of how fast the transfer is going. The speed might change throughout the day due to load, or disk bottleneck, etc. I started with this one-liner:

i=0;j=0; while true; do j=$i; i=`ssh toaster01 "snapmirror status -l volname" | grep Progress | awk '{print $2}'| cut -d \( -f2`; if [ $j -eq 0 ]; then sleep 1m; continue; fi; echo -n "$i-$j = ";echo "scale=2;($i-$j)/(1024*1024)" | bc | tr '\n' ' '; echo "GB/min"; sleep 1m; done;

Which lead to this short script:

#!/bin/bash
# FILE: netapp_sm_monitor.sh
# AUTHOR: For Do Done <fordodone@fordodone.com>
# DATE: 2014/03/26
# NOTES: 
# 

if [ $# -lt 2 ]
then
  echo ""
  echo "usage: netapp_sm_monitor.sh <filer> <srcvol> [-v]"
  echo ""
  exit
fi

i=0;
j=0; 

while true; 
do 
  j=$i; 
  i=`ssh $1 "snapmirror status -l $2" | grep Progress | awk '{print $2}'| cut -d \( -f2`; 
  if [ $j -eq 0 ]; 
    then 
    sleep 1m; 
    continue; 
  fi; 
  if [ "$3" == "-v" ]
  then
    echo -n "$i-$j = ";
  fi
  echo "scale=2;($i-$j)/(1024*1024)" | bc | tr '\n' ' '; echo "GB/min"; 
  sleep 1m; 
done;

awk multiple input field separators

Posted on by
Reply

We want just the last directory in the tree from this list:

# tail -3 list
/mnt/fs92/vol3/users/520/520/1680520 -- 8631
/mnt/fs92/vol3/users/568/8568/1578568 -- 2
/mnt/fs92/vol3/users/429/7429/1757429 -- 2

You can use both cut and awk, or awk twice…
weak:

# tail -3 list | cut -d / -f 8 | awk '{print $1}'
1680520
1578568
1757429

Or you can just tell awk to use multiple field separators. They are bound by square brackets, and in this case we can use both / and a space to be the input field separators, making the 8th column the one we are interested in.

strong:

# tail -3 list | awk -F "[/ ]" '{print $8}'
1680520
1578568
1757429

find music directories

Posted on by
Reply

I was recently handed an old Windows laptop, and told “It is broken so I know you can put it to use, and if you get my music off of it that would be awesome.” Right away I knew I had a great chance of recovering everything from the hard drive.

I took the hard drive out of the laptop and plugged it into my workstation via a SATA to USB converter. It showed right up and I mounted the partition that I thought would be the windows partition:

# ls
autoexec.bat  config.sys  doctemp                 found.000  found.003     MSOCache     pagefile.sys  ProgramData       Program Files              Users
Boot          DELL        Documents and Settings  found.001  hiberfil.sys  newfile.enc  pending.un    ProgramData.LOG1  $Recycle.Bin               Windows
bootmgr       dell.sdr    Drivers                 found.002  Intel         newkey       PerfLogs      ProgramData.LOG2  System Volume Information

Well, that looks familiar. Then went into the person’s user directory and did this:

# find . -type f -name '*.m4a' -o -name '*.mp3' -ls > ~/music_file_list

I could have been more thorough and looked for more file extensions (acc,m4u, etc.), but I figured iTunes would just put every music file in the same folder. The resultant file looked like this:

# tail music_file_list
 82837 9632 -rw-------   2 fordodone fordodone  9861791 Sep 16  2008 ./Users/laptop/Music/iTunes/Alanis\ Morissette\ -\ Jagged\ Little\ Pi\ 12.mp3
   307 27696 -rw-------   2 fordodone fordodone 28357414 Oct 22  2008 ./Users/laptop/Searches/Documents/3L\ First\ Semester/Energy/dem\ now.mp3
 53814 4856 -rw-------   2 fordodone fordodone  4972361 Feb 17  2007 ./Users/laptop/Searches/Documents/Old\ Computer/My\ Music/01\ Bouncing\ Around\ The\ Room.mp3
 53817 6116 -rw-------   2 fordodone fordodone  6259086 Feb 17  2007 ./Users/laptop/Searches/Documents/Old\ Computer/My\ Music/01\ Come\ Together.mp3
 53834 8132 -rw-------   2 fordodone fordodone  8325962 Feb 17  2007 ./Users/laptop/Searches/Documents/Old\ Computer/My\ Music/01\ Funky\ Bitch.mp3
 53962 31512 -rw-------   2 fordodone fordodone 32266213 Dec 21  2004 ./Users/laptop/Searches/Documents/Old\ Computer/My\ Music/01\ Inflate-_Barnacles.mp3
 53975 4424 -rw-------   2 fordodone fordodone  4527885 Feb 17  2007 ./Users/laptop/Searches/Documents/Old\ Computer/My\ Music/01\ Julius.mp3
 53979 12288 -rw-------   2 fordodone fordodone 12579091 Apr  1  2002 ./Users/laptop/Searches/Documents/Old\ Computer/My\ Music/01\ Mike's\ Song.mp3
 54019 8476 -rw-------   2 fordodone fordodone  8677963 Mar 31  2002 ./Users/laptop/Searches/Documents/Old\ Computer/My\ Music/01\ Vultures.mp3
 54028 6004 -rw-------   2 fordodone fordodone  6146289 Feb 17  2007 ./Users/laptop/Searches/Documents/Old\ Computer/My\ Music/01\ Wilson.mp3

Now the goal was to get a list of unique directories in which music could be found. I would then take that list and rsync those directories to a local hard drive. Since the music files could be located at any unpredictable level in the tree, and I only wanted the directory listing I did this:

# cat music_file_list | cut -d / -f 2- | rev | cut -d / -f 2- | rev | sort | uniq -c
    926 Users/laptop/Music/iTunes
     27 Users/laptop/Music/iTunes/iTunes\ Music/Podcasts/GreenBiz\ Radio
     10 Users/laptop/Music/iTunes/iTunes\ Music/Podcasts/NPR_\ Planet\ Money\ Podcast
     51 Users/laptop/Music/iTunes/iTunes\ Music/Podcasts/This\ American\ Life
      4 Users/laptop/Music/iTunes/iTunes\ Music/Podcasts/WNYC's\ Radiolab
      2 Users/laptop/Music/iTunes/iTunes\ Music/Smeal\ College\ of\ Business/Wall\ Street\ Bootcamp\ Series
      1 Users/laptop/Searches/Documents/3L\ First\ Semester/Energy
      8 Users/laptop/Searches/Documents/Old\ Computer/My\ Music

That gave me the list I was looking for and how many mp3 and m4a files were in each unique directory. I’ll probably skip the Podcasts, and just recover the rest. It looks like this will be about 30G of files, so I will probably use adrive.com to upload and share this amount of data.

TODO: revisit this exercise with awk.

Linux rebuild software RAID1

Posted on by
Reply

Set the bash field separator to newline:

IFS="
"

See what disk and partitions are currently up, then generate commands to re-add the missing disk and partitions, then run them:

for i in `cat /proc/mdstat | grep md | cut -d [ -f1 | sed -e 's/\(md[0-9]\).*\(sd[a-z][0-9]\)/mdadm --add \/dev\/\1 \/dev\/\2/' | sed -e 's/sdb/sda/'`;  do eval $i; done;

TODO: make it determine which disk to add (/dev/sda or /dev/sdb)