Monthly Archives: May 2013

chkconfig –list for Debian using sysv-rc-conf

Posted on by
Reply

In Redhat based distros you can use chkconfig to see what daemons are supposed to run at what run levels. In Debian you can use the sysv-rc-conf package.

Install it:

# apt-get install --yes sysv-rc-conf

Run it:



# sysv-rc-conf

┌ SysV Runlevel Config   -: stop service  =/+: start service  h: help  q: quit ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│                                                                                                                                                                                                                                           │
│ service      1       2       3       4       5       0       6       S                                                                                                                                                                    │
│ ----------------------------------------------------------------------------                                                                                                                                                              │
│ acpid       [ ]     [X]     [X]     [X]     [X]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ bootlogd    [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ bootlogs    [X]     [X]     [X]     [X]     [X]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ console-s$  [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ cron        [ ]     [X]     [X]     [X]     [X]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ fancontrol  [ ]     [X]     [X]     [X]     [X]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ halt        [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ hdparm      [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ ifupdown    [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ ifupdown-$  [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ kbd         [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ keyboard-$  [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ killprocs   [X]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ lm-sensors  [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ lvm2        [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ mdadm       [ ]     [X]     [X]     [X]     [X]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ mdadm-raid  [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ module-in$  [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ mountover$  [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ multipath$  [ ]     [X]     [X]     [X]     [X]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ multipath$  [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ mysql       [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ ndbd        [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ networking  [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ nfs-common  [ ]     [X]     [X]     [X]     [X]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ ntp         [ ]     [X]     [X]     [X]     [X]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ open-iscsi  [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ portmap     [ ]     [X]     [X]     [X]     [X]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ postfix     [ ]     [X]     [X]     [X]     [X]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ procps      [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ rc.local    [ ]     [X]     [X]     [X]     [X]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ reboot      [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ rmnologin   [ ]     [X]     [X]     [X]     [X]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ rsyslog     [ ]     [X]     [X]     [X]     [X]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ sendsigs    [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ single      [X]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ snmpd       [ ]     [X]     [X]     [X]     [X]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ ssh         [ ]     [X]     [X]     [X]     [X]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ stop-boot$  [ ]     [X]     [X]     [X]     [X]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ stop-boot$  [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ udev        [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ udev-mtab   [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]                                                                                                                                                                   │
│ umountfs    [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ umountroot  [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]                                                                                                                                                                   │
│ urandom     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [ ]     [X]

This shows a list of Runlevel configuration for daemons. For most user land stuff, you will just want 2, 3, 4, and 5. You can edit the list, and changes will be made when you exit. This is a graphical alternative to update-rc.d

get memcache statistics

Posted on by
Reply

To obtain some stats about a memcached process, use nc to talk directly to it:


# echo "stats" | nc -w 1  11211
STAT pid 1750
STAT uptime 29481383
STAT time 1369781775
STAT version 1.4.5
STAT pointer_size 64
STAT rusage_user 6974.991909
STAT rusage_system 13000.624488
STAT curr_connections 10
STAT total_connections 132871296
STAT connection_structures 1674
STAT cmd_get 227296759
STAT cmd_set 113549712
STAT cmd_flush 0
STAT get_hits 221783239
STAT get_misses 5513520
STAT delete_misses 0
STAT delete_hits 0
STAT incr_misses 36444
STAT incr_hits 19304751
STAT decr_misses 3
STAT decr_hits 19367598
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 26222582271
STAT bytes_written 25905663432
STAT limit_maxbytes 67108864
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT threads 4
STAT conn_yields 0
STAT bytes 27010093
STAT curr_items 366610
STAT total_items 113554333
STAT evictions 0
STAT reclaimed 5291352
END

#

printing large integers with awk

Posted on by
2

When printing with awk, it uses scientific notation by default. Take this snippet from an example file. The first column is a count of how many times a file is present, the second column is the md5sum of that file and the third is the number of bytes that the file is.

# tail -3 md5sums
  14737 113136892f2137aa0116093a524ade0b        53
  19402 1c7b413c3fa39d0fed40556d2658ac73        44
  52818 b7f10e862d0e82f77a86b522159ce3c8        45
#

If I wanted to sum up the number of files counted in this file, and how much total space they are all taking up, I do this:

# awk '{i=i+$1;j=j+($3*$1);} END {print i; print j}' md5sums
22412000
1.45255e+13

So awk counted 22412000 files, totaling about 14.5 TB. Let’s make that a little more readable:

# awk '{i=i+$1;j=j+($3*$1);} END {printf ("%d\n", i); printf("%d\n", j)}' md5sums
22412000
2147483647

Um… that’s not right. But 2147483647 is a special number. You should recognize it as the maximum value of a 32 bit unsigned integer or ((2^32)/2)-1. In this case printf doesn’t handle large integers at all. Instead, use print, but tell awk what the output format should look like:

awk 'BEGIN {OFMT = "%.0f"} {i=i+$1;j=j+($3*$1);} END {print i; print j}' md5sums 
22412000
14525468874034

time mysql select without output

Posted on by
Reply

To see how long a query will take without actually outputting anything, change the pager in mysql client to /dev/null instead of STDOUT

mysql> pager;
Default pager wasn't set, using stdout.

mysql> pager >/dev/null
PAGER set to '>/dev/null'

mysql> select md5sum from table1;
5245618 rows in set (1 min 21.50 sec)

mysql> select md5sum from table1 where md5sum is not NULL and md5sum!='not calculated';
4832676 rows in set (1 min 11.52 sec)

mysql> select md5sum from table1 where md5sum not in ('NULL', '', 'not calculated');
4832078 rows in set (1 min 21.91 sec)

blacklist debian package

Posted on by
Reply

After upgrading synergy, every time I would hit the shift key on the remote screen, it would crash. I found the bug report, and decided to roll back the package. To make sure it didn’t get upgraded automatically, I did this:

# echo "synergy hold" | dpkg --set-selections

Of course this will also work for Ubuntu.

find duplicate entry in sql dump

Posted on by
Reply

Recently, I tried to import a SQL dump created by mysqldump that somehow had a duplicate entry for a primary key. Here’s a sample of the contents:

INSERT INTO `table1` VALUES ('B97bKm',71029594,3,NULL,NULL,'2013-01-22 09:25:39'),('dZfUHQ',804776,1,NULL,NULL,'2012-09-05 16:15:23'),('hWkGsz',70198487,0,NULL,NULL,'2013-01-05 10:55:36'),('n6366s',69480146,1,NULL,NULL,'2012-
12-18 03:27:45'),('tBP6Ug',65100805,1,NULL,NULL,'2012-08-29 21:32:39'),('yfpewZ',18724906,0,NULL,NULL,'2013-03-31 17:12:58'),('UNz5qp',8392940,2,NULL,NULL,'2012-11-28 02:00:00'),('9WVpVV',71181566,0,NULL,NULL,'2013-01-25 06:15:03'),('kEP
Qu5',64972980,9,NULL,NULL,'2012-09-01 06:00:36')

It goes on for another 270,000 entries. I was able to find the duplicate value like this:

# cat /tmp/table1.sql | grep INSERT | sed -e 's/),/\n/g' | sed -e 's/VALUES /\n/' | grep -v INSERT | awk -F, '{print $2}' | sort | uniq -c | awk '{if($1>1) print;}'
    2 64590015
#

The primary key value 64590015 had 2 entries. I removed the spurious entry, and subsequently the SQL imported fine.

find exec with grep pipe

Posted on by
Reply

If you have to search 62,000 log files for a specific string what’s the best way to do it? This will not work:

# zgrep string www1*/apache2/fordodone.com/201*/*/*/*error*.log.gz

Because shell will expand the list, there will be too many arguments for zgrep to process.

Instead use find to find the list of logfiles. You could redirect to a file, then run a forloop on each one, but we can just use -exec with find to run commands on the log files as we find them. This is nice, because you can process the files, and have output as it chugs along. Either of these works:

# find www1*/apache2/fordodone.com/201*/*/*/ -name '*error*.log.gz' -exec zgrep string {} \;

# find www1*/apache2/fordodone.com/201*/*/*/ -name '*error*.log.gz' -exec sh -c 'zgrep string $0' {} \;

In my head it sounds something like this: “find the files in the matching directories, that are named like ‘*error*.log.gz’, and as you find them, execute a command on them. The command is a new shell command to zgrep for the string in the file you just found.”

The first one works fine, BUT if you need to pipe your zgrep or whatever to some other command you need to execute a sub shell for that.

## do sed substitution after
-exec sh -c 'zgrep string $0 | sed -e \'s/A/B/g\'' {} \;

## read backwards and find first (aka last) occurrence
-exec sh -c 'zcat $0 | tac | grep -m1 string' {} \;

Always use single quotes for the subshell command sh -c , becuase you don’t want the current shell to interpret it, but pass the $0 as a literal so that the subshell can interpret it. The $0 in the subshell refers to the FIRST argument it is passed, which in this case is {}, or the file that find has currently found.

mount windows disk image in linux

Posted on by
Reply

I was recently converting a Windows installation from a physical desktop to a virtual machine. Because the installation was an OEM installation the P2V conversion failed the “hardware” check and the target VM was unregistered. The only way to fix was to do a repair install changing the installation from an OEM to a Retail version of Windows. This needed to be done on the physical source desktop. There’s no way I was going to attempt this without a total backup of the system. I could have just copied important files, but I decided a disk clone was more appropriate. I booted off of a Debian rescue cd, mounted a NFS share, and dd’d a copy of the entire drive to a flat file.

# mkdir /mnt/nfsserver/diskclone
# mount nfsserver:/vol/diskclone /mnt/nfsserver/diskclone
# cd /mnt/nfsserver/diskclone
#
# dd if=/dev/sda of=desktop.img
156250000+0 records in
156250000+0 records out
80000000000 bytes (80 GB) copied, 7079.64601 seconds, 11.3 MB/s
#
# ls -l
total 78278496
-rw-r--r-- 1 root root 80000000000 2013-05-20 15:47 desktop.img

I then needed to mount the image and take a look at the contents. A flat disk image file is just a block for block copy of the actual disk, so the first step is to look at the partition table listed in sectors and find the offset. The offset will tell mount where the beginning of the file system is.

# losetup /dev/loop0 desktop.img
#
# fdisk -l -u  /dev/loop0

Disk /dev/loop0: 80.0 GB, 80000000000 bytes
255 heads, 63 sectors/track, 9726 cylinders, total 156250000 sectors
Units = sectors of 1 * 512 = 512 bytes 

      Device Boot      Start         End      Blocks   Id  System
/dev/loop0p1   *          63   156232124    78116031    7  HPFS/NTFS
#
#

Find the offset by multiplying the 63 (start sector) by 512 bytes ( 63*512 ) and the offset is 32256. Now we can tell mount where the start is. 

# mkdir /mnt/diskimage
# mount -t ntfs -o offset=32256,ro desktop.img /mnt/diskimage
#

I mounted readonly so that we don’t mess anything up. You may need to install ntfsprogs to be able to read an NTFS file system (apt-get install ntfsprogs) if you don’t have it installed already

remove many empty directories

Posted on by
Reply 
# find . -depth -mindepth 1 -maxdepth 3 -type d -exec rmdir {} \;

This finds directories, between 1 and 3 levels deep and attempts to remove them. The -depth flag finds the deepest child directories, before finding parents. This is great, because it tries to remove foo/bar/ before it will try to remove foo/. Without removing foo/bar/ first, rmdir foo/ would fail. Because rmdir will fail if there are any contents in a directory, the operation is safe to run without removing any files. You could redirect STDERR to a file, and capture all the directories that are not empty for processing later.