IRQBALANCE_BANNED_CPUS explained and working on Ubuntu 14.04

Despite the explanation and examples in the manual, there is still some confusion on how to use irqbalance to move interrupts to a specific set of CPUs. (For the sake of simplicity, I use the words CPUs in the same way Linux sees them; it doesn’t care whether they are physical, or cores, or threads.)

Let’s start with a box that has 8 CPUs

# grep processor /proc/cpuinfo 
processor       : 0
processor       : 1
processor       : 2
processor       : 3
processor       : 4
processor       : 5
processor       : 6
processor       : 7

irqbalance uses a variable set at runtime ( IRQBALANCE_BANNED_CPUS ) to make decisions which CPUs should not receive interrupts. IRQBALANCE_BANNED_CPUS is a hexidecimal representation of a “bitmask” to ignore certain CPUs when deciding which CPU to use for a specific interrupt. If we have a box with 8 cpus we will need to start with a binary string 8 digits long: “00000000”

0 means “allow interrupts on this CPU”
1 means “prevent interrupts on this CPU”

In the case of “00000000” we are allowing interrupts on all 8 CPUs. In binary numbers the least significant bit is on the right, mapping the CPUs like this:
“CPU7 CPU6 CPU5 CPU4 CPU3 CPU2 CPU1 CPU0″

Let’s say we want to allow interrupts on the 6th and 7th CPU, and exclude the 1st, 2nd, 3rd, 4th, 5th, and 8th CPU from interrupts. Our bitmask in this case would look like this”

   1    0    0    1    1    1    1    1
"CPU7 CPU6 CPU5 CPU4 CPU3 CPU2 CPU1 CPU0"

This can get tricky. Most humans start counting from 1 but computers start counting at 0. So be careful: the 7th CPU is CPU6 in this case.

Now that we have our bitmask of “10011111” we need to convert it to hexadecimal. You could use this handy bash function:

#
bin2hex() {
   echo "obase=16;ibase=2;$1"|bc
}
# bin2hex 10011111
9F
#

So now we can edit /etc/default/irqbalance and set the variable:

export IRQBALANCE_BANNED_CPUS="9F" 

NOTE: I have been able to get irqbalance working on Ubuntu 14.04 trusty only by exporting the IRQBALANCE_BANNED_CPUS variable using “export” in /etc/default/irqbalance AND using the latest (at the time of this writing) irqbalance 1.0.9

Once you understand how the CPUs translate to the bitmask you can extrapolate out from there and come up with a way to ban specific CPUs from irqbalance for any number of CPUs.

share:

summing proc interrupts (testing irqbalance)

We can sum the interrupts by CPU to see how the balance is or if IRQBALANCE_BANNED_CPUS is working

# awk '{if(NR>1){for (i=2; i<=9; i++){ a[i]+=$i;}}} END {for (i in a){ j=i-2; printf "CPU"j" "; print a[i];}}' /proc/interrupts 
CPU0 173547874
CPU1 17912
CPU2 17845
CPU3 17081
CPU4 17110
CPU5 21396188
CPU6 27354442
CPU7 17073
share: