Vyatta monitor and log NAT translation

Logging to record NAT translations. This might be helpful for finding users using bit torrent (along with tshark), or for watching what IPs are connecting to what external services, and when.

while true; do d=`date +%Y%m%d%H%M`; show nat translations detail | grep -v Pre-NAT | paste - - | sort -nk1 >nats.$d.log; sleep 5m; done;
# tail -1 nats.201408261250.log
a.a.a.a:21845    z.z.z.z:443     b.b.b.b:21845       z.z.z.z:443         tcp: snat: a.a.a.a ==> b.b.b.b  timeout: 42 use: 1 

vyatta create firewall network group from geo ip table

The global Geo IP table is huge, and contains ranges of IP addresses that don’t necessarily fit into CIDR networks. Converting the IP ranges in the table into networks results in around 129,000 separate networks. Trying to define every network scope in the global table for inclusion in a firewall is well outside the realm of possibility. In the case of a few small countries, their ranges do actually fit into network groups. This is just a fun little exercise not to be used in production of course.

#!/usr/bin/perl
# FILE: netGroupCountry.pl
# AUTHOR: ForDoDone
# DATE: 2013-12-02
# NOTES: create a firewall network group for a specific country using a geoip table

use strict;
use Net::Netmask;

# download geoip table
# TODO: wget geoip table if doesn't exist, or if 1st wed of month

# set country here
# TODO: read from ARGV
# use Estonia for example it only has 236 IP ranges in table
my $country = 'EE';

# set geoip database CSV
my $geoIPDB='/var/lib/geoip/GeoIPCountryWhois.csv';

# open geoip table
open FILE, "$geoIPDB" or die $!;

# start the vyatta config wrapper
system("/opt/vyatta/sbin/vyatta-cfg-cmd-wrapper begin");

# clear the old network group
system("/opt/vyatta/sbin/vyatta-cfg-cmd-wrapper delete firewall group network-group $country");

# commit here optional
#system("/opt/vyatta/sbin/vyatta-cfg-cmd-wrapper commit");

# setup commit counter for chunked commits
#my $i=0;
#my $chunk=100;

# read each line of the geoip table (~82k lines)
while(<FILE>){
  chomp;

  my @line = split(/"/,$_);

  # if country matches the one we want
  # convert the ip range to CIDR(s)
  # add the network to the network group
  if ("$line[9]" eq "$country" ){

    my @blocks = range2cidrlist("$line[1]","$line[3]");

        foreach(@blocks){
          my $setcmd = "set firewall group network-group $country network $_";
          system("/opt/vyatta/sbin/vyatta-cfg-cmd-wrapper $setcmd");

          #print "set firewall group network-group $country network $_\n";

          # chunked commit
          #if ( $i % $chunk ){
          #  # do nothing
          #} else {
          #  print "committing after 100 entries... total: $i\n";
          #  system("/opt/vyatta/sbin/vyatta-cfg-cmd-wrapper commit");
          #}
          #$i++;

        }

  }

}

# close geoip table
close FILE;

# run vyatta config wrapper to commit and end
system("/opt/vyatta/sbin/vyatta-cfg-cmd-wrapper commit");
system("/opt/vyatta/sbin/vyatta-cfg-cmd-wrapper end");
vyatta@vyatta:/usr/local/sbin$ time ./netGroupCountry.pl
Nothing to delete (the specified node does not exist)

real	1m3.515s
user	0m22.660s
sys	0m27.650s
vyatta@vyatta:/usr/local/sbin$

This definitely won’t work for larger networks, and who knows how it actually affects performance, I’ve never actually tried using the network group in the firewall.

Vyatta create and update IP based ban lists from Spamhaus

You can use Spamhaus, or a number of other lists in a Vyatta firewall configuration. In this case we create a network group called ‘blocked’ from the Spamhaus blacklists. Then this network group can be used in firewalls to drop traffic. Use cron to update the list every day, or once a week.

#!/bin/bash
# FILE: /usr/local/sbin/updateBanList.sh
# AUTHOR: ForDoDone fordodone@fordodone.com
# DATE: 2013-10-01
# NOTES: Script to update IP ban list.  Run from cron, and integrate into firewall
# 

# variables
VERBOSE=0
DROPURL='http://www.spamhaus.org/drop/drop.txt'
EDROPURL='http://www.spamhaus.org/drop/edrop.txt'

# simple logger function
logger(){
  if [ "$VERBOSE" == "1" ]
  then
    echo "$@"
  fi
}

# set verbose flag if given
if [ "$1" == "-v" ]
then
VERBOSE=1;
fi

# create or truncate tmp file
>/tmp/block

# get drop file
wget -q $DROPURL -O - | grep ^[0-9] | sed -e 's/;.*//' >> /tmp/block
if [ $? -ne 0 ]
then
  logger "error getting drop file"
  logger "exiting..."
exit
fi

# get edrop file
wget -q "$EDROPURL" -O - | grep ^[0-9] | sed -e 's/;.*//' >> /tmp/block
if [ $? -ne 0 ]
then
  logger "error getting edrop file"
  logger "exiting..."
exit
fi
logger "received `wc -l /tmp/block | awk '{print $1}'` networks to block..."

logger "starting vyatta cmd wrapper"
/opt/vyatta/sbin/vyatta-cfg-cmd-wrapper begin

# remove existing list, in case a network has been removed"
logger "deleting existing blocked network group"
/opt/vyatta/sbin/vyatta-cfg-cmd-wrapper delete firewall group network-group blocked

# add each network to the block list
logger "building new blocked network group"
logger "this might take a while..."
for i in `cat /tmp/block`;
do
  /opt/vyatta/sbin/vyatta-cfg-cmd-wrapper set firewall group network-group blocked network $i
done;

# now commit the changes
logger "committing changes"
/opt/vyatta/sbin/vyatta-cfg-cmd-wrapper commit

logger "ending vyatta cmd wrapper"
/opt/vyatta/sbin/vyatta-cfg-cmd-wrapper end

# clean up
rm -rf /tmp/block >/dev/null 2>&1

vyatta generate config set commands

To see the set commands that are run to make your Vyatta configured the way it is, use the vyatta-config-gen-sets.pl script:

# sudo /opt/vyatta/sbin/vyatta-config-gen-sets.pl

This will output every single command that was run to get your system configured to where it is now. This is great for copying and pasting default bits to all your Vyatta servers, such as static routes config, package repository config, firewall config, etc. You can put this in your .bashrc to have a shortcut to it:

alias gensets='/opt/vyatta/sbin/vyatta-config-gen-sets.pl'

test bandwidth throughput using iperf

Testing bandwidth between 2 hosts can be helpful to determine maximum transfer rates, and how much bandwidth can be pushed. The iperf utility is a great option for this. Set up one side as a listening server and the other side as a client. In this scenario I want to test an OpenVPN tunnel between a Raspberry Pi and a Vyatta server. The layout looks like this:

Laptop --(wireless)--> RPi --(wireless)--> Linksys router -> Enterasys router -> Cable Modem -> ISP -> INTERNET -> Vyatta -> remote host

Start iperf in server mode on the remote host listening for UDP datagrams:

# iperf -s -u
------------------------------------------------------------
Server listening on UDP port 5001
Receiving 1470 byte datagrams
UDP buffer size:   110 KByte (default)
------------------------------------------------------------

Start up the client. The -c flag tells it what remote host to connect to with iperf, the -u makes it use UDP, the -b tells it what bandwidth to try to achieve, and the -t sets how many seconds to run for:

# iperf -c 10.211.0.141 -u -b 1000K -t30

As the test runs, the server and client will output the rates. Here’s the server side output:

[  3] local 10.211.0.141 port 5001 connected with 10.211.0.6 port 40816
[ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total Datagrams
[  3]  0.0-30.0 sec  3.57 MBytes    998 Kbits/sec  14.738 ms    4/ 2552 (0.16%)

So we were able to push a full 1 Mbps and only lost 4 datagrams, that’s pretty good. I would say this can comfortably push 1Mbps. (Side note: top reported openvpn process on RPi was at 9% CPU utilization) Let’s try for 2Mbps:

# iperf -c 10.211.0.141 -u -b 2000K -t30

And the output:

[  4] local 10.211.0.141 port 5001 connected with 10.211.0.6 port 56707
[  4]  0.0-30.0 sec  6.21 MBytes  1.74 Mbits/sec  7.346 ms  670/ 5103 (13%)

We were able to achieve 1.74 Mbps, but we lost 13% of the traffic. That is unacceptable! I would run again at 1500K and narrow in where exactly the break point is for throughput. To find the bottleneck you can run iperf from every (Linux) host in the chain to pinpoint the limitation. I suspect it could be the upload speed of my home internet connection, or the poor wireless signal of the client side of the RPi.