Dave Bullock / eecue

photographer, director of engineering: crowdrise, photojournalist, hacker, nerd, geek, human

secure SNMP monitoring of IPFilter

i've been working on setting up a system of bridging ip-less packt filters with ipfilter (for logging, filtering, accounting, proxying, NAT) and ipfw (for bandwidth limiting). I also wanted to set up a private mointoring network with a thrid NIC in each box. Even though this link was privat I still wanted to keep everything on the wire encrypted.

I started with a few spare servers:

[red] pentium 1 200MHz MMX w/ 160MB Ram 3 100BaseT NICs (this server may end up being replaced with a more modern system)

[blue] pentium II 350MHz w. 256MB ram, 3 100BaseT NICs

[yellow] dual pentium II 400MHz w/ 512MB RAM 2 NIC cards.

I compiled the latest release of FreeBSD, 4.6.2 with a kernel patch and the options:

options IPFIREWALL #firewall
options IPFIREWALL_VERBOSE #enable logging to syslogd(8)
options IPFIREWALL_FORWARD #enable transparent proxy support
options IPFIREWALL_VERBOSE_LIMIT=100 #limit verbosity
options IPFIREWALL_DEFAULT_TO_ACCEPT #allow everything by default
options IPV6FIREWALL #firewall for IPv6
options IPV6FIREWALL_VERBOSE
options IPV6FIREWALL_VERBOSE_LIMIT=100
options IPV6FIREWALL_DEFAULT_TO_ACCEPT
# IPFIREWALL as well. See the dummynet(4) and ipfw(8) manpages for more info.
# You can use IPFIREWALL and DUMMYNET together with bridging.
# BRIDGE enables bridging between ethernet cards -- see bridge(4).
options BRIDGE
# DUMMYNET enables the "dummynet" bandwidth limiter. You need
# When you run DUMMYNET it is advisable to also have "options HZ=1000"
# You can use IPFIREWALL and DUMMYNET together with bridging.
options DUMMYNET
# Some subsystems, such as DUMMYNET or DEVICE_POLLING, might benefit from
# IPDIVERT enables the divert IP sockets, used by ``ipfw divert''
options IPDIVERT #divert sockets

i built the kernel on a nice fast quad xeon i use for web hosting and the just tarred it up along with the other programs i intened to use on each sensor:

[red],[blue]
net-snmp
snort (with mysql support)
zebeedee to encrypt my udp which i ended up not using due to the high overhead it created.. then i noticed i could set snmp to use tcp (over ssh)
sqlsyslogd

on the collector
[yellow]
mysqld
apache
php
mrtg (which i scrapped for rrd)
rrd
remark (my php snort front end)

[yellow] connects to [red] and [blue] amd does the following:
creates tunnels for snmp from port 162 on the [yellow] to port 161 on [red] and from port 163 on the [yellow] to port 161 on [blue]
it also creates tunnels for mysql from port 3306 on [red] to port 3306 on [yellow] and from port 3306 on [blue] to port 3306 on [yellow]

it then launches a small script on [red] and [blue] that kill all snort processes, sleep 2, start snort

ucd-snmp comes with a perl script to get the counters from ipfilter so i just set up rules to count the packets i'm interested in like icmp and stuff on port 80 etc.

now i just need to configure rrd to make pretty graphs from all that info.