Friday, March 11 2005 @ 12:47 AM CET Contributed by: bart Views: 20043
A few days ago, someone tried a bandwidth exhaustion type denial of service attack against my home connection. They did this by flooding my (public) dns with requests for the nameservers for the . zone.
My nameservers refuse such queries, but I still found the cpu use and upstream bandwidth use to be annoying (8mbit downstream, only 1mbit upstream, so the 'refused' messages would exhaust the upstream bandwidth) so I decided to see if I could reduce both by using a bit of filtering.
The attack lasted for approx 10 hours, flooding my nameserver with over 1 gigabye worth of requests (actually just repeating the same one as fast as the uplink on the other side allowed), comming from a single client.
Filtering out the requests
The obvious solution is to just drop all traffic from that client and be done with it (well, and inform the people responsible for the netblock and law enforcement if you feel like it).
The solution I opted for is a bit different in that it works against any such attack from any ip. The idea is to have a per client limit on dns traffic, ie, allow only so many requests/time/client and drop the rest.
The machine running the nameserver runs FreeBSD 5.3 and has ipfw and dummynet in the kernel already, if you don't have that you will have to load the kernel module for it. Add a line to /boot/loader.conf like:
Be aware that ipfw will be loaded as a dependency and will default to denying all traffic, so you will also have to ensure proper ipfw rules are added on boot. If you are not very familiar with ipfw don't try this without having physical access to the console.
When you have the line in loader.conf and have ensured you can access the machine once ipfw is loaded, you can reboot the machine.
Alternatively, you can load dummynet manually by typing 'kldload dummynet'. Loading it on boot is recomended however.
After doing some measurements on my local network, I determined that for my nameserver that serves some 12 domains, I needed approx 4kbit bandwidth/client in order to not cause any noticable lag.
This can be done with dummynet on FreeBSD, you'll have to create a 'pipe' with the correct bandwidth, number of slots and source/destination masks.
You may have to set the number of buckets also. 64 is the default and should do for most situations, but if you run a really really big nameserver, you may need more buckets (you need one for each concurrent dns client). Also, I serve some 12 domains from my dns, if you have substantially more then 12 domains, you will have to allow for a somewhat bigger queue, ensure it is a little bit bigger then the total number of meaningfull requests a client could be doing, 1.2 times the number of domains you serve seems reasonable as the number of slots, 1024 * domains/3 seems to be reasonable for bandwidth.
Also, mind the mask on the src-ip, this is what makes this work on a per client basis.
Now you have to direct traffic for the dns into this pipe:
ipfw add pipe 1 udp from any to me 53 in
This assumes you are not using ipfw for anything, if you are using it already, make sure to insert the second line at a proper location so that it actually gets to see the dns traffic.
Also, when already using ipfw, you may have to turn off 'one pass' for the firewall:
From man ipfw:
Pass packet to a dummynet(4) ``pipe'' (for bandwidth limitation,
delay, etc.). See the TRAFFIC SHAPER (DUMMYNET) CONFIGURATION
Section for further information. The search terminates; however,
on exit from the pipe and if the sysctl(8) variable
net.inet.ip.fw.one_pass is not set, the packet is passed again to
the firewall code starting from the next rule.
In other words, not disabling one_pass will result in the packet leaving the firewall when it matches the pipe rule, and no other rules being applied to the packet. This is often NOT what you want.
Instead of the above sysctl command, you can try 'ipfw disable one_pass', supposedly that does the same thing.
How does it work?
First of all it limits the bandwidth a client can use. On top of that, it limits the total number of outstanding requests/client. WHen a client sends too many requests too quickly, excess requests will simply be dropped.
So, how well does it work?
It dropped approx 95% of all incomming requests from the machine trying to do the DOS attack, only answering some 5% of those requests. It did not drop any of the thousands of normal requests from other nameservers during that time. This means to me that it works pretty well, it reduces the DOS attack substantially while not hindering normal traffic. Also, it does not depend on the source IP of the attacker. It will not completely protect against a distributed denial of service attack but it will still reduce its effects on the nameserver.
When implementing a setup like this, use ipfw pipe show at regular intervals to see if it is dropping valid requests, if so, slightly increase the number of slots and/or bandwidth.
The numbers for bandwidth and slots are based on observation and not on 'solid' calculations. If you have rather large zones, you may need more bandwidth and slots. The assumption is that a client will only have one or at times two outstanding requests for each zone.
The allowed bandwidth is higher then strictly required. This is to ensure the nameserver stays reasonably responsive to valid requests. I have aimed for answering less then 1 out of 15 requests from a client that is flooding the nameserrver at 1mbit/sec without dropping requests from 'normal' clients or causing more then 10% latency increase for a directly conencted client (100 mbit ethernet)
This setup only makes sense for a public nameserver that is authoritive for some zones, but doesn't answer queries for domains it is not authoritive for. If you also use your nameserver to do hostname lookups for your lan clients, you need to make sure they are excluded from the filter/pipe. Such clients can do lots of valid requests in a very short time, and using a bandwidth/rate limiter will cause problems for this.
This setup protects your nameserver and upstream link, it does not protect your downstreamlink. You may have to talk to your provider for filtering on their side for this
This setup protects very well against a single client trying to flood your nameserver, it provides only limited protection against distributed attacks
Last but not least, I am still wondering what they were trying to achieve with attacking a DNS that serves a few private domains (and that has a good secondary). No important infrastructure there so it seems utterly pointless... but asking for the point of what happens on the net seems a bit much anyway.