DoS Attacks
There are two types of DoS attacks - inbound and outbound. An inbound attack targets a johncompanies IP, on the johncompanies network.
Our doswatch script will generally notice this right away - within 5 seconds, and it will almost always stop a UDP flood. Everything else (TCP) will be noticed by Castle. They have the ability to insert a null-route for the target.
Inbound DoS attacks are generally not serious and they get stopped by castle (usually) before you would even have a chance to do anything about it. When a customer is DoS’d, our agreement with Castle is that the customer is moved to the bad boy block. There is a(n email) paste for this notification. If the IP is nulled then they’re dead anyway and giving them a new IP is the only option anyway.
The other type of DoS attack is much more serious. This is an outbound attack from one of our systems (always a linux system, it seems) to the outside world.
The problem with this type of attack is that the data path between the jail and virt systems and our firewall is so fast, that any DoS application that is run from a customer system will almost certainly use up all the CPU on the firewall, and stop all network traffic – basically we are completely down. And since the app is running on the inside, it will never stop - if someone doesn't do something, the attack will go on indefinitely.
It should be noted that castle also null-routes outbound attacks, so their headache is over - the problem is, our public network is still completely down.
We’ve mostly solved this problem with iptables rules in place on all virts which restrict the speed of outbound UDP packets. It doesn’t restrict DNS traffic so we’re still succeptible to UDP attacks targeted at a remote DNS server.
When an outbound attack occurs Castle will usually have the offending IP so all you have to do is find the VPS and stop it. If you’re unable to use the mgmt site to lookup the customer machine, you can do a database search from the shell on mail:
mail# m 69.55.238.98 status 1 cid col00214 first_name Shawn last_name Yeager email shawn@shawnyeager.com system jail15.johncompanies.com dir /mnt/data1/69.55.238.98-col00214-DIR veid ip 69.55.238.98; notes canc'ld PP S-8SK44779NL028624B on 8/6/03 Now paying bulk
Once you find the machine, ssh to it’s private IP, then do a ps to see what was/is running at the time of attack (vp <veid>). To further see where the process is run: ps wwp <pid> If the machine is barely responsive you might just have to stop their ve the first thing you do.
If you can’t reach the offending machine, tell castle to unplug the yellow cable from the back of that machine
If this were a jail machine, you would simply log onto that jail machine, run `top` and see the obvious problem process, run jpid and ps wwp <pid> to see who it belongs to, jailkill that server, and so on. If you can note what username it is running under, all the better.