Editing
FreeBSD Reference
(section)
Jump to navigation
Jump to search
Warning:
You are not logged in. Your IP address will be publicly visible if you make any edits. If you
log in
or
create an account
, your edits will be attributed to your username, along with other benefits.
Anti-spam check. Do
not
fill this in!
= ipfw (firewalling) = ipfw is a userland command (/sbin/ipfw) that is, in a sense, the FreeBSD equivalent of the linux `iptables` command. Although the ipfw command is present in all FreeBSD installations, it also requires kernel support. You can enable ipfw by adding this line to your kernel configuration line: options IPFIREWALL and building and installing that new kernel. It should be noted, however, that there are two other options that should probably be added to your kernel configuration file as well: options IPFIREWALL_VERBOSE options IPFIREWALL_VERBOSE_LIMIT=100 The first allows ipfw to do verbose logging of packets and events, and the second limits the number of log entries that a single, consecutive event can log. ipfw support can also be loaded as a module, by simply running: /sbin/kldload ipfw.ko It is VERY IMPORTANT to understand that ipfw, when existent in a kernel, or when loaded as a kernel module with kldload, will _always_ have a single default, final rule. That is, there will always be at least one rule present, and that rule will be the final, catch-all rule for any packet that does not match any prior rules. By default, the final, default rules is: deny ip from any to any Which means (VERY IMPORTANT) that if you enable the firewall in your kernel and then reboot, you will have no network connectivity to the machine in question _at all_. The machine will have a single rule, which is set to deny all ip traffic, and you will not be able to reach it in any way. Similarly, if you simply run `/sbin/kldload ipfw.ko` on a normal FreeBSD system, you will be instantly locked out of the system, and the system will be completely unreachable from the network. There are a few ways to solve this problem. First, you can add a fourth line to your kernel configuration file: options IPFIREWALL_DEFAULT_TO_ACCEPT This means that ipfw support will be added to the kernel, but with a final, default rule of: allow ip from any to any With this rule in place as the final, default rule, even if you add no other rules, when the system restarts you will be able to see it on the network just fine, and no packets will be blocked. Another way to solve the problem is to leave ipfw in the kernel set to the default (which is to deny ip from any to any) but a rule to your startup configuration to open up some or all network services. The default will still be to deny ip from any to any, but any traffic the first matches your allow rules will be allowed in. This method will work fine, regardless of whether you added ipfw support in the kernel or by loading a module. In fact, you can (theoretically, although I distrust this method) run this command: /sbin/kldload ipfw.ko ; ipfw add 65500 allow ip from any to any and not be locked out ... the default rule is always number 65535, so by instantly adding a rule at 65500 to allow all traffic, you can load the ipfw module safely without locking yourself out. I do not recommend this, however, as I do not trust that this will work 100% of the time. Finally, a third method to enable ipfw without locking yourself out is to rebuild the ipfw.ko module to set its default to allow, instead of deny - just like we were able to set a kernel configuration line to set the default to allow. Building this module with that custom configuration is beyond the scope of this document. Why does FreeBSD have a default setting for ipfw to deny all traffic ? The reason is, you do not want to allow a malicious party to circumvent your firewall rules by crashing your machine. In some configurations, firewall rules may not be loaded at boot time, so if the default was "allow all from any to any" then one could circumvent the firewall rules by crashing the firewall - when it came back up the rules would not be loaded, and all traffic would pass. REMEMBER - no matter what you do, there will always be a single rule in place - rule number 65535. You cannot delete this rule, and it is set to deny all or allow all depending on how it was set (as we discussed above). You can view all of the rules currently active on a system by running: /sbin/ipfw show A useful way to see all the rules that might apply to a certain IP, for instance 10.10.10.10, is: /sbin/ipfw show | grep "10.10.10.10" Here is what the results of `ipfw show` would look like on a system with a default accept configuration: # ipfw show 65535 109490 44385056 allow ip from any to any As you can see, there is only one rule - number 65535. You can only configure 65535 rules, and that is always the last one, and is always present. The second number in the output is the packet count - how many packets have passed through that rule, and the third number is the byte count - how many bytes have passed through that rule. You add rules with commands like: ipfw add 100 allow tcp from any to 10.10.10.10 22 That line allows all tcp traffic on port 22 destined for 10.10.10.10. If you now run ipfw show, you would see: # ipfw show 00100 0 0 allow tcp from any to 10.10.10.10 22 65535 116103 46464923 allow ip from any to any You could then delete rule #100 with: ipfw del 00100 A very useful and command setup for a system behind a firewall is to open up only the ports that correspond to services actually running on that system, and deny all other traffic. Here is an example, where the IP in question is 10.10.10.10: ipfw add 100 allow tcp from any to 10.10.10.10 established ipfw add 200 allow tcp from any to 10.10.10.10 22,25,80,443 setup ipfw add 300 deny tcp from any to 10.10.10.10 So, in this example, we first allow any previously established tcp connections to this IP, then we allow any tcp connections that are in a setup (TCP 3-way handshake) mode - these two rules together account for all possible legitimate tcp traffic. Then the third rule simply denies all other tcp traffic. As you can see, ipfw applies rules FIFO - so as a packet travels from rule 0 to rule 65535, as soon as it matches a rule, the packet is processed, and leaves the ruleset - no further rules are applied to that packet, as it has been passed. In the above example, the user is running ssh, smtp, http, and https ... however, for a system to be workable on the network, a few other things should be open if they run a dns server - since dns uses UDP as well: ipfw add 100 allow tcp from any to 10.10.10.10 established ipfw add 200 allow tcp from any to 10.10.10.10 22,25,53,80,443 setup ipfw add 300 allow udp from any to 10.10.10.10 53 ipfw add 400 deny ip from any to 10.10.10.10 Note the addition of rule 300, which allows udp to come in on port 53. Also notice that since dns uses both tcp and udp, we have added 53 to the allowed tcp list as well. Finally, note that we can refer to ipfw rule numbers as either 00200 or 200 - it is the same thing. Also, note that the final rule for this IP, #400, does not simply deny all tcp or all udp (or both), rather, it denies all IP _period_. So if it doesn't match the tcp list, and if it is not udp53, then it gets dropped as soon as it hits rule 400. It should be noted that there is performance penalty on the firewall machine for each rule added to it _that packets pass through_. (so, even if you have 1000 rules on the firewall, if the first rule is `allow ip from any to any` then there is no difference in performance than if the ruleset had only one rule - all packets get passed right away and none pass trough the other 999 rules) If a poor ruleset design is in use, and the firewall takes in a lot of traffic (or passes a DoS attack) it can knock the firewall off the network. The machine will not be crashed - as soon as the traffic lets up it will respond again - but no communication will pass while the CPU is overloaded. The number one consideration when tuning the ruleset for performance is to pass packets that should be passed as fast as possible, and dump packets that should be dumped as fast as possible. For example, we know that we don't want to ever pass certain types of packets (a tcp packet with all option fields set, a tcp packet with no MSS setting, and so on) so, the very first four rules on our firewall are: 00003 49913883 2009604713 deny tcp from any to any tcpflags syn tcpoptions !mss 00003 23958169 1342587681 deny icmp from any to any icmptypes 4,5,9,10,12,13,14,15,16,17,18 00003 142 8496 deny tcp from any to any tcpflags syn,fin 00003 0 0 deny tcp from any to any tcpflags fin,psh,rst,urg This means that these packets, if we ever encounter them (and we frequently do in a DoS attack) are dropped immediately. If we had 1000 rules in our ruleset, and put these at the end, a big DoS attack would cripple the firewall, because _every single_ other rule would have to be processed against every single packet until it got spit out the end. On the other side of the coin, we also allow what we know should be allowed _as fast as we can_. Take the example above: ipfw add 100 allow tcp from any to 10.10.10.10 established ipfw add 200 allow tcp from any to 10.10.10.10 22,25,80,443 setup ipfw add 300 deny tcp from any to 10.10.10.10 Now, lets say we have four machines on our network, so we set up a block of rules for all four: ipfw add 100 allow tcp from any to 10.10.10.10 established ipfw add 150 allow tcp from any to 10.10.10.10 22,25,80,443 setup ipfw add 200 deny tcp from any to 10.10.10.10 ipfw add 250 allow tcp from any to 10.10.10.20 established ipfw add 300 allow tcp from any to 10.10.10.20 22,25,80,443 setup ipfw add 350 deny tcp from any to 10.10.10.20 ipfw add 400 allow tcp from any to 10.10.10.30 established ipfw add 450 allow tcp from any to 10.10.10.30 22,25,80,443 setup ipfw add 500 deny tcp from any to 10.10.10.30 ipfw add 550 allow tcp from any to 10.10.10.40 established ipfw add 600 allow tcp from any to 10.10.10.40 22,25,80,443 setup ipfw add 650 deny tcp from any to 10.10.10.40 Now, each ruleset applies to a particular IP. The problem is, we can tell by looking at this that we _always_ pass established tcp packets – no matter what IP they are for. If an established packet comes in for 10.10.10.40, it first has to pass through 9 other rules before it gets to: ipfw add 550 allow tcp from any to 10.10.10.40 established Since we know we always pass established packets, we can shorten this dramatically by doing this: ipfw add 001 allow tcp from any to any established ipfw add 150 allow tcp from any to 10.10.10.10 22,25,80,443 setup ipfw add 200 deny tcp from any to 10.10.10.10 ipfw add 300 allow tcp from any to 10.10.10.20 22,25,80,443 setup ipfw add 350 deny tcp from any to 10.10.10.20 ipfw add 450 allow tcp from any to 10.10.10.30 22,25,80,443 setup ipfw add 500 deny tcp from any to 10.10.10.30 ipfw add 600 allow tcp from any to 10.10.10.40 22,25,80,443 setup ipfw add 650 deny tcp from any to 10.10.10.40 Now an established tcp connection (that we always want to pass) is passed at rule #1 - as fast as it possible can be passed. If we have hundreds of systems behind this firewall, with rulesets such as this, this can cause the CPU of the firewall to be totally idle, vs. being 50% utilized – just by putting that change in. I have seen it. This is true because a large percentage of all traffic at all times is established tcp traffic. Also, note that we always deny ip from any to (the IP) as soon as that IPs ruleset is over - this is not technically necessary - you could just let the final 65535 "deny all" rule catch it and deny it there - but that would kill performance because the packet would have to be matched against every other rule inbetween first. If you forget to put in index #, it will add as 65535 == Firewall Rule Configuration == The firewall startup script is found here: /etc/firewall.sh It is created periodically based on the current ruleset. The only thing we do with ipfw on the firewall is block or accept packets and occasionally cap some ips (we do not do any counting, or accounting). The first rule is to allow traffic pointed at the firewall itself to pass – this is to facilitate access in the event of a DoS attack. 00001 allow ip from any to 69.55.230.1 Rules 2-10 are for bandwidth capping and blocking bad people: 00002 pipe 2 ip from 69.55.224.109 to any xmit em0 00003 pipe 3 ip from { 69.55.227.54 or 69.55.227.55 } to any xmit em0 00004 pipe 4 ip from 69.55.238.194 to any xmit em0 00005 pipe 5 ip from 69.55.238.162 to any xmit em0 00006 deny ip from 69.22.167.138 to any Rule 100 is for our infrastructure machines: 00100 allow udp from any 53 to 69.55.230.2 00100 allow udp from 69.55.230.2 123 to any 00100 allow udp from 69.55.230.2 to any dst-port 53 00100 allow tcp from any to 69.55.230.2 dst-port 22,25,80,443,110,123,1984,8080 setup 00100 allow icmp from any to 69.55.230.2 icmptypes 0,3,8 keep-state 00100 allow udp from 69.55.230.1 161 to 69.55.230.2 00100 deny ip from any to 69.55.230.2 00100 allow tcp from any to 65.55.238.150 dst-port 25 setup Rules 101-150 are for jails/virts they disable all traffic from the pub net except from mail, backup, dns, and virtuozzo: <pre> 00101 deny ip from any to 69.55.238.120 00102 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.228.53 00102 deny ip from any to 69.55.228.53 00103 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.238.64 00103 deny ip from any to 69.55.238.64 00104 allow ip from { 69.55.230.2 or 69.55.230.9 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.238.92 00104 deny ip from any to 69.55.238.92 00106 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.238.180 00106 deny ip from any to 69.55.238.180 00107 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.238.210 00107 deny ip from any to 69.55.238.210 00109 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.237.129 00109 deny ip from any to 69.55.237.129 00110 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.236.128 00110 deny ip from any to 69.55.236.128 00111 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.236.92 00111 deny ip from any to 69.55.236.92 00112 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.235.200 00112 deny ip from any to 69.55.235.200 00113 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.225.2 00113 deny ip from any to 69.55.225.2 00114 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.226.128 00114 deny ip from any to 69.55.226.128 00115 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.224.32 00115 deny ip from any to 69.55.224.32 00116 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.224.110 00116 deny ip from any to 69.55.224.110 00117 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.228.2 00117 deny ip from any to 69.55.228.2 00130 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 80.89.140.126 or 12.109.148.175 or 69.64.46.27 or 194.67.59.14 or 69.55.238.150 } to 69.55.227.2 00130 deny ip from any to 69.55.227.2 00132 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 80.89.140.126 or 12.109.148.175 or 69.64.46.27 or 194.67.59.14 or 69.55.238.150 } to 69.55.237.220 00132 deny ip from any to 69.55.237.220 00133 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 80.89.140.126 or 12.109.148.175 or 69.64.46.27 or 194.67.59.14 or 69.55.238.150 } to 69.55.236.192 00133 deny ip from any to 69.55.236.192 00134 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 80.89.140.126 or 12.109.148.175 or 69.64.46.27 or 194.67.59.14 or 69.55.238.150 } to 69.55.236.64 00134 deny ip from any to 69.55.236.64 00135 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 80.89.140.126 or 12.109.148.175 or 69.64.46.27 or 194.67.59.14 or 69.55.238.150 } to 69.55.235.170 00135 deny ip from any to 69.55.235.170 00136 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 80.89.140.126 or 12.109.148.175 or 69.64.46.27 or 194.67.59.14 or 69.55.238.150 } to 69.55.234.151 00136 deny ip from any to 69.55.234.151 00137 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 80.89.140.126 or 12.109.148.175 or 69.64.46.27 or 194.67.59.14 or 69.55.238.150 } to 69.55.225.77 00137 deny ip from any to 69.55.225.77 00138 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 80.89.140.126 or 12.109.148.175 or 69.64.46.27 or 194.67.59.14 or 69.55.238.150 } to 69.55.226.2 00138 deny ip from any to 69.55.226.2 00139 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 80.89.140.126 or 12.109.148.175 or 69.64.46.27 or 194.67.59.14 or 69.55.238.150 } to 69.55.226.161 00139 deny ip from any to 69.55.226.161 00140 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 80.89.140.126 or 12.109.148.175 or 69.64.46.27 or 194.67.59.14 or 69.55.238.150 } to 69.55.224.150 00140 deny ip from any to 69.55.224.150 00141 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 80.89.140.126 or 12.109.148.175 or 69.64.46.27 or 194.67.59.14 or 69.55.238.150 } to 69.55.227.70 00141 deny ip from any to 69.55.227.70 00141 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 80.89.140.126 or 12.109.148.175 or 69.64.46.27 or 194.67.59.14 or 69.55.238.150 } to 69.55.229.2 00142 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 80.89.140.126 or 12.109.148.175 or 69.64.46.27 or 194.67.59.14 or 69.55.238.150 } to 69.55.227.70 00142 deny ip from any to 69.55.227.70 00143 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 80.89.140.126 or 12.109.148.175 or 69.64.46.27 or 194.67.59.14 or 69.55.238.150 } to 69.55.230.18 00143 deny ip from any to 69.55.230.18 00144 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 80.89.140.126 or 12.109.148.175 or 69.64.46.27 or 194.67.59.14 or 69.55.238.150 } to 69.55.229.100 00144 deny ip from any to 69.55.229.100</pre> In addition to rule 00010 (allow all established) and rule 65500 (allow all) we also have a few more special rules: <pre>00012 deny tcp from any to any tcpflags syn tcpoptions !mss 00012 deny icmp from any to any icmptypes 4,5,9,10,12,13,14,15,16,17,18 00012 deny tcp from any to any tcpflags syn,fin 00012 deny tcp from any to any tcpflags fin,psh,rst,urg 00012 allow icmp from any to any 00013 allow udp from any to 69.55.225.225 dst-port 53 00014 deny tcp from any to any dst-port 135</pre> These are the four DoS attack lines we have in place right at the beginning of the ruleset. When the machine boots, and is running only three total rules, you then log in and run /etc/firewall.sh - that contains all the additional rules - running the script will put them all in place - then the firewall is fully configured. Now, by default, we do not put any rules in place at all for a customer - they are left wide open. Most customers do not ever change this. However, if a customer requests a ruleset on our firewall, we implement it in the general form that was described above - allow all ports that need to be open, and deny all others. The firewall rule numbers are not numbered arbitrarily - they are numbered by customer number. So customer 327 gets 03270 - 03279, and customer 589 gets 05890 - 05899 ... once we get to customer 1000, they will have 10000 - 10009. This does not mean that every customer can only have 10 rules - as you can see from the four DoS attack rules that are all numbered 00003, you can create multiple rules at the same rule number. I don't advise it though. Because customer requests are generally "allow these and block everything else" we actually have a script on the firewall to create a typical ruleset. The script is called "rulemaker", and it runs like this: # rulemaker usage: rulemaker [cust#] IP [port1,port2,...,port10] So, it has three command line options - the customer number (significant digits only), the IP, and a comma-delimited list of ports (with no spaces). So, if customer 398 comes to you and says: "please open up tcp ports for ssh, smtp and http and close all the rest" then you would run: rulemaker 398 10.10.10.10 22,25,80 And this is what would happen: <pre>gateway# rulemaker 398 10.10.10.10 22,25,80 /sbin/ipfw add 03981 allow udp from 10.10.10.10 to any 53 /sbin/ipfw add 03982 allow udp from any 53 to 10.10.10.10 /sbin/ipfw add 03983 allow tcp from any to 10.10.10.10 22,25,80 setup /sbin/ipfw add 03989 deny ip from any to 10.10.10.10 or, if they have a dns server: /sbin/ipfw add 03981 allow udp from 10.10.10.10 to any 53 /sbin/ipfw add 03982 allow udp from any 53 to 10.10.10.10 /sbin/ipfw add 03983 allow tcp from any to 10.10.10.10 22,25,53,80 setup /sbin/ipfw add 03984 allow udp from any to 10.10.10.10 53 /sbin/ipfw add 03989 deny ip from any to 10.10.10.10 REMEMBER TO ADD YOUR PASTE TO /usr/local/etc/ipfw.sh gateway#</pre> if they have dns, put a 53 in the command line arg to rulemaker You are shown a list of rules to paste into place if they don't run a dns server, and one if they do. Note that rulemaker does not actually put any rules in place at all, it just echos the commands you should run. So, since the customer did not specify port 53, we can assume they do not run a dns server, and we can simply paste this: <pre> /sbin/ipfw add 03981 allow udp from 10.10.10.10 to any 53 /sbin/ipfw add 03982 allow udp from any 53 to 10.10.10.10 /sbin/ipfw add 03983 allow tcp from any to 10.10.10.10 22,25,80 setup /sbin/ipfw add 03989 deny ip from any to 10.10.10.10</pre> into the shell, and hit enter once or twice afterwards. Very simple. We then email the customer and tell them that the lines are in place, and to test them. customer numbers larger than 999 will work fine with this script because: ipfw add 010000 (rule) and ipfw add 10000 (rule) translate into the same thing. So adding unnecessary zeroes does not hurt anything. (the rulemaker script outputs 0$1 as the rule number - so it always prepends a zero to make the three-digit customer numbers correct, and that zero prepended to a four digit customer number will not hurt anything - it will just be ignored) Almost every rule in the firewall is part of a little 4 or 5-line set like rulemaker outputs. Some exceptions are when people want you to open up icmp for them as well (since the above rulemaker output denies it) in which case you would simply paste the rulemaker output, and then afterwards add another rule: ipfw add 03984 allow icmp from any to 10.10.10.10 Remember, if they run a dns server, they need to have tcp port 53 in their port list and you need to paste the second block that rulemaker outputs. Some customers, however, do not request a formal ruleset - they simply say to block off port 3306 from the outside (mysql) or they say to block all netbios ports (135,137,139) or something like that. If they do this, do not use rulemaker - simply add a rule just for that: /sbin/ipfw add 05431 deny tcp from any to 10.10.10.10 3306 or /sbin/ipfw add 05431 deny tcp from any to 10.10.10.10 135,137,139 On the other hand, a customer may request a normal ruleset, but then request that you only open ssh for a certain IP block or IP. Here is an example of a ruleset that was started with rulemaker, but then additional rules were added: <pre> 07471 47802 3991038 allow udp from 69.55.225.125 to any 53 07472 14490 1309166 allow udp from any 53 to 69.55.225.125 07473 85950 4252824 allow tcp from any to 69.55.225.125 22,25,53,80,443,110,143,220 setup 07474 45358 3378454 allow udp from any to 69.55.225.125 53 07475 84 5016 allow tcp from any to 69.55.225.127 22,443 07475 94 5472 allow tcp from any to 69.55.225.128 22,443 07476 38805 3552124 allow icmp from any to 69.55.225.127 07476 38524 3536996 allow icmp from any to 69.55.225.128 07478 6 288 allow tcp from 66.166.221.232/29 to 69.55.225.125 3309 07478 286 13728 allow tcp from 66.166.221.232/29 to 69.55.225.125 3306 07479 109767 6222136 deny ip from any to { 69.55.225.125 or dst-ip 69.55.225.127 or dst-ip 69.55.225.128 }</pre> So ... 69.55.225.125 is the main IP, and what was used in rulemaker, and the main allow line is very familiar: 07473 85950 4252824 allow tcp from any to 69.55.225.125 22,25,53,80,443,110,143,220 setup but then they wanted allow only 22 and 443 to the other two IP addresses: 07475 84 5016 allow tcp from any to 69.55.225.127 22,443 07475 94 5472 allow tcp from any to 69.55.225.128 22,443 (note they share an ipfw rule number) then icmp should also be allowed to the other two IPs: 07476 38805 3552124 allow icmp from any to 69.55.225.127 07476 38524 3536996 allow icmp from any to 69.55.225.128 then there are two addresses out in the world that should be totally unfettered in their ability to talk to the main IP: 07478 6 288 allow tcp from 66.166.221.232/29 to 69.55.225.125 or to two ports 07478 286 13728 allow tcp from 66.166.221.232/29 to 69.55.225.125 3306 07478 286 13728 allow tcp from 66.166.221.232/29 to 69.55.225.125 3309 (note, again, sharing ipfw numbers, and also specifying a netblock instead of a single IP: 66.166.221.232/29) then finally, the last rule that rulemaker outputs was thrown out and this was used instead: 07479 109767 6222136 deny ip from any to { 69.55.225.125 or dst-ip 69.55.225.127 or dst-ip 69.55.225.128 } Since we are dealing with three IPs total. Some more example requests: Replacing a rule (customer wants port 21 access): <pre>gateway# g 69.55.225.3 07161 22462 1795170 allow udp from 69.55.225.3 to any dst-port 53 07162 21220 3283214 allow udp from any 53 to 69.55.225.3 07163 52962 2989600 allow tcp from any to 69.55.225.3 dst-port 22,80,443,25,110,995,143,993,53 setup 07164 20234 1314826 allow udp from any to 69.55.225.3 dst-port 53 07169 30715 2409544 deny ip from any to 69.55.225.3 gateway# gateway# ipfw del 07163 ; ipfw add 07163 allow tcp from any to 69.55.225.3 20,21,22,80,443,25,110,995,143,993,53 setup 07163 allow tcp from any to 69.55.225.3 20,21,22,80,443,25,110,995,143,993,53 setup gateway#</pre> Please block all traffic from this range of IPs: Inet num: 195.238.48.0 - 195.238.63.255 <pre>gateway# g 69.55.226.144 08441 356 21668 allow udp from 69.55.226.144 to any dst-port 53 08442 6744 1114132 allow udp from any 53 to 69.55.226.144 08443 7358 411368 allow tcp from any to 69.55.226.144 dst-port 22,25,80,110,443 setup 08449 3135 280030 deny ip from any to 69.55.226.144 gateway# gateway# ipfw add 08440 deny ip from 195.238.48.0/20 to 69.55.226.144</pre> in reply, say “your ruleset is now…” /etc/firewall.sh is backed up daily locally (/etc/oldrules) and to the backup server ---------------- We add rules to block traffic from directly contacting our jails/virts. Each rule is basically the same except for the id (which reflects the machine) and the machine’s IP Here’s some examples: Jail2: 00102 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.238.2 00102 deny ip from any to 69.55.238.2 Quar1: 00130 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 80.89.140.126 or 12.109.148.175 or 69.64.46.27 or 194.67.59.14 or 69.55.238.150 } to 69.55.227.2 00130 deny ip from any to 69.55.227.2 Virt12: 00142 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 80.89.140.126 or 12.109.148.175 or 69.64.46.27 or 194.67.59.14 or 69.55.238.150 } to 69.55.229.2 00142 deny ip from any to 69.55.229.2 The IPs listed for access are mail (the new mail), backup2, ns1c, and virtuozzo To dump/watch traffic: tcpdump –vvv –n –i em1 == Setting up bandwidth caps on firewall == Creating a new pipe to limit someone's outbound speed: First make sure that you're not about to use a pipe that already exists. <pre>newgateway# ipfw pipe list 00001: 1.000 Mbit/s 0 ms 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 tcp 69.55.224.109/44027 67.28.113.10/25 22 1320 0 0 0 newgateway#</pre> there's already a pipe 1, so we'll use pipe 2, we're also going to add this as rule 2. (in this case the customer's IP is 69.55.224.109, and we only want to catch stuff going out so we use xmit em0. <pre>newgateway# ipfw add 2 pipe 2 ip from 69.55.224.109 to any xmit em0 00002 pipe 2 ip from 69.55.224.109 to any xmit em0 newgateway#</pre> Now all we have to do is set the speed limit: <pre>newgateway# ipfw pipe 2 config bw 1Mbit/s newgateway#</pre> Lastly, list the pipes to make sure everything is the way we want it: <pre>newgateway# ipfw pipe list 00001: 1.000 Mbit/s 0 ms 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 tcp 69.55.224.109/44027 67.28.113.10/25 747 44980 0 0 0 00002: 1.000 Mbit/s 0 ms 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 tcp 69.55.224.109/80 62.172.72.131/26327 8468 9259344 40 42972 1038 newgateway#</pre> Removing a pipe: the rule to match on and the pipe itself have to be deleted separately: <pre>newgateway# ipfw delete 1 newgateway#</pre> and to delete the pipe itself: <pre>newgateway# ipfw pipe delete 1 newgateway#</pre> list the pipes again: <pre>newgateway# ipfw pipe show 00002: 1.000 Mbit/s 0 ms 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 tcp 69.55.224.109/80 62.172.72.131/26327 38383 42636953 48 50955 5111 newgateway#</pre> more than one rule can feed into a pipe, so the speed of everything that matches will get lumped together in the same pipe. this is useful when a customer has more than on IP or system, and you want to limit his total combined speed. == Setting up bandwidth caps on a jail == If there's a situation where a VPS (or VPSs) on a jail are spiking bandwidth, but it's hard to tell who's the culprit, you can put in a global ipfw rule that will limit the connect speed for the entire server and/or each IP on the server. The commands to limit the overall outbound connect speed are: /sbin/ipfw pipe 1 config bw 40Mbit/s /sbin/ipfw add 1 pipe 1 ip from me to any The commands to limit the individual outbound connect speed are: /sbin/ipfw pipe 2 config bw 12Mbit/s mask src-ip 0xffffffff /sbin/ipfw add 2 pipe 2 ip from me to any == ipfw on jail machines == The jail machines also have ipfw loaded, however all of the jail machines have ipfw loaded to default-accept. This is because ipfw on the jail machines is not used for firewalling - it is used for traffic counting. Note- ipfw will not run inside a jail- a jail vps customer cannot have its own self-managed ipfw rules. You see, in addition to deny and allow rules, you can also do things like: ipfw add 00001 count ip from 10.10.10.10 to any ipfw add 00002 count ip from any to 10.10.10.10 which counts traffic bound from that IP, and bound to it. Where does it count it ? Simply in the ipfw rule itself. Here is a sample from one of the jail systems: <pre># ipfw show 00201 10 815 count ip from 198.78.70.176 to any 00202 10 802 count ip from any to 198.78.70.176 01631 1 62 count ip from 69.55.238.225 to any 01632 1 481 count ip from any to 69.55.238.225 01801 72 70154 count ip from 69.55.238.214 to any 01802 82 9047 count ip from any to 69.55.238.214 01811 3 245 count ip from 69.55.238.215 to any 01812 2 167 count ip from any to 69.55.238.215 01821 5 656 count ip from 198.78.66.216 to any 01822 4 377 count ip from any to 198.78.66.216 01841 0 0 count ip from 69.55.238.218 to any 01842 0 0 count ip from any to 69.55.238.218 01851 0 0 count ip from 198.78.66.219 to any 01852 0 0 count ip from any to 198.78.66.219 01861 3 218 count ip from 198.78.66.220 to any 01862 3 263 count ip from any to 198.78.66.220 01921 0 0 count ip from 69.55.238.224 to any 01922 0 0 count ip from any to 69.55.238.224 02241 0 0 count ip from 69.55.238.118 to any 02242 0 0 count ip from any to 69.55.238.118 02261 0 0 count ip from 69.55.238.223 to any 02262 0 0 count ip from any to 69.55.238.223 02271 0 0 count ip from 69.55.237.9 to any 02272 0 0 count ip from any to 69.55.237.9 02291 46 37023 count ip from 69.55.237.8 to any 02292 50 10939 count ip from any to 69.55.237.8 02311 20 1974 count ip from 69.55.237.7 to any 02312 22 1540 count ip from any to 69.55.237.7 03351 0 0 count ip from 69.55.237.163 to any 03352 0 0 count ip from any to 69.55.237.163 65535 102592563 113861945636 allow ip from any to any</pre> Note two things - first, the packet counts and byte counts are very low, since this was taken shortly after system boot. Second, notice that the last line is "allow ip from any to any". That last line is the only line that affects actual traffic in any way - the others are just count rules. Also, note that the rules are done by customer number - the customer number plus either a 1 or a 2 at the end - since every customer needs two total rules to count both inbound and outbound traffic. For instance: 01811 3 245 count ip from 69.55.238.215 to any 01812 2 167 count ip from any to 69.55.238.215 those are the two rules for customer 181 (col00181). Remember how the `jailmake` utility asks for an "ipfw#" ? The three or four digit representation of the customer number is what it is asking for. As you can see, the last section of jailmake contains these lines: /sbin/ipfw add `echo 0"$7"1` count ip from any to $ip /sbin/ipfw add `echo 0"$7"2` count ip from $ip to any since the IP and the ipfw# are specified on the jailmake command line, this is very easy to do. Note again that the rule number is prepended with a zero - which does not hurt anything if it is an extra zero in the case of a four digit customer number. The jails do not add each individual ipfw line at boot time like the firewall does. They are added by the postboot script which should only be run once to avoid duplicate ipfw entries. The only odd thing about ipfw on the jails is that it is loaded as a module on jails 1-10, and loaded in the kernel in all jails 11 and beyond. This is because doing traffic counting on the jails was not thought of until after we had loaded jail10. So, rather than schedule a maintenance and reboot all 10 jail systems, we simply built a default-allow module, placed it in the / directory, and loaded that at boot time. Therefore, on all jails 1-10, you will not only see something like this in the / directory: ipfw.4.7.accept.ko but you will also see a line like this in /usr/local/etc/rc.d/boot.sh: /sbin/kldload /ipfw.4.7.accept.ko (the module is named for the version of freebsd it was built for) Jails 11-15 (and any future ones) have these lines: options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_VERBOSE_LIMIT=100 options IPFIREWALL_DEFAULT_TO_ACCEPT in the kernel configuration file - note the default-accept line. The traffic counting on the freebsd machines is not just for our benefit -to see by running `ipfw show` ... there is also a cron job on every freebsd system: 4,9,14,19,24,29,34,39,44,49,55,59 * * * * /usr/local/jail/bin/trafstats that matches up the rules with the directories, and every five minutes overwrites the users' /jc_traffic_dump with the latest traffic stats.
Summary:
Please note that all contributions to JCWiki may be edited, altered, or removed by other contributors. If you do not want your writing to be edited mercilessly, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource (see
JCWiki:Copyrights
for details).
Do not submit copyrighted work without permission!
Cancel
Editing help
(opens in new window)
Navigation menu
Personal tools
Not logged in
Talk
Contributions
Create account
Log in
Namespaces
Page
Discussion
English
Views
Read
Edit
View history
More
Search
Navigation
Main page
Recent changes
Random page
Help about MediaWiki
Tools
What links here
Related changes
Special pages
Page information