I was having trouble understanding the iptables hashlimit module and couldn’t dig up anything that really helped. The man pages are definitely lacking a clear explanation and /proc/net/ipt_hashlimit/ leaves out some information that would clarify things immensely. After some testing I managed to work it all out, so let’s go through it and see if I can help make sense of it for you too.
I’ll try not to assume too much prior knowledge about the module. We’ll be coming at this with the goal of blocking traffic that exceeds a certain amount of packets per second. From the man page:
hashlimit uses hash buckets to express a rate limiting match (like the limit match) for a group of connections using a single iptables rule. Grouping can be done per-hostgroup (source and/or destination address) and/or per-port. It gives you the ability to express “N packets per time quantum per group” or “N bytes per seconds”
There are three settings that are most important, in my opinion:
–hashlimit-above amount[/second|/minute|/hour|/day] Match if the rate is above amount/quantum.
–hashlimit-burst amount Maximum initial number of packets to match: this number gets recharged by one every time the limit specified above is not reached, up to this number; the default is 5. When byte-based rate matching is requested, this option specifies the amount of bytes that can exceed the given rate. This option should be used with caution — if the entry expires, the burst value is reset too.
–hashlimit-htable-expire msec After how many milliseconds do hash entries expire.
Each of these settings controls how the packets we’re hoping to control match on our iptables rule — and for the purpose of explaining things properly we’ll use this as our example:
-A INPUT -p icmp -m hashlimit --hashlimit-name ICMPTEST --hashlimit-mode srcip --hashlimit-srcmask 32 --hashlimit-above 5/minute --hashlimit-burst 2 --hashlimit-htable-expire 30000 -j DROP
This rule will drop icmp traffic that exceeds the configuration. If you’re following along and testing, this rule should go above any state checks in your firewall. Please note that a reload on iptables does not refresh these buckets properly. Once this rule is in place, any tweaks to the hashlimit module’s values (e.g., –hashlimit-above) requires restarting iptables!
With this in place, if you ping your server from another host, after 2 packets the rest will drop until 12 seconds elapse, then one will be let through, after another 12 seconds one will be let through, and so on. What’s going on behind the scenes?
To get an idea of that you can
watch the buckets through their
# watch --interval 1 "cat /proc/net/ipt_hashlimit/ICMPTEST"
And after a couple pings you might see something like the following:
28 10.0.0.2:0->0.0.0.0:0 198912 768000 384000
What are we seeing here?
28: This is the remaining number of seconds before the bucket expires. The maximum value of this is what was set in
--hashlimit-htable-expire, even though that was done in milliseconds. Every time a new packet comes in that matches this hash table entry, this value will reset to the maximum. When it reaches zero, the hash table entry expires.
10.0.0.2:0->0.0.0.0:0: This is the hash table entry as defined by the
--hashlimit-mode, in our case
198912: Remaining number of tokens in this entry’s bucket. Each matched packet will cause a number of tokens to be removed from the bucket (that value is shown shortly). If subtracting those tokens from the bucket would cause it to reach 0 (or less), the
-joperation for the rule takes place, otherwise it continues through the rule chain. Tokens do not get removed from the bucket unless the full amount can be removed
768000: Maxmimum number of tokens in this entry’s bucket – this value is the result of a calculation based on the
384000: Token value of a packet. Each packet matching the rule subtracts this number from the remaining number of tokens in this entry’s bucket if there are enough tokens available to do so
Now that’s all well and good but it’s hiding something important and for me that was what made part of it difficult to wrap my head around — it doesn’t tell you how many tokens are being restored to the bucket per second! It wasn’t until I
watch‘d the proc interface that I understood what was going on. How do you get that number? Let’s use the above example
- We’re allowing 5 packets per minute (
- The packet value is
- That means we need to restore
1920000tokens to the bucket per minute, or
32000tokens per second. After 12 seconds, we’ve restored
384000tokens to the bucket, allowing one packet through
We can see this in action, working as we expect:
PING 10.0.0.3 (10.0.0.3): 56 data bytes
64 bytes from 10.0.0.3: icmp_seq=0 ttl=63 time=0.527 ms
64 bytes from 10.0.0.3: icmp_seq=1 ttl=63 time=0.526 ms
Request timeout for icmp_seq 2
Request timeout for icmp_seq 3
Request timeout for icmp_seq 4
Request timeout for icmp_seq 5
Request timeout for icmp_seq 6
Request timeout for icmp_seq 7
Request timeout for icmp_seq 8
Request timeout for icmp_seq 9
Request timeout for icmp_seq 10
Request timeout for icmp_seq 11
64 bytes from 10.0.0.3: icmp_seq=12 ttl=63 time=0.470 ms
Request timeout for icmp_seq 13
Request timeout for icmp_seq 14
Request timeout for icmp_seq 15
Request timeout for icmp_seq 16
Request timeout for icmp_seq 17
Request timeout for icmp_seq 18
Request timeout for icmp_seq 19
Request timeout for icmp_seq 20
Request timeout for icmp_seq 21
Request timeout for icmp_seq 22
Request timeout for icmp_seq 23
64 bytes from 10.0.0.3: icmp_seq=24 ttl=63 time=0.443 ms
So let’s summarize:
- hashlimit uses buckets of tokens
- A calculated, static value of tokens is assigned for packets
- The maximum number of tokens in the bucket is
--hashlimit-burst * packet value
- Hash table entries are created based on the
- A new entry into the hash table creates a bucket
- When no packets have matched that entry in
--hashlimit-htable-expirems, the entry is expired
- Packets matching the iptables rule subtract tokens from the bucket for the hash table entry and reset the expire timer
- When a bucket reaches 0, or would reach 0 after a subtraction, the rule action is performed
- Every second the bucket adds enough tokens to achieve the desired