Understanding iptable’s hashlimit module

I was having trouble understanding the iptables hashlimit module and couldn’t dig up anything that really helped. The man pages are definitely lacking a clear explanation and /proc/net/ipt_hashlimit/ leaves out some information that would clarify things immensely. After some testing I managed to work it all out, so let’s go through it and see if I can help make sense of it for you too.

I’ll try not to assume too much prior knowledge about the module. We’ll be coming at this with the goal of blocking traffic that exceeds a certain amount of packets per second. From the man page:

hashlimit uses hash buckets to express a rate limiting match (like the limit match) for a group of connections using a single iptables rule. Grouping can be done per-hostgroup (source and/or destination address) and/or per-port. It gives you the ability to express “N packets per time quantum per group” or “N bytes per seconds”

There are three settings that are most important, in my opinion:

–hashlimit-above amount[/second|/minute|/hour|/day] Match if the rate is above amount/quantum.

–hashlimit-burst amount Maximum initial number of packets to match: this number gets recharged by one every time the limit specified above is not reached, up to this number; the default is 5. When byte-based rate matching is requested, this option specifies the amount of bytes that can exceed the given rate. This option should be used with caution — if the entry expires, the burst value is reset too.

–hashlimit-htable-expire msec After how many milliseconds do hash entries expire.

Each of these settings controls how the packets we’re hoping to control match on our iptables rule — and for the purpose of explaining things properly we’ll use this as our example:

-A INPUT -p icmp -m hashlimit --hashlimit-name ICMPTEST --hashlimit-mode srcip --hashlimit-srcmask 32 --hashlimit-above 5/minute --hashlimit-burst 2 --hashlimit-htable-expire 30000 -j DROP

This rule will drop icmp traffic that exceeds the configuration. If you’re following along and testing, this rule should go above any state checks in your firewall. Please note that a reload on iptables does not refresh these buckets properly. Once this rule is in place, any tweaks to the hashlimit module’s values (e.g., –hashlimit-above) requires restarting iptables!

With this in place, if you ping your server from another host, after 2 packets the rest will drop until 12 seconds elapse, then one will be let through, after another 12 seconds one will be let through, and so on. What’s going on behind the scenes?

To get an idea of that you can watch the buckets through their /proc interface

# watch --interval 1 "cat /proc/net/ipt_hashlimit/ICMPTEST"

And after a couple pings you might see something like the following:

28 10.0.0.2:0->0.0.0.0:0 198912 768000 384000

What are we seeing here?

  • 28: This is the remaining number of seconds before the bucket expires. The maximum value of this is what was set in --hashlimit-htable-expire, even though that was done in milliseconds. Every time a new packet comes in that matches this hash table entry, this value will reset to the maximum. When it reaches zero, the hash table entry expires.
  • 10.0.0.2:0->0.0.0.0:0: This is the hash table entry as defined by the --hashlimit-mode, in our case srcip
  • 198912: Remaining number of tokens in this entry’s bucket. Each matched packet will cause a number of tokens to be removed from the bucket (that value is shown shortly). If subtracting those tokens from the bucket would cause it to reach 0 (or less), the -j operation for the rule takes place, otherwise it continues through the rule chain. Tokens do not get removed from the bucket unless the full amount can be removed
  • 768000: Maxmimum number of tokens in this entry’s bucket – this value is the result of a calculation based on the --hashlimit-above and --hashlimit-burst values
  • 384000: Token value of a packet. Each packet matching the rule subtracts this number from the remaining number of tokens in this entry’s bucket if there are enough tokens available to do so

Now that’s all well and good but it’s hiding something important and for me that was what made part of it difficult to wrap my head around — it doesn’t tell you how many tokens are being restored to the bucket per second! It wasn’t until I watch‘d the proc interface that I understood what was going on. How do you get that number? Let’s use the above example

  • We’re allowing 5 packets per minute (--hashlimit-above 5/minute)
  • The packet value is 384000
  • That means we need to restore 1920000 tokens to the bucket per minute, or 32000 tokens per second. After 12 seconds, we’ve restored 384000 tokens to the bucket, allowing one packet through

We can see this in action, working as we expect:

So let’s summarize:

  • hashlimit uses buckets of tokens
  • A calculated, static value of tokens is assigned for packets
  • The maximum number of tokens in the bucket is --hashlimit-burst * packet value
  • Hash table entries are created based on the --hashlimit-mode setting
  • A new entry into the hash table creates a bucket
  • When no packets have matched that entry in --hashlimit-htable-expire ms, the entry is expired
  • Packets matching the iptables rule subtract tokens from the bucket for the hash table entry and reset the expire timer
  • When a bucket reaches 0, or would reach 0 after a subtraction, the rule action is performed
  • Every second the bucket adds enough tokens to achieve the desired --hashlimit-above setting

One thought on “Understanding iptable’s hashlimit module

Leave a Reply

Your email address will not be published. Required fields are marked *