Neighbour table overflow

After implementing a network gateway and firewall server solution to one of our clients having 500+ users, we encountered “Neighbour table overflow” error upon checking of the server logs.

This is the command to view server logs:

$sudo tail -f /var/log/messages

Mar 30 10:39:25 www kernel: Neighbour table overflow. Mar 30 10:39:30 www kernel: printk: 7 messages suppressed. Mar 30 10:39:30 www kernel: Neighbour table overflow. Mar 30 10:39:36 www kernel: printk: 4 messages suppressed. Mar 30 10:39:36 www kernel: Neighbour table overflow. Mar 30 10:39:40 www kernel: printk: 10 messages suppressed. Mar 30 10:39:40 www kernel: Neighbour table overflow. Mar 30 10:39:45 www kernel: printk: 12 messages suppressed. Mar 30 10:39:45 www kernel: Neighbour table overflow. Mar 30 10:39:50 www kernel: printk: 24 messages suppressed. Mar 30 10:39:50 www kernel: Neighbour table overflow. 

This message indicates that the server is too busy processing some firewall rulesets.

The immediate and rational solution that we opted is to increase the internal ARP cache size of the server and DNS threshold for ARP.

At the console, edit sysctl.conf

$sudo vi /etc/sysctl.conf

Append the following values:

# Force gc to clean-up quickly
net.ipv4.neigh.default.gc_interval = 3600
# Set ARP cache entry timeout
net.ipv4.neigh.default.gc_stale_time = 3600
# Setup DNS threshold for arp
net.ipv4.neigh.default.gc_thresh3 = 4096
net.ipv4.neigh.default.gc_thresh2 = 2048
net.ipv4.neigh.default.gc_thresh1 = 1024

To make sure that the solution is working, load the new changes using sysctl command.

$sudo sysctl -p

Upon several tests, the solution proved to work.