15.6. Circumventing Path MTU Discovery issues with per route MTU settings
For sending bulk data, the Internet generally works better when using larger
packets. Each packet implies a routing decision, when sending a 1 megabyte
file, this can either mean around 700 packets when using packets that are as
large as possible, or 4000 if using the smallest default.
However, not all parts of the Internet support full 1460 bytes of payload
per packet. It is therefore necessary to try and find the largest packet
that will 'fit', in order to optimize a connection.
This process is called 'Path MTU Discovery', where MTU stands for 'Maximum
Transfer Unit.'
When a router encounters a packet that's too big too send in one piece, AND
it has been flagged with the "Don't Fragment" bit, it returns an ICMP
message stating that it was forced to drop a packet because of this. The
sending host acts on this hint by sending smaller packets, and by iterating
it can find the optimum packet size for a connection over a certain path.
This used to work well until the Internet was discovered by hooligans who do
their best to disrupt communications. This in turn lead administrators to
either block or shape ICMP traffic in a misguided attempt to improve
security or robustness of their Internet service.
What has happened now is that Path MTU Discovery is working less and less
well and fails for certain routes, which leads to strange TCP/IP sessions
which die after a while.
Although I have no proof for this, two sites who I used to have this problem
with both run Alteon Acedirectors before the affected systems - perhaps
somebody more knowledgeable can provide clues as to why this happens.
15.6.1. Solution
When you encounter sites that suffer from this problem, you can disable Path
MTU discovery by setting it manually. Koos van den Hout, slightly edited,
writes:
The following problem: I set the mtu/mru of my leased line running ppp to
296 because it's only 33k6 and I cannot influence the queueing on the
other side. At 296, the response to a key press is within a reasonable
time frame.
And, on my side I have a masqrouter running (of course) Linux.
Recently I split 'server' and 'router' so most applications are run on a
different machine than the routing happens on.
I then had trouble logging into irc. Big panic! Some digging did find
out that I got connected to irc, even showed up as 'connected' on irc
but I did not receive the motd from irc. I checked what could be wrong
and noted that I already had some previous trouble reaching certain
websites related to the MTU, since I had no trouble reaching them when
the MTU was 1500, the problem just showed when the MTU was set to 296.
Since irc servers block about every kind of traffic not needed for their
immediate operation, they also block icmp.
I managed to convince the operators of a webserver that this was the cause
of a problem, but the irc server operators were not going to fix this.
So, I had to make sure outgoing masqueraded traffic started with the lower
mtu of the outside link. But I want local ethernet traffic to have the
normal mtu (for things like nfs traffic).
Solution:
ip route add default via 10.0.0.1 mtu 296 |
(10.0.0.1 being the default gateway, the inside address of the
masquerading router)
In general, it is possible to override PMTU Discovery by setting specific
routes. For example, if only a certain subnet is giving problems, this
should help:
ip route add 195.96.96.0/24 via 10.0.0.1 mtu 1000 |
Take our 
