We have a bunch of Xserves running Mac OS X server at work. Most of the machines are only using one of the 2 built in Ethernet ports as they are on the LAN. We have 2 machines that are customer facing and we just added a dedicated inbound connection for them. Being the clever person that I am, I decided that to ease the transition between the old and new IP blocks, I’d block the new connection into the second Ethernet port and we’d be good to go. Turns out it isn’t that easy with the Darwin kernel. I setup the default connection to be the new network connection and traffic to the new IP addresses worked fine. However, traffic to the old address got hung up. After a lot of investigation, I determined it was due to Asymmetric routing. No problem, I thought, a few commands and it would work. I managed to do this in Linux by following an article, but it wasn’t so easy in Mac OS X. Basically the traffic coming in to the old IP block had the responses going out through the other Ethernet interface out over the new IP block. Many routers block this as it kind of looks like an attack of sorts.
OK, so now that I was stuck, what would I do? I spent about 12 hours on this issue and through some magic use of Apache proxies and another server, I was able to get things working. It isn’t pretty, but it solves the problem until we can get an A record changed (we have no control over that record).
Lesson learned, really, really, think carefully before deploying Mac OS X as a server; it can be quite frustrating to do things that I believe should be simple. Maybe I expect too much and OS X Server isn’t designed for me.