Leave it to a phone system to cry havoc on a Small Business Server 2011 network. In this case it was an older version (5) of Avaya IP office. To make matters worse, the VOIP capability isn’t being used. It’s a straight digital phone system.
We receive a morning call from a client that some Windows 7 systems can’t get online. Not all, some. You have to confirm that kind of thing. A couple phone calls to the offline users revealed that they couldn’t see any network shares or get to the internet. This was the first time this had happened at this site. The SBS environment was about a month old and performing quite well. I never like the first time problems…something changed, now to figure out what.
Luckily, it didn’t take long. After remotely connecting to the server (and exhaling the sigh of relief that I could), the SBS Server Manager showed me the Big Red X. The DHCP Server had stopped. No problem. Click Start, the service state changed to Started (Ah, OK. That was easy), service changed to Stopped (Mothe******.). This is a SBS self-protection feature. It will stop its own DHCP Service if it sees another one on the network.
So I hop in the truck and head out there. Once on-site, I set up the offline users with static IP addresses to get them online and turn the heat down. While doing this, I notice that none of them have IP addresses. All of them had the standard (169.x.x.x) IP address that windows machines give themselves if they can’t find a DHCP server. This worried me a little because I had expected them to be getting an address from somewhere. If there is a rogue DHCP server on the network, it sucked because it won’t even give out addresses. What a tease, and not the good kind. I’m in a race against time now as the other users will drop off the network as their DHCP leases expire.
I catch a break by finding out the Avaya phone vendor had made a visit on-site the week before. A-ha. I call Matt, our phone guy. He says by default the Avaya IP Office software DHCP service is on, and if a change had been made recently that could be our culprit. I login to the Avaya IP Office and notice that DHCP is turned on, but for a different subnet. I ensure DHCP is disabled for both subnets, alert the client that the phones will going down and reboot the phone system.
After the Avaya system reboots, the SBS is still angry. The DHCP service still won’t start. Rebooting the SBS doesn’t change anything. Checking the premises for wireless routers, WAPs or anything else that could be a DHCP server came up empty. It had to be the Avaya, and we had to do something soon, before the leases expired. It wasn’t just the leases though; SBS is a control freak. It has to know and be everything on the network. If the DHCP service didn’t start, other errors would appear soon.
Matt looked into the Avaya protocols, while I looked to try and trick SBS into thinking it was the only DHCP server. Turns out, there’s no trick; just a registry entry to tell SBS to ignore anything it detects on the DHCP Ports (UDP 67,68).
To disable rogue DHCP detection on the SBS
To disable rogue DHCP detection on the SBS by creating following registry entry:
1. Open Registry Editor.
2. Navigate to
3. Create a new DWORD with the following registry value:
Value name: DisableRogueDetection
Data type: REG_DWORD
Value data: (Hexadecimal) 1, which will be saved as 0x00000001
4. Restart the server.
That worked. DHCP started, and we were in good shape. Matt and I wanted answers, though, as we can’t have Avaya systems hijacking our SBS environments. Avaya has the DHCP and TFTP services bundled into one on/off switch. The DHCP service runs on the standard UDP ports 67,68, while TFTP runs on UDP 69. The problem here is that with DHCP disabled, the BOOTP protocol is still active on the DHCP reserved ports. Disabling BOOTP will clear the UDP ports of traffic that trigger the DHCP Shutdown on the SBS.
If the Avaya BOOTP is disabled, as it is now in our environment, all phones needing configuration information will need the IP address of the DHCP and TFTP servers entered manually.
Whew. Just another day at the office.
Categories: IT Pros