An unexpected wifi outage occurred on the Keene State campus last week on Monday, Oct. 2. According to an email from the Help Desk at Keene State, the outage was “impacting wireless network access in offices, classrooms, and residence halls and spaces.”
While the email sent out by the help desk was received by KSC staff and students at 12:33 p.m., Dan Corbeil, the director of USNH Network and Telecom Services, said reports were being received as early as 10:30 a.m. He said they began noticing wireless Access Points (APs) dropping off the network. An AP is essentially what allows wireless devices to connect to wired networks. After originally investigating to see if the cause was a connectivity issue, Corbeil and the teams working with him discovered the root of the problem to be centered around the Dynamic Host Configuration Protocol (DHCP) servers.
DHCP servers are responsible for assigning new IP addresses and leases to APs that request them. IPs are necessary for a device to communicate with a network and the lease dictates how long the IP is available for use. Corbeil said that the error arose because of a “sync issue” between the two DHCP servers responsible for the eduroam network. The inclusion of two DHCP servers in the network is supposed to allow for redundancy, an alternate path for data to flow, within the system in case of an issue with one of the servers.
As requests came in from the APs to the DHCP servers, “One was issuing an IP address with a long lease time at the same time that the other was issuing an [IP address] with a short lease time,” he said. “What’s supposed to happen is you make a request, one of the servers responds, the other one knows the first one responded and doesn’t try to do the same actions.”
Once they had identified the issue, Corbeil said that they removed the secondary DHCP server from the network, allowing the correct information to flow through the primary.
“As soon as we removed the secondary server, everything started to recover. At that point, we then focused on, how do we make the secondary server recover and become functional again,” he said. “We rebooted the whole server at that point, brought it back up, restarted the services individually and everything synced back up.”
Corbeil said that from when they discovered there was a problem to the point where they believed the system was recovering took a little more than four hours. Elise Morrissette, who works for USNH Network and Telecom Services at Keene State, said that one of the issues that slowed the recovery process was how the scale increased at an unexpected rate. When they had first received reports, it was thought to be a more localized or classroom service issue, but as the reports grew in frequency they knew that there was a larger issue to troubleshoot.
Morrissette said that once the help desk had received the all clear from Corbeil and his team, they began to communicate to the faculty and students that things were recovering. At 4:24 p.m., the Keene State Help Desk sent out a second email confirming the resolution of the Wi-Fi issue.
Harvie Marcotti can be contacted at
hmercotti@kscequinox.com
