Jan 22 - Jim Kennedy reported routing problems to Chile which lasted a
couple of hours.
Jan 24 - Routing problems were found between Maui and Hilo; lasted about 10 minutes.
Jan 25 - More of the same problems occurred; they lasted a couple of hours.
Jan 26 - More routing problems occurred. By then, the SQL Server Slammer Worm was announced by CERT. It was reported to be much more powerful than the Code Red Worm of 2001. UH-ITS network staff while combating attacks on the UH networks, inadvertently filtered off four computers serving as DNS and Web servers within the MKOCN for almost a 24 hours period. UH-ITS staff was asked to contact Miranda and the observatory support staff directly in the future for urgent matters if Pui Hin cannot be reached.
Jan 29 - Both primary and secondary links to the commodity Internet and I2 were down for about an hour due to routing problems. It was not clear whether this incident was related also to SQL worms.
The network has been problem free since Jan 29th. Connection to the
commodity Internet has improved since DREN has switched from AT&T to
WorldCom.
We need from the following from everyone that wants to participate:
Pui Hin asked all present to consult with their directors on this issue.
Pui Hin wished to thank SMA, in particular Antony Schinckel and Keith Brian for their help in assessing the fiber damage, and Mac Cooper for his help on a possible wireless implementation.
Pui Hin explained that for this particular incident, there is no as-built for the segment of fibers affected. It was known that the fiber path is within the area under construction. Pui Hin will work with Ron Koehler on this issue.
Pui Hin went on to clarify that even though MKOCN fibers run alongside the Visitor Center fibers between the HP utility room and the HP main building, connection for the summit would not have been disrupted even if this segment of the MKOCN fibers were damaged.
The MKOCN consists of MKOCN private fibers and Verizon fibers:
Connections from the summit to Hilo and Waimea run along the Verizon fiber bundle, which is a different bundle of fibers than that that goes to HP.
Further, most observatories have a DS3 service between their summit and base facilities which again runs on Verizon fibers. A Verizon T1 serves as a backup from HP to UH Manoa. For Verizon fibers, an emergency plan was worked out with Verizon -- which was GTE Hawaiian Tel in 1998. For the details of this emergency plan, please refer to the URL:
A discussion on how to implement a facility-wide WLAN followed. It was recognised that security is a big concern for wireless connections. Henry Stilmack suggested separating the wireless from the rest of the network so that bandwidth can be restricted if necessary and facilities can configure their firewalls to treat the subnet as unsecured. Users of the wireless connection should recognise the security issues and treat it accordingly.
However, Pui Hin pointed out that even though the Hilo LAN is working fine for now, we will need to consider moving the connection point from the handhole into the IfA building. Being outdoors in the wet Hilo climate, the fiber panel is rusting. The move will involve pulling new fibers from each facility into the IfA building. Mac Cooper pointed out that SMA will be working with Verizon on their private DS3 connection from the summit to their new facility. The connection will need fiber connection from their new building to Verizon equipment already installed in the IfA building. There might be an opportunity to work out something with Verizon for the rest of the Hilo LAN.