Following the upgrades over the weekend we have identified an issue with PPPoE/LCP negotiation which affects Sonicwall routers. Firebrick have released a firmware update which overcomes this issue, and we intend to roll the firmware update out to the primary LNS Wednesday evening during the usual mid week maintenance window. The actual upgrades will…
Author: jon
Latest
DSL Network Firmware Upgrades
by jon • • 2 Comments
We will be performing maintenance on the ADSL network over the weekend to upgrade the firmware on our core routers. This is in response to the earlier unexpected resets which have now been tracked to a problem with the BGP implementation on the Firebrick. We have worked with the vendors and now have a firmware…
Broadband
Unexpected DSL Reset
by jon • • 0 Comments
During minor routine maintenance this evening at approx 23:30 BST to add additional peers to our network, one of our LNS router’s BGP sessions flapped causing a brief outage for customers connected through lns-1 which is one of our Firebrick routers. After some investigation we believe we have traced this to an issue with the…
CoLocation Centres, London 2 (TFM120 - Telehouse West), Planned Maintenance
Firmware update DBC – core-2-b2
by jon • • 2 Comments
We need to perform a reboot of core-2-b2 in DBC this evening to load a new firmware release, which will bring this switch up to date and enable certain IPv6 features currently missing or disabled due to bugs. The reboot should take approximately 5 minutes, during which time access to some of our Virtuozzo clusters…
Bulletin Board, Hosting, Latest
RFO: switch failure dbc-1
by jon • • 1 Comment
At approximately 17:00 BST this evening we experienced a total failure of one of the Cisco switches in our London 2 facility which caused a small number of customer connections (those without dual uplinks) and hosted servers to go offline. Unfortunately, due to the nature of the failure, it took a short while to identify…
CoLocation Centres, Hosting, Latest
Routine Maintenance – 27.03.12
by jon • • 0 Comments
We will be performing routine maintenance Wednesday evening between 9pm and 11pm During this maintenance we will be disconnecting, upgrading and relocating one of our border gateway routers. As a result we will be operating on reduced capacity and at risk of service interruption in the event of a failure of our other border gateway…
Broadband
Fiber Maintenance
by jon • • 1 Comment
Apologies for the short notice. We will be moving all connections across to a new fiber backbone at 10am on Saturday 17th March 2012. The work should be seamless however DSL connections will drop briefly but should all auto reconnect within a few seconds. With the introduction of the new fiber links we have shorter…
Broadband
Broadband Maintenance
by jon • • 1 Comment
We will be loading a new code release on the Broadband Network Servers at midnight tonight. This new code provides additional support for enhanced services we have recently launched including Bonding. Connections should drop briefly as they are moved to backup servers during the reboot process
Bulletin Board, Hosting, Planned Maintenance
Emergency Maintenance: Cloud Clusters 2 & 6
by jon • • 0 Comments
This evening we detected a faulty power relay feeding cloud clusters 2 and 6. The fault was resulting in disconnects and unexpected reboots on both clusters. Engineers were despatched and the faulty relay replaced at approximately 23:30 during which time both clusters may have become inaccessible briefly. Apologies for any inconvenience caused by this unplanned…
Latest
Fido Cloud Maintenance (Cluster 6)
by jon • • 0 Comments
Cluster 6 has undergone another disk reset, almost exactly a month after the last outage. We are continuing to investigate this issue and in the mean time are resetting the cluster controller. We anticipate normal service to be resumed within 15 minutes. UPDATE Famous last words. During the reboot an inconsistency has been detected and…