Whilst engineers were working on the fault between our sites in Telehouse North and Telehouse West we experienced a further break in our backup connection(s) between North and West resulting in the isolation of the facility in West.
We have been working to configure a temporary route via Telehouse East as an additional backup route while Telehouse staff work to repair the damaged fibres between North and West
In addition we have experienced a strange hang of the new Juniper EX4200 virtual chassis which resulted in a prolonged outage earlier this evening whilst engineers investigated and identified the problem – which required a physical power cycle of the chassis.
We are also investigating an issue with one of the failure modes where by the master/backup configuration should have resulted in the THW backup chassis becoming a master if split from the THN master, however instead all nodes in the VC in West defaulted to line card mode. We will be discussing this with Juniper once the more immediate issues have been addressed
Apologies for the break in service. A more detailed RFO will be published in the next 7-10 days once all the facts have been collated