Update – By way of further information we understand the issue may have originated with Telekom Malaysia who leaked routes to Global Crossing (AS3549) which were trusted by Level3 (AS3356), the two now being the same company and combined making the largest tier1 network on the planet. This resulted in Level3 tripping max prefix limits…
Author: jon
CoLocation Centres, London 2 (TFM120 - Telehouse West), Planned Maintenance
Emergency Maintenance 5-7th June
by jon • • 0 Comments
Following on from the failure of IP House operated by iRack Networks Ltd (and previously also Media City in Salford Manchester) we have been forced to take emergency action to secure customer servers and all customer data housed within the IP House London facility. Steps have been taken to migrate all customer data and servers securely…
Bulletin Board, Hosting, Planned Maintenance
web-11 updates and reboot
by jon • • 0 Comments
We are preparing to perform upgrades and a reboot on web-11 Initial work is well underway however the actual reboot will most likely be scheduled for the early hours of Thursday morning as we are running towards the end of our planned maintenance window tonight and not all checks have been completed. The upgrades constitute…
Hosting
web-21 unexpected reboot
by jon • • 1 Comment
web-21 underwent an unplanned reboot this morning at approximately 03:30 BST As a result the local file system is in need of a check and repair, which it is anticipated will take upwards of 45 minutes Once the fsck process has completed we should be able to restore the web server to normal…
CoLocation Centres
LINX Extreme LAN Broadcast Storms
by jon • • 0 Comments
During scheduled maintenance on the LINX Extreme LAN, there appears to have been an issue which resulted in a high volume of broadcast traffic affecting numerous LINX members. The broadcast traffic resulted in sessions dropping ungracefully and caused connections to other peering LANs to also fluctuate. LINX are investigating this issue currently, in…
CoLocation Centres, Hosting, Latest, London 1 (Hex), London 2 (TFM120 - Telehouse West), London 3 (TFM3 - Telehouse North), London-4 (TFM42 - Telehouse East)
LINX Juniper LAN Broadcast Storms
by jon • • 0 Comments
The LINX (London Internet Exchange) Juniper peering LAN is currently experiencing high volumes of broadcast traffic. As a precaution, we have disabled connections to the LINX Juniper LAN in order to restore stability We are awaiting an update from LINX as to the cause of the broadcasts and their plans to resolve the situation
Hosting, Latest, London 2 (TFM120 - Telehouse West), London 3 (TFM3 - Telehouse North), Planned Maintenance
AT RISK: Bank Holiday Maintenance 23-25th May 2015
by jon • • 0 Comments
We are planning a full range of works over the Bank Holiday weekend which will result in upgraded capacity and improved network performance. During the maintenance window a number of segments of the network will be upgraded which may result in brief periods where services should be considered “At Risk” when redundant paths are…
London 3 (TFM3 - Telehouse North)
THN line card failure
by jon • • 0 Comments
We are experiencing what appears to be a repeat failure on the replacement line card which was installed earlier this week in Telehouse Engineers are en route to investigate and attempt a work around. Current estimates for an ETA are 2am Attempts have already been made to route around the affected area of the…
CoLocation Centres
Emergency Maintenance May 1st-3rd
by jon • • 0 Comments
Following on from yesterday’s DDoS attack on one of our clients, we have been working to further minimise the impact of any further attacks During this mitigation process we identified an optimisation which could be quickly applied to the network and it was decided to implement this earlier this evening Unfortunately this caused…
London 3 (TFM3 - Telehouse North), Planned Maintenance
Edge Router Reboot (THN)
by jon • • 0 Comments
One of our edge routers / BGP speakers started to run into difficulties at around 00:15 this morning. After a brief investigation it was deemed that the router (which had been up for over 300 days) was starting to exhibit “out of memory” behaviour which we believe can be attributed to a memory leak caused…