During minor routine maintenance this evening at approx 23:30 BST to add additional peers to our network, one of our LNS router’s BGP sessions flapped causing a brief outage for customers connected through lns-1 which is one of our Firebrick routers.
After some investigation we believe we have traced this to an issue with the BGP implementation on the Firebrick and its interoperation with our Juniper routers.
We are currently opening TAC faults with both vendor to see who admits to the problem first, and hopefully fixes it. In the mean time we will refrain from making BGP session updates during core hours and will only add/remove a peer between 23:00 and 01:00 Wed / Fri / Sun which are our standard “intrusive” maintenance windows.
I will post more of an update once I’ve heard back from the respective vendors
Jon