All affected circuits are now up, and we expect them to remain stable. We apologise or the disruption caused, we will be working with the vendor to establish the cause.
Posted Dec 16, 2022 - 21:45 GMT
Update
The bulk of our NNI's are now restored and we can see circuits back up. There remain 2 NNI which are causing issues are being restored as soon as possible.
Posted Dec 16, 2022 - 20:18 GMT
Update
As result of hardware failure, NNI 's are in the process of being moved across to alternative hardware. All affected circuits are now back online on another switch. Replacements are being built & configured.
Posted Dec 16, 2022 - 19:32 GMT
Update
We believe a memory leak caused the device to crash, ending all seasons on one of the switches. Unfortunately, upon reload, a second switch on the same chassis crashed. An engineer has now been dispatched to site to resolve. The network is still considered At Risk.
Posted Dec 16, 2022 - 18:42 GMT
Identified
All affected broadband seasons have reconnected. Leased Lines via Telehouse West were not affected however as part of the emergency work to resolve this issue, we will need to reload our core device. This will see leased lines connected via Telehouse West will drop and reconnect. Upon a successful reload, Telehouse West will start connecting any new broadband seasons and the network will no longer be treated as At Risk, as the network will have multiple routes to the wider internet. We apologise for any disruption that may be encountered.
Posted Dec 16, 2022 - 17:25 GMT
Investigating
At approx. 17:05 today, our point of presence at Telehouse West suffered an unexpected outage causing broadband seasons to disconnect. Any broadband seasons using our Slough PoP were unaffected, All affected seasons are now routed through Slough and as such, the network is treated At-Risk due to reduced redundancy.
Posted Dec 16, 2022 - 17:15 GMT
This incident affected: xDSL Broadband (SOGEA, SOGFAST).