This morning during a scheduled customer cut-over a standard ACL change seemed to have triggered unexpected behaviour on a device's (pe1.per1) OSPF process, leading to a loss in communications from pe3.per1 to pe1.per1, pe2.per1 and RS1.WA, after attempts at bringing OSPF back to a functional state proved unsuccessful we rebooted pe1.per1 which resolved the issue.
At this time we suspect a long device uptime may be the culprit and discussions are underway to assess a reboot schedule across exchanges to avoid this in the future.
Posted Jul 19, 2022 - 00:34 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Jul 18, 2022 - 23:35 UTC
Investigating
IAA Engineers have lost communication to hardware at QV1 and is under investigation now.
RS1.WA will be unreachable too.
Member service ports delivered to this location will be experiencing an outage.
Updates to follow as soon as we get more information.
If you need more information or assistance during this time please contact us on 1300 793 320 or noc@ix.asn.au