[OUTAGE] NSW-IX | Peer Flapping

Incident Report for Internet Association of Australia

Resolved

This incident has been resolved.

Our investigation found that an IGP costing change related to resolving intercapital congestion on SYD-ADL and ADL-PER paths triggered LDP TLV messages from Extreme (EXOS) devices across the network. While this behaviour is expected and previously never caused an issue, these particular messages were received as malformed by Arista (EOS) devices. As a result, all LDP adjacencies toward EXOS devices were torn down and rebuilt.

This caused a disruption of VPLS sessions toward EXOS devices lasting approximately 15 to 45 seconds, affecting a number of peers across NSW-IX.

We are working with vendors to replicate and mitigate this behaviour in our lab environment. If any maintenance is required on NSW-IX as a result, a separate notification will be issued.
Posted May 30, 2025 - 07:24 UTC

Identified

We have identified that malformed LDP TLV messages disrupted NSW-IX VPLS services during an intercapital congestion scenario. We have temporarily disabled this cause of congestion while we investigate and work toward a resolution.

A subset of peers may have experienced flapping BGP sessions as a result. We do not expect this to recur.
Posted May 30, 2025 - 03:04 UTC
This incident affected: Peering Fabric (NSW-IX).