Networking Services - Latency or Timeouts When Deploying Networking Services - Mitigated

Incident
January 25, 2:09am EST

Networking Services - Latency or Timeouts When Deploying Networking Services - Mitigated

Status: closed
Start: January 25, 2:08am EST
End: January 25, 2:09am EST
Duration: 1 minute
Affected Components:
Cloud Providers Azure Network Infrastructure
Update

January 25, 2:08am EST

January 25, 2:08am EST

Summary of Impact: Between 7:08 UTC and 12:30 UTC on 25 Jan 2023, you were identified as a customer in Canada East, East US, South Central US, West US, and Canada Central who may have experienced latency or timeouts when deploying networking services through the Azure portal.

 

Preliminary Root Cause: We determined that our services were affected by a network latency brought about by a networking router that was taken out of rotation for maintenance during a spike in traffic. This led to increased congestion on some links.

 

Mitigation: The router was removed from service, and optimized the routes for the flow of traffic so services could resume normally.

 

Next Steps: A full root cause to investigate why this router caused wide spread impact will be developed. Stay informed about Azure service issues by creating custom service health alerts: https://aka.ms/ash-videos for video tutorials and https://aka.ms/ash-alerts for how-to documentation.

Resolved

January 25, 2:09am EST

January 25, 2:09am EST

Resolved