The issue was resolved at 1031 UTC.

The cause was a server misconfiguration which made the Sydney CDN location see all requests coming from the same IP address, instead of the true client IP address. This triggered DDOS protection, to rate-limit the excessive number of requests from the same IP. 429 responses were returned for many of the requests.

The issue appeared to have been an extremely short issue when the Sydney datacentre network became unavailable. That issue lasted approximately 1 minute only. However, the actual incident commenced immediately after our team had restored the Sydney CDN location, which is when the server misconfiguration was applied. The issue was solved once we discovered the misconfiguration and corrected it.

Multiple actions are being taken to prevent this issue from recurring and to reduce the impact of similar but different issues in future:

• Additional monitoring and alerting
• New issue submission form
• Context sensitive resolution advice
• Wider internal alerting

Only the Sydney CDN location was affected.

From 0419 UTC, an elevated level of 429 errors were returned from the Sydney CDN location.