Get in touch
Back
Degraded

Some CDN requests returning 502 responses from Washington DC POP

Started 26 Dec at 02:32pm GMT, resolved 26 Dec at 08:55pm GMT.

CDN request from N.America
Resolved

The issue has been fully resolved.

The errors were returned by the Washington CDN server after its hard drive rapidly reached maximum capacity. Such an issue has not occurred before because each CDN server automatically deletes less-frequently accessed files, to maintain space. However, unusually heavy load of CDN MISSes outpaced the rate at which other files were deleted, causing heavy filesystem load and the failure of some requests.

The 502s occurred before the server had detected the matching account, so a count of the exact number cannot be displayed in account analytics. Approximately 117,000 responses were affected, less than 1% of total requests over the period.

To prevent this from happening again, the Washington DC location is being upgraded with faster drives and updated cache clearing logic.

Posted 26 Dec at 08:55pm GMT.
Created

Some requests to the Washington DC CDN are being returned slower than normal and some are returning a 502 response. This is due to hard disk reaching its caching capacity much faster than usual. We are resolving the issue and responses will gradually return to normal.

Posted 26 Dec at 02:32pm GMT.