Previous incidents
Elevated rate of 415 errors
Resolved Dec 06 at 07:47pm GMT
The issue that caused the elevated rate of 415 errors has been resolved.
The truncated files were caused by a software issue that wrote some files with zero bytes. As it was software, this caused the truncated files to be replicated across all versions of the master file. Sirv triplicates every file to protect against hardware issues (server or hard drive failures), however, the updated files passed all checks and faithfully triplicated the files. It existed without detection because cach...
6 previous updates
Elevated rate of 429 errors
Resolved Dec 02 at 10:31am GMT
The issue was resolved at 1031 UTC.
The cause was a server misconfiguration which made the Sydney CDN location see all requests coming from the same IP address, instead of the true client IP address. This triggered DDOS protection, to rate-limit the excessive number of requests from the same IP. 429 responses were returned for many of the requests.
The issue appeared to have been an extremely short issue when the Sydney datacentre network became unavailable. That issue lasted approximat...
1 previous update
API searches failing
Resolved Nov 23 at 10:35am GMT
At 1035 UTC on 23 November, the upgrade was complete and searches resumed normal operation. There was no downtime and all systems functioned as normal, except that API search requests had been returning error responses.
The issue was not identified because other API requests were returning as normal, so total API error responses were only slightly elevated and no alerts were triggered. The issue was identified after the event, after a report by a user prompted an investigation. To automati...
1 previous update
Control panel at my.sirv.com loading slower than normal
Resolved Nov 15 at 02:09pm GMT
The new server was deployed in the evening of 14 November and gradually filled its caches. The control panel has now returned to its original fast performance. Additional servers are now being upgraded, so the control panel will load faster still over the coming 2 weeks.
2 previous updates