High API Latency
Incident Report for Discord
Resolved
The incident has been resolved.
Posted Sep 29, 2017 - 08:30 PDT
Update
Latencies and resize times look to have returned back to normal now.
Posted Sep 29, 2017 - 08:01 PDT
Monitoring
We have been able to provision more capacity and service should be recovering for API latency. Images backlog is still being processed, and we expect another 10-15 minutes until everything is all caught up.
Posted Sep 29, 2017 - 07:57 PDT
Update
We have isolated the issue to our autoscalers for our API and Media Proxy services being unable to provision more instances to handle the normal load increase that happens during the day. We are working with Google Compute to resolve the issue.
Posted Sep 29, 2017 - 06:54 PDT
Update
We are also aware of images failing to load and be resized with our image proxy, causing things like uploads to appear to fail.
Posted Sep 29, 2017 - 06:37 PDT
Investigating
We are aware of elevated API latency and are working on resolving the issue.
Posted Sep 29, 2017 - 06:28 PDT