Issues connecting and sending messages
Update - We have remediated and are recovering slowly
Dec 15, 10:37 PST
Investigating - We are currently investigating this issue.
Dec 15, 10:18 PST
Uptime over the past 90 days. View historical uptime.
API Partial Outage
90 days ago
99.83 % uptime
Today
Gateway Operational
90 days ago
99.99 % uptime
Today
CloudFlare ? Operational
Media Proxy ? Operational
90 days ago
100.0 % uptime
Today
Voice Operational
EU West Operational
EU Central Operational
Singapore Operational
Sydney Operational
US Central Operational
US East Operational
US South Operational
US West Operational
Brazil Operational
Hong Kong Operational
Russia Operational
Japan Operational
South Africa Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
had a major outage
had a partial outage

No incidents or maintenance related to this downtime.

System Metrics Month Week Day
API Response Time
Fetching
Past Incidents
Dec 14, 2019

No incidents reported.

Dec 13, 2019
Intermittent Connection Errors / Elevated Latency
Resolved - This incident is resolved. Keep an eye on the Help Center for updates on the Spotify integration.
Dec 13, 17:31 PST
Update - Latency and API access has returned to normal levels. Select integrations with Spotify and a handful of other apps are currently disabled while we work on an underlying issue that powers this feature.
Dec 13, 17:31 PST
Update - The majority of user impact has been mitigated, however there is still no root caused determined. Certain endpoints are disabled while the underlying issue is identified.
Dec 13, 17:01 PST
Update - We are continuing to investigate the issue. A root cause has yet to be identified.
Dec 13, 15:40 PST
Update - A possible cause has been identified and temporary mitigations have been put in place. We are continuing to investigate the underlying issue.
Dec 13, 13:21 PST
Update - We are continuing to investigate this issue.
Dec 13, 12:28 PST
Investigating - We're experiencing an increase in errors connecting to Discord and a higher delay in processing user actions. We are currently looking into the issue.
Dec 13, 12:27 PST
Dec 12, 2019

No incidents reported.

Dec 11, 2019

No incidents reported.

Dec 10, 2019

No incidents reported.

Dec 9, 2019

No incidents reported.

Dec 8, 2019
Elevated API Errors
Resolved - At this time service has returned to normal. This was the result of degraded performance in one of our databases as fallout from the earlier outage related to slow disk IO in GCP. At this time the database is back to being in the expected state.
Dec 8, 07:43 PST
Monitoring - Our database capacity has been restored and throttling disabled.
Dec 8, 07:23 PST
Update - We're in the process of restoring expected capacity and working to reduce the amount of throttling currently impacting connections. We should have more information soon.
Dec 8, 06:18 PST
Identified - We are currently working to resolve database connection issues across the product. New connections to Discord are throttled while we address the issue.
Dec 8, 05:10 PST
Investigating - We're experiencing an elevated level of API errors and are currently looking into the issue.
Dec 8, 05:03 PST
Dec 7, 2019
API Errors / Latency
Resolved - Service looks stable and everyone is back online and chatting. We are still working on recovering the search APIs.
Dec 7, 14:35 PST
Monitoring - Traffic levels have been restored to their usual levels, and most users are back in. Our search API is currently down and our engineers are working on recovering that, but otherwise service looks to be stable.

We are following up with google for a postmortem of the underlying issue, and are monitoring.
Dec 7, 14:01 PST
Update - We've started opening the flood-gates and are letting users back in. Recovery is proceeding as planned - and we will continue to ramp up traffic levels and monitor.
Dec 7, 13:36 PST
Update - Google has updated their status page:

> Current data indicates that approximately 0.01% of PD-SSD volumes in us-east1-b are affected by this issue, from a peak of approximately 8.45%.
> At this time, the issue is contained, and we can confirm that this is only impacting us-east1-b.

We are currently beginning to work on service recovery on our end. Please standby!
Dec 7, 13:06 PST
Update - Google's Engineering team continues to work on mitigating the issue, and have posted the following update:

> Mitigation work is currently underway by our engineering team.
> We do not have an ETA for mitigation at this point.

We have all hands on deck awaiting the resolution of this issue to begin restoring service.
Dec 7, 11:58 PST
Update - We continue to observe elevated IO latency. Google has finally gotten to updating their status page. We continue to await resolution.

https://status.cloud.google.com/incident/compute/19012
Dec 7, 11:35 PST
Update - Google is currently investigating an issue with SSD Persistent Disk in our region (what our database clusters store their data on). We are awaiting their resolution.

Given the IO starvation, we are expecting continued API latency - as most if not all of our datastores are currently degraded. We will post updates as we get them.
Dec 7, 10:54 PST
Identified - Multiple engineers are online, investigating the issue. We are noticing anomalously high iowait across most of our database clusters all other instances leading us to believe that this is a google cloud issue. We are working on a mitigation, however latency is expected to be high as we are currently starved for IO resources.
Dec 7, 10:37 PST
Investigating - We're experiencing an elevated level of API errors and are currently looking into the issue.
Dec 7, 10:22 PST
Dec 6, 2019

No incidents reported.

Dec 5, 2019

No incidents reported.

Dec 4, 2019

No incidents reported.

Dec 3, 2019

No incidents reported.

Dec 2, 2019

No incidents reported.

Dec 1, 2019

No incidents reported.

← Incident History Powered by Statuspage