Unscheduled Outage

Major incident natTRAK Pilot Training Department (PTD) VATSIM.net Website Stats Indicators Membership Helpdesk CERT (SSO) Web Services
16 hours, 10 minutes, 56 seconds

Updates

Resolved

After some investigation our team determined that the root cause was not the load balancing of our apps but a bug in our database cluster’s clustering which caused the cluster to halt all operations and hence all of our websites which make use of a database to go down. For those interested to know more about this issue, the patch which we have now applied to fix this issue and all its related information can be found here: https://jira.mariadb.org/browse/MDEV-21002

June 6, 2020 · 23:35 UTC
Monitoring

Our network infrastructure team has implemented a fix to resolve this issue. We are monitoring the situation, and will post an update as soon as the issue is fully resolved.

June 6, 2020 · 15:27 UTC
Update

We have identified a problem with some of our load balancers, causing most of our web services to be affected.

FSD or AFV servers are not affected in any way and users are able to file flight plans directly through their chosen pilot client.

June 6, 2020 · 09:36 UTC
Investigating

We are aware of an issue currently affecting a number of our web services. We are currently investigating this.

June 6, 2020 · 07:25 UTC

← Back