[Update – 11:18] Networking service has been restored and appears stable. Please contact support if you continue to experience any issues. More updates will be posted once a cause is determined.
[Update – 07:53] Commvault services are enabled and working — file restores are successful.
[Update – 07:42] The reboot did not fully correct the problem. Access has been restored to the SDSC Cloud services, including Compute, Storage, and the Dashboard. However, internet access to XSEDE resources may still be unavailable.
[Update – 00:57] Engineers have found a problem with one of the Oasis switches, and they are performing a reboot of the switch.
[Update – 00:29] Engineers are still working to determine the cause and appropriate remediation.
[Update – 23:07] The Commvault backup service is unavailable as it cannot communicate with the storage arrays. Backups generally finished before the outage, but file restoration will be unavailable.
—-
At approximately 18:00, connectivity to some resources behind the Oasis network device stopped responding to the general Internet. Network engineers are investigating and this will be updated as there is more information.
Services that are unavailable include SDSC Cloud Compute and Cloud Storage endpoints, including the Cloud Dashboard, and some resources on HPC networks which have Internet-facing services.