SDSC engineers are working to determine a cause for a brief outage of UCSD owned networks that reside in the SDSC colo that occurred from ~14:48-15:05.
Category Archive: Incident Notification
SDSC Outage Notification – VM – 6 Mar 2025, 0500
[Update 1001] Engineers have resolved the latency issue and are continuing to investigate the root causes. [Update 0839] Engineers continue to diagnose the latency issues. Thank you for your patience. Engineers are working to address storage latency issues with the VM systems. This is leading to unresponsive guests.
SDSC Incident Notification – USS Storage Slowness – 27 Jan 2025
[Update 21:16] Vendor provided a fix to address the quorum events. The storage cluster is now stable and performing as expected. Please contact support@sdsc.edu with any questions or concerns. SDSC engineers are investigating quorum events within the Universal Scale Storage service. Updates will be posted here.
SDSC Outage Notification – SDSC Cloud Compute – 30 Sep 2024, 16:12
[Update – 08:14] Issue has been resolved. Please contact support@sdsc.edu with any concerns or questions. [Update – 19:56] Issue is ongoing and engineers are continuing troubleshooting efforts. SDSC Engineers are investigating API and service issues in the SDSC Cloud Compute environment. This may prevent users from deploying new instances or un-shelving. Additional information will be …
SDSC Incident Notification – M52 top-of-rack switch reboot – 14:04, 12 Sept 2024
A top-of-rack switch in the M52 area of the datacenter was inadvertently rebooted. The services are back online at this time. Please contact SDSC Operations with any questions or concerns.
SDSC Outage Notification – Networking issue – 12:25, 6 Sept 2024
[update – 13:58] Engineers have isolated the issue and resolved the immediate problems. They are now working with network vendors to address the underlying cause of the outage – SDSC is investigating network issues affecting systems. Updates will be posted here.
SDSC Network Outage / Emergency Maintenance – 15 Aug 2024, 10:00
[Update 15:20] Network appears to be stable — please contact SDSC Operations with any concerns or questions. Networking staff continues to work with vendor technical support. [Update 14:47] Hardware swaps have caused unexpected impact to other networks. The networking team is still investigating and working with vendor technical support. — SDSC Networking is investigating a …
SDSC Outage Notification – Core Firewall Connection Issues – 12 Jun 2024, 0515
[UPDATE 0610] Firewall issues have resolved. Please contact SDSC Support with any continued interruptions. — SDSC Engineers are aware of connectivity issues involving core firewalls. Many services such as hosted websites and managed systems may be unavailable. We will update with more information.
SDSC Network Interruption – 18 May 2023, 21:00
[Update – 22:05] Upstream network service providers have reconfigured networking and the issue appears to be resolved. Please contact SDSC Operations with any questions or concerns — SDSC Network Engineers are investigating a loss of external network connectivity. More information will be posted here.
SDSC Incident Notification – VM Slowness – 12 Apr 2023, 13:30
(Update – 22:52) Disk load has been reduced by rebalancing array use. Vendor support has been contacted for additional input. — Around 13:30, 12 April 2023 hosted VMs started to experience slow disk access. Engineers are investigating and more information will be posted here.