[Update: 17:10] Hardware logs have been handed over to vendor support for review. At this time it appears that a hardware failure on the hypervisor node delayed I/O and caused the guests to automatically reboot on other hypervisors. At approximately 16:25 on 11 July 2020 the following systems rebooted due to a crash of the …
Category Archive: Emergency Maintenance
Emergency Maintenance announcements
SDSC Outage Notification – VMware guest outage – 22:00, 6 Dec 2019
[09:06 – 7 Dec 2019] Affected hosts include: actreal1 bahamut cycorephaser dev.mygeohub.org dev.purr.hubzero.org dev.smarteragriculture.org ea-repository elprado exec01.cdmhub.org exec01.communityhub.hubzero.org exec01.demohubzero.org exec01.dev.mygeohub.org exec01.help.hubzero.org exec01.mygeohub.org exec01.pharmahub.org exec01.smarteragriculture.org exec01.stemedhub.org gitlab grunwald16 itss-dynamics johndenver massive2 nexposevase osc-dev packages.help.hubzero.org qball3.sdsc.edu rsm-print rsm-print2 russell sc-dynamics sc-lansweeper sentinel-new stage.cdmhub.org web01.int.citscibio.org web01.int.extlearn.org web01.int.help.hubzero.org web01.int.vhub.org yellowfin [08:15 – 7 Dec 2019] Around 22:00 on …
SDSC Outage Notification – Network Uplink Outage – 18 Nov 2019, 08:48
(Update 09:40, 19 Nov) Crews repaired the two largest broken fiber bundles last night between 20:00 and 23:30. All severed links should be operational — please contact SDSC Operations if you continue to notice connection issues. (Update 16:48) Crews continue to work to repair the largest broken fiber with an anticipated completion of 01:00 on …
SDSC Outage Notification – East Building Networking – 10 Oct 2019
The SDSC East Building is experiencing intermittent network outages. UCSD Networking and Datacomm is aware and working on the issue. Users in the East Building may see outages to wired and wireless networking.
SDSC Emergency Network Maintenance – West/Central/East Building wifi and East Building office networking – 10:00, 30 Aug 2019
To resolve ongoing networking slowness in the East Building, SDSC network engineers will reload a central aggregation switch this morning at 10am. The reload will take between 5 and 10 minutes, during which the wifi in all buildings and office physical connections in East will be unavailable.
SDSC Emergency Maintenance – USS – 24 June 2019, 18:00-19:00
[ Update – 18:24 ] The maintenance is complete. Please contact support with any questions or issues. On June 24th 2019 at 18:00 a security patch will be applied to the Universal Scale Storage. The system will be unavailable briefly during the patching process.
SDSC Outage – M47 network switch – 5/24 18:00
[update 22:16] The new switch is online and systems appear to be recovered. Please contact SDSC operations with any questions or concerns [update 20:32] engineers are on site swapping a chassis. No recovery ETA at this point The top of rack switch in data center rack M47 is offline. Some data center hosted systems are …
SDSC Outage Announcement – HPC, Commvault, Cloud resources unavailable – 6 May 2019, 18:00
[Update – 11:18] Networking service has been restored and appears stable. Please contact support if you continue to experience any issues. More updates will be posted once a cause is determined. [Update – 07:53] Commvault services are enabled and working — file restores are successful. [Update – 07:42] The reboot did not fully correct the …
SDSC Emergency Maintenance – Apache HTTPD update / website restart – 15 Apr 2019, 17:00-18:00
[Update – 17:34] All servers are complete. Please contact support with any concerts or questions [Update – 17:21] awesomelabs, artichoke, ciabatta, and nsgr are complete. Engineers are still working on the other hosts. — SDSC will apply critical Apache HTTPD webserver patches to the Linux environment tonight, 15 Apr 2019, starting at 17:00. These updates …
SDSC Emergency Maintenance – Apache HTTPD update / website restart – 11 Apr 2019, 17:00-18:00
[Update – 17:25] All services have been restarted. Please contact support with any concerns or questions —- SDSC will apply critical Apache HTTPD webserver patches to the Linux environment tonight, 11 Apr 2019, starting at 17:00. These updates address recently announced security vulnerabilities related to CVE-2019-0211. The web services will require a restart causing a …