[Update – 17:25] All services have been restarted. Please contact support with any concerns or questions —- SDSC will apply critical Apache HTTPD webserver patches to the Linux environment tonight, 11 Apr 2019, starting at 17:00. These updates address recently announced security vulnerabilities related to CVE-2019-0211. The web services will require a restart causing a …
Category Archive: Emergency Maintenance
Emergency Maintenance announcements
SDSC Emergency Maintenance – Linux patch/reboot – 17:00-23:00, 22 Jan 2019
[Update 20:58] All systems are patched and rebooted. Please contact support with any questions. —- SDSC will be applying critical patches to Centos 7 Linux systems tonight, 22 January 2019, starting at 17:00. This maintenance will require a reboot of all systems listed below. Please contact support@sdsc.edu with any questions or concerns. Updates will be …
SDSC Outage Notification – Firewall Network Issues – 26 Nov 2018, 14:35-14:50
On 26 November 2018 between roughly 14:30 and 14:50, systems behind the SDSC firewalls experienced limited network connectivity. Engineers have corrected the issue and systems are accessible. Please contact SDSC Operations or regular support channels with further questions.
SDSC Outage Notification – Project Storage ps-078:/ps-data/yeolab – 25 Sept 2018, 10:45
[Update – 11:19] The JBOD was brought back online and verified. The JBOD storage for ps-078:/ps-data/yeolab has faulted and the array is offline. Engineers are investigating and working to repair the array.
SDSC East Data Center Cooling – 30 Aug 2018, 11:00
(Update 13:55) All air handlers have restarted and the room temperature is normal. Please contact Operations with any concerns. —— The air handlers in the east data center did not properly restart after the power outage. As a result, there are higher than normal ambient temperatures throughout the room. Engineers are working to restore operation.
SDSC Outage Notification – Power Outage Resolved – 30 Aug 2018, 08:15
Update 8/30/2018 10:00 AM PT: A general power outage occurred this morning, Thursday, 8/30/2018, at SDSC around 8:10 AM PT. Power was restored by campus FM around 9:40 AM PT. Individual systems in the data center may need to be powered back on if they were affected by the outage. Generator-backed systems should have remained powered …
SDSC Outage Notification – multiple networks in building/datacenter – 15 Aug 2018, 13:15
[Update – 22:59] All guests are online and appear to be acting normally. Please contact support with any questions or concerns. [Update – 22:00] Engineers are slowly booting guests and monitoring stability of the VMware cluster. Additional updates will be posted once all guests are on and the environment is operating normally. [Update – 18:42] …
SDSC Outage Notification – Project Storage Hotel Node – 6 June 2018, 04:00-07:00
At approximately 04:00 on 6 June 2018 a single project storage hotel node lost connectivity with the attached disk. At approximately 07:00 a storage engineer reattached the disk pool and repaired the disk export. The services appear up and functioning. Please contact support if problems continue. The affected exports are listed below. ps-004:/ps-data/roylab ps-013:/ps-data/dsc ps-015:/ps-data/ucm_engineering …
SDSC Outage Notification – Citrix remote desktop – 12 Apr 2018, 18:30
The Citrix remote desktop and application service is currently not working. Engineers are investigating and this post will be updated as more information is gathered.
SDSC Outage Notification – 21 Feb 2018, 12:45-13:00
[Update 13:22] During routine system rack maintenance, engineers removed a power distribution unit to a network switch believed to be connected to redundant power. The switch was incorrectly single connected and lost power during this work. The switch has been reconnected to verified dual power and all other storage switches will be verified for correct …