Monthly Archive: August 2018

SDSC East Data Center Cooling – 30 Aug 2018, 11:00

(Update 13:55) All air handlers have restarted and the room temperature is normal. Please contact Operations with any concerns. —— The air handlers in the east data center did not properly restart after the power outage. As a result, there are higher than normal ambient temperatures throughout the room. Engineers are working to restore operation.

SDSC Outage Notification – Power Outage Resolved – 30 Aug 2018, 08:15

Update 8/30/2018 10:00 AM PT: A general power outage occurred this morning, Thursday, 8/30/2018, at SDSC around 8:10 AM PT. Power was restored by campus FM around 9:40 AM PT. Individual systems in the data center may need to be powered back on if they were affected by the outage. Generator-backed systems should have remained powered …

Continue reading »

SDSC Post Incident Report – VMware Outage, 15 Aug 2018

On Wednesday, 15 August 2018, at approximately 1pm, the primary VMware cluster at SDSC experienced a complete outage. During the outage all VM guests were unable to access disk resources, and eventually needed to be powered off to allow cluster recovery. At 10pm the cluster was stable, at which point all VM guests were placed …

Continue reading »

SDSC Outage Notification – multiple networks in building/datacenter – 15 Aug 2018, 13:15

[Update – 22:59] All guests are online and appear to be acting normally. Please contact support with any questions or concerns. [Update – 22:00] Engineers are slowly booting guests and monitoring stability of the VMware cluster. Additional updates will be posted once all guests are on and the environment is operating normally. [Update – 18:42] …

Continue reading »

SDSC Planned Maintenance – Linux Patch/Reboot – 21 Aug 2018, 17:00-21:00

On Tuesday, 21 August 2018 between 17:00 and 23:00, general system maintenance, including patching and reboots, will occur for the following systems. Systems will be unavailable during reboots. Please save all work prior to the 17:00 maintenance period. Updates will be posted on the SDSC status page at http://status.sdsc.edu. anchor athena awesomelabs bastiat beeker bolero …

Continue reading »

SDSC Outage Notification – Project Storage Hotel Node – 2 Aug 2018, 15:00

At approximately 15:00 on 2 August 2018 a single project storage hotel node froze and required a hard reboot. Following the reboots, some NFS clients may experience stale NFS file handles which a remount should correct. The services appear up and functioning. Admins of systems may need to be contacted to perform remounts. Please contact …

Continue reading »