Monthly Archive: February 2016

SDSC Outage Notification – Commvault – 21:00, 29 Feb 2016

[Update: 23:06, 29 Feb 2016] Deduplication stores have been sealed and systems requiring the deduplication are running backups. At approximately 21:00 on 29 Feb 2016 the Commvault media agent named ‘cvma3’ hung and was power cycled. This caused jobs which were running to pause. There were some jobs which required manual restarts. Some jobs utilizing …

Continue reading »

SDSC Emergency Maintenance – Project Storage – 18 Feb 2016, 1800

SDSC will be applying critical patches to the Linux Project Storage systems tonight, 18 February, 2016 starting at 6pm. This maintenance will require a reboot of all systems listed below and is expected to take no more than 60 minutes. This is in response to a new critical vulnerability announced Tuesday, CVE-2015-7547. HOTEL: ps-006:/ps-data/cmmap ps-009:/ps-data/ngbt …

Continue reading »

SDSC Emergency Maintenance – Linux patch/reboot – 17 Feb 2016, 1900-2300

[Update: 18 Feb 2016, 00:29] All systems are online. Please contact SDSC Support with questions or concerns. [Update: 17 Feb 2016, 23:14] Most systems are back online. Some of the systems are still in the process of rebooting, but should be completed shortly. Once complete a final update will be posted. Colleagues: SDSC will be …

Continue reading »

SDSC power issues

As of 1230 all issues have been resolved and all systems are operating as expected. Jeff Filliez SDSC Data Center Manager

SDSC Planned Maintenance – Kerberos rekey – 11 Feb 2016, 07:55

At approximately 0755 on 11 Feb 2016, SDSC will premptively rekey the primary KRBTGT within the SDSC realm. Moving to a stronger encryption method will make impersonation attacks against the SDSC kerberos realm much more difficult. This change will invalidate the existing krb5 credential cache and require a new ‘kinit’.

SDSC Outage Notification – Cloud Storage & Compute – 4 Feb 2016, 12:39

[Update, 13:19] Service has been restored. A 10Gb cable was inadvertently removed which provided service to the load balancer. That cable has been replaced. The loadbalancer which provides access to both SDSC Cloud Storage and SDSC Cloud Compute has gone offline. Engineers are investigating.

SDSC Planned Maintenance – 16 Feb 2016, 17:00-21:00 – Windows, Linux, Solaris, NFS

– “cavern” NFS file sharing unavailable during fileserver reboot – NFS file sharing from some project storage nodes – SDSC e-mail temporarily delayed (UCSD Exchange service will not be affected) – Hosted web servers briefly unavailable – Virtualized hosts may pause – Linux systems patched without reboot – Standard Windows patching Additional Detail – Solaris …

Continue reading »