Moving targets

Amongst other things my team issues SSL certificates to the University. Whilst much of the process is automated, there is a call and response routine we need to follow with our certificate provider that has to be dealt with by hand. Earlier this year the Heartbleed vulnerability caused us to work like demons to replace all impacted SSL certificates. At the time we were grateful that only some of the certificates we had issued were impacted.

Back in September my colleague Jon, who masterminds the technical side of this service, told us that SHA-1 certificates were to be deprecated on a rather aggressive time scale by Chrome with other browser producers falling in line with them in due course. So we sat down and worked out how we were going to handle this for the ~300 certificates we had issued that were impacted. our cert issuing authority was helpful, and after a few conversations they agreed to issue free of charge replacement SHA-2 certificates.

We crafted a few carefully worded announcements explaining the situation to colleagues across the University for dissemination firstly via our online news service. These were subsequently rolled into a web page. Jon built a test page for computer officers to check the status of their certificates, this also showed for individual web sites at which dates different warnings would be displayed.

The thing that worried us most was the call-and-response challenge needed to confirm the reissue of 300 certificates. My team looked wearily at me for clemency. Jon and I put our heads together, and we found a way of automating that side of the process as well for this one-off batch. Come mid-November we had agreed with our certificate provider that the reissues would be handled on one day, and we would not process any new requests on that day to avoid confusion.

Just after 9am my in box started to fill up with the challenge verification messages. I took my team out for coffee. Throughout the day there were bursts of these messages, and eventually it was done. We had a few queries from people who had taken over responsibility for web servers about how to install these on their systems, but mostly the air was full of the resounding silence that indicates a mostly happy University. Now all I need fear is the renewal anniversary, when at least some of these 300 certificates will need to be replaced as it will be a spike in our workload. Massive thanks go to Jon for handling the whole technical side of things.