Sunday 4 April 2010

Interesting Daylight Saving Time Problem

I was teaching an Oracle Database Administration class last week and ran into an interesting problem involving the change to Daylight Saving Time  (you did know that it's Daylight Saving, not Daylight Savings, right?)


For this class, we usually install just the base version of Oracle Database 10gR2 (10.2.0.1.0), because we're on a private network behind a firewall and we don't cover how to upgrade and patch Oracle until the advanced class. And then we show Enterprise Manager Database Control.


Except that this week, it wouldn't come up. We got Java errors, and the home page said that the status of the database instance couldn't be determined. When we tried to start the DB Console service (this is on Windows), it said it was already running; but when we tried to shut it down, it said it wasn't running. Very weird.


In doing a web search for the Java messages (Google is my friend), time and again we were advised to check how our time zone was set. That seemed odd; you wouldn't think that would matter. But it does: the EM agent checks to see that its time zone is (a) valid, and (b) configured is the same as the server.


Enter Daylight Saving Time. In 2007, the US and Canada changed the start date of Daylight Savings Time to be in mid-March (March 11 that year) instead of the first Sunday in April as it had been. This required patching Oracle products with a new timezone file, so they could properly detect the change. (The document ID on My Oracle Support is 359145.1).  This year, we changed to DST on Sunday, March 14.


The class was held the week of March 29, and the unpatched 10.2.0.1 software still thought we should be on Standard Time until April 4. So the EM agent wouldn't start. Had we done the class a week later, or three weeks earlier, this wouldn't have been a problem; and in fact we have done this class many times since 2007 and never hit this issue because the timing wasn't JUST right.


So we did an unscheduled lab to download 10.2.0.4 patchset and apply it, and EM came right up.


Stuff like this underscores the importance of patching your databases, even in a training environment!


There's something to learn from a training perspective here too. We often think that if things go wrong, our students will lose faith in us and think we don't know what we're talking about. But in this case, I was able to model the proper response a DBA should take: stay calm, research the issue, test. I've had students comment in the past when things went wrong that seeing how to resolve such issues was an unexpected bonus of the class. This class felt the same, they enjoyed the "detour" and the problem solving.



No comments:

Post a Comment

Note: only a member of this blog may post a comment.