Google

Monday, May 05, 2008

Who Needs Sleep??

Apparently, not me.

Server upgrade went smooth. We were messing around with some web pieces for awhile but things were good. We called it a success around midnight and I went to bed.

Then my pager goes off. At 12:40am. Things had slowed to a crawl and for whatever reason the server was restarting itself. WTF?? Get vendor on the phone. Do some looking and don't see anything in the log files. We think maybe this is a fluke and go back to bed about 2:30 am.

Pager goes off at 2:57 am. What is going on? I get up again and gather the troops. We are now getting everybody who was on earlier back in. People are pouring over files and digging deep to try an figure out what is going on. At about 5:00 am we decide that we have to back out. We can't go into a Monday running as we are. The DBA and sys admin guys are geniuses and determine that we only have to do a partial backout. We can stay on the same database cluster and just move back to the old application server cluster as that is what is throwing the errors and restarting. It takes about an hour to form the plan and begin execution. By about 6:30am we are back in business. We have our key areas doing some testing for us and all seems well. We hang out and monitor until about 7:15 am. We decide then that we will do a status check at 8:30am.

I try to lay down and get a nap but the dogs have other ideas. They want to go out and eat and such. By time I take care of them I figure I may as well stays up and get some work done.

At 8:30 am, everything is still going smooth. YEAH!! Music to my ears. I cancel my two afternoon meetings and go back to bed. And there I lay... wide awake. I am now running on so much adrenaline that I can't sleep. After about an hour of tossing I finally sleep.

The garbage people (2nd cousins to the Village people) wake me up around 2:00pm. The dogs were restless and needed to get up. So here I am. I figure that if I'm going to sleep decent tonight I need to get up now. I have a headache and can barely keep my eyes open. I am hoping that a shower will help.

Peace Out!!

1 comment:

OMW said...

I won't be sleeping tonight either. Maybe we can 'stay up all night' together??????