Fun with Deployments

I'm sure many of you have had deployments such as this, but it was my first.

The day started at 2:00 AM. Joy.

In the office a little after 3 (40 minute drive) and the place is bustling. There are about 10 people in the office already (we have 50 employees) and 2 on the phone.

Mr. DBA had a whopping 1 hour of sleep.

I managed to get about 4 as I just feel asleep putting LC (not Lewis Cunningham, Little Chet) to bed.

To summarize the deployment, massive changes to our core OLTP system which needed to be propagated to the BI/DW environment. We started meeting/planning for this back in December as downtime was required.

As part of the preparation, a code freeze was implemented last week as well as a deployment (fixes/patches) freeze.

Before leaving yesterday, I disabled the production OLTP jobs that would run during the deployment window. I then disabled all jobs on the BI/DW side.

Just before deployment of the database changes, Mr. DBA created a restore point.

We set up a 2 hour window to deploy and validate. Deployment of the database changes took about 15 minutes. Validation took a little over an hour. Once the call that we were backing out was made, I began work on the BI/DW side.

I had DROPped, ALTERed and CREATEd about 200 objects. Everything went smoothly (3 or 4 practice runs). I turned all the jobs back on.

I then began copying the 187 OBIEE reports that were affected over to the production environment. I know there are better ways, or at least I've heard of them, but I was going with what I was sure of. Copied the old reports to a new folder and began replacing them.

Once I was done with that, I went through my checklist of changed reports to make sure they were working (at least from a functional standpoint). Nine of them were still broken. I made fixes to those and moved forward.

I then fixed and validated our Financial reports because we have a daily 11 AM deadline.

I did however forget to include a few grants from objects I DROPped and reCREATEd. I fixed a few of them in the morning then notified (and apologized) the business that they may have issues copying the Oracle (we don't really have any others) DBA group of the objects that may be affected.

My piece from start to finish took about 4 hours. I was done* by 9.

Had lunch with my parents (they bought!) and brought them back to meet everyone. I was laughed at (nothing new there) for bringing them in. I think my Mom cried too (might be an only child thing or the fact that it took me 10 years to graduate from college so she thought I would be a lawn man forever).

I'll still have work to do on the reports as the Business folks begin to validate, but overall it was a big win. The planning and implementation on this deployment was excellent and I was proud to be a part of it.