My EBS Install Saga

(Yes, it is sad that I am writing this on New Year's Eve...I'm on a mission at this point and nothing will stop me).

So I've been trying to follow along with John's EBS Install series...and by trying to follow along I mean I'm still on step 0 (zero).

First, it was a lack of space. 300 GB to be exact. So I went out and bought a 320 Seagate expansion drive. I can't remember that far back, but I think other things came up and I had to requisition that space for other projects.

A couple of weeks ago, I found a 1 TB external drive (USB 2.0) for $100. Couldn't pass it up. Sweet...now I'm all set to do this thing.

Somewhere, somehow, I removed all the carefully managed installation files. Couldn't find them. So I started over. A quick reminder, it's 46.25 GB of download, which takes a total of about 8 hours...even with 25 Mbps down. I am really thankful that Kris Rice doesn't do EBS...he'd be in for a world of hurt.

OK, I'm downloading the files and unzipping them simultaneously...to save a little bit of time. Get everything downloaded and unzipped...this might be where the fun really begins.

If this is your first time, I'm using VirtualBox. The rest of the setup is like this:
-- Dell XPS M1530
-- Intel Duo Core T1555 1.83 Ghz
-- 3 GB RAM
-- 1 TB Seagate External Drive (USB 2.0)
-- Ubuntu Karmic Koala (9.10)

Not the most powerful machine in the world...but decent enough for 99% of everything. Of course by everything I mean everything but EBS.

The first time I created a 300 GB Dynamic HDD. That's the one I downloaded and unzipped all the files too. Upon completion, it had grown to 68.5 GB. OK, let's run it. Kick off the rapidwiz script, run through the first couple of screens and then it does a system check. Oops...there's not enough space. John did say that 350 GB would be preferable...I obviously didn't listen.

OK, so I could either expand that HD or shrink it and add another. Expansion is not possible (yet) in VirtualBox. I tried cloning (not sure why) and still ended up with a 300 GB disk of which 68.5 GB was used.

I then tried to use zerofree so that I could compact it (again, I have no idea why I was trying to do this). Those attempts were very unsuccessful because I couldn't figure out how to make the HD read-only. I did figure out a way later...just add a bad line in /etc/fstab and then you can run zerofree. What was the poing of compacting it though? No idea.

In one of the more recent versions of VirtualBox (I think 3.0), you could add a second HD. At least that's when I noticed that ability. So I created a new Dynamically-expanding 300 GB HD and attached it through the VirtualBox GUI.

Now I was stuck trying to figure out how to mount the stupid thing. Mr. Brad Tumy to the rescue.

Kick off the rapidwiz script, get through all the setup screens and it's installing...I left it over night...and one more night...78 GB had been written to disk.

I killed it.

How about just creating a 300 GB Fixed HD? Tried it...ran for 24 hours...about 60 GB had been created.

I killed it.

Maybe I could use Shared Folders. I consulting The Twitter and was reminded of my recent fun with Shared Folders.

Hmmm....How am I going to do this?

Then an IM with a colleague where he asked about The Cloud. It would be nice to have a test environment, as we are all remote, that we could share. I gave him 2 suggestions, I could buy a beefy machine and host it here at home or I (we) could try to do something in The Cloud.

That got me thinking...The Cloud? Hmmm...let's see if I can install OBIEE there.

I was immediately sold when I downloaded the OBIEE software (1.5 GB) in like 3 seconds (14 or 15 Mbps). I got OBIEE installed and running on Windows...but when I was creating the Instance, I noticed some pretty beefy boxes.

Maybe I could create the 300 GB VDI in The Cloud?

Yes, absolutely!

So I fired up a 20 ECU Windows Server Datacenter (2008?), with 8 dual core CPUs and 7.5 GB of memory. It took 2 hours to create the VDI.

Sweet!

Now...how in the hell am I going to get that 300 GB down to my machine? Since my local drive was attached, perhaps I could just drag and drop it there. Nope...didn't like that.

I'll zip it up!

Using 7zip, I was able to compress it to a very modest 360.9 MB. That's got to be close to a million percent compression right?

Downloaded that via IIS (had to set that up as well) and began to unzip it. First 5 GB took about 5 minutes, so I'm thinking 5 hours. Then it started to crawl. I was at 75 GB after 10 hours which was going to put me at 40 total hours. But it looked like it was exponential...that 40 hours probably would have ended up around 1 million hours. I don't have that kind of time.

I killed it.

Consulted The Twitter again.

@TedC and @jtdavies offered up some suggestions. @TedC told me to use Amazon S3. @jtdavies told me to split it up and use rsync.

Being incredibly impatient and quite possibly very stupid, I "split" the difference. Using my 400 GB attached Volume, I decided to use The File Splitter to split the files. 10 MB chunks. 3000 some odd files. No worries, I'll just zip up every 100 or so, download and unzip.

Until I ran out of room.

*%##@$!!~!@#$%#$#$**&%(*##@@!!!!!!!

Shoot me...now!

I created a second (well, third) volume, 315 GB, which would be the target for the split files. Attached it and began the split...again. This time, 10 GB chunks. I wasn't worried about the size as they compress very nicely. Two of those equals about 24 MB of download.

So I'm almost done with that...about 95% complete. I just have to download 2 more zipped files.

The funner part will be getting those put back together...if it's even possible. I've read up on cat, I just hope it comes through.

If you have any suggestions...or you want to come over to my house or remote into my computer and help me...that would be awesome! :)