Frustrating computer project, or where I’ve been

Ironically, Michael Buble’s Home was playing as I start this entry, not at the end of a trip (when I really need to go home), but at the beginning of it. The last week or so has been so busy I haven’t hardly had time to breathe. While it has contained the usual getting-ready-for-work stuff, and, of course, Christmas (and all that goes along with that), the thing that really ate up a lot of my attention was a set of computer projects mostly revolving around Edi. A part of me thinks Michael Buble needed to write a song called “Away”… 😉

The whole computer project started with Edi wanting a new laptop. Lately, I have been in the habit of giving her hand-me-down laptops. I generally get a new laptop every two years or so, often around Christmas time (the sales are good). If you averaged out my last 6 laptops, they’ve lasted about a year. My most recent, a Toshiba U305, was about one year old so I was not ready to get a new one yet. Without getting into the history of it, Edi was on the one that was two laptops earlier (i.e. ~3 years old). I decided instead of getting her a brand new one, I’d get a new one for myself and give her my now-only-one-year-old one. She’d have that one for a couple years, making it about 3 years old when I’m ready to move on from this one. Hopefully that plan will work.

I got a Toshiba U405 to replace the U305. The U405 is more or less the successor to the U305 – it’s their 13.3” laptop and the size of it is perfect. I looked at competitors and nearly bought one (an HP) but wound up back with this Toshiba, and I’m glad I did. I like it (so far… it’s less than a month old still). Two factors, though, made this entire process infinitely more complicated than it should have been.

First, the new Mandriva 2009.0 distribution is really not that great. They did a pretty good job with 2008.0 and I heard 2008.1 was excellent, but I got the kinks ironed out of 2008.0 and wound up very happy with it. But with my new laptop, I figured I’d upgrade to 2009.0. The KDE desktop is transitioning to version 4 which I heard was quite different. I thought I ought to try it.

Moving to 2009.0 wound up being a bigger challenge than I expected it to be. In a nutshell, I hated KDE 4. After maybe less than an hour of using it, I decided KDE 4 and I weren’t ready for each other. I installed the KDE 3.5 package (thankfully Mandriva made that mostly easy) and was back in familiar territory. That said, 2009.0 is buggier than previous releases, and in some ways that are important to me (WiFi being one of them). After quite a bit of time ‘fighting’ with the software, I got the essentials working and committed to keeping what I had, despite the painful process getting it all working.

All that said… in defense of Mandriva (and gnu/linux in general), once I got everything working correctly, it just plain works. Microsoft’s stuff seems to be just the opposite. It works right out of the box, once, and then becomes less and less reliable over time. With linux, unless you do something to break it (that’s possible with any OS), it’ll be stable and reliable. That has been the case for me all along, even with the buggy 2009.0.

The next complication came when Edi asked me to help me get her moved in to her new (my old) laptop. She had been using Windows on her laptop all along. When she first started to do that (3 years ago or so), I warned her about two things. First, I wouldn’t be able to be ‘tech support’ for her laptop. Second, it was her responsibility to make sure she kept all her data on the office machine. Needless to say, she wound up with out-of-sync data on both machines. I told her I’d give her my laptop with a fresh re-format and install of the factory software (Vista + bloatware) but that’s it. That’s when she asked what it would take for me to support her laptop again.

Enter my backup nightmare.

As I examined the conditions, I concluded that I needed a solution that would meet the following specifications:

1: Provide a nearly identical user experience on both machines (this necessarily requires syncing pretty much the whole home directory, including config files, etc.). Obviously the different hardware would be different (i.e. no battery on the office machine, no scanner on the laptop), but otherwise they need to be identical.

2: Allow login from either machine.

3: Keep a complete and usable set of data on the office (server) machine. (i.e. the laptop can’t be the ‘master’ data set).

4: Allow use of the laptop even when it’s not connected to any network (i.e. no using it simply as a terminal).

5: Keep the system simple enough for Edi to use.

These specs set up a need for bidirectional ‘destructive’ backup (non-destructive backup would result in a never ending cancerous growth of garbage on both machines). I’ve looked at various backup and mirroring solutions and haven’t found a brilliant, safe, and inexpensive way to do that on machines that might be periodically disconnected from each other. That led me to think I might want to engineer such a solution myself (I should have just offered to help her back up her files and moved on with my life… hah!).

I’ll leave out the whole train of thought (I know, this is way too long already), but here’s what I came up with. Only ‘allow’ login on one machine at a time. Upon logout, destructively back up to the other machine. Of course, under these circumstances, it might be necessary to login when the other machine isn’t available (i.e. at home, with the laptop powered down, or away from home from the laptop). Consequently, I have to allow login when the machine can’t connect to ensure the other machine has been backed up and logged out.

So I embarked on writing a set of scripts that will do all that work (backing up using rsync over ssh), and writing them in a way that Edi could use them. That wound up being more challenging than I thought. What I ended up with was a script that would run automatically on login, and would replace the usual method for logging out (i.e. click the start menu and select log out).

That script starts by recording the login time and checking that the other machine is available and gets its latest timestamps (the script records login, backup start, and backup complete times). If the other machine isn’t available, it simply warns the user that it can’t tell if it’s safe to use this machine.

If the other machine is available, it checks to see that it has not been logged in since completing its last backup (that means it’s safe to use this machine). It then displays the status to the user so they can decide what action to take next.

The options the program offers are simple: backup and logout, backup and don’t logout, don’t backup and log out, don’t backup and don’t logout. The backup options are self-explanatory (a start time is logged at the beginning and a finish time is logged upon a successful exit) and are not offered if it can’t connect to the other computer. The logout option (in either case) ends the script with a logout command. The don’t log out option (in either case) ends the script by logging a fresh login time. The only way to make the ‘other’ machine safe to use is to backup and log out, which records a finish time and logs out without recording a new start time. The other machine can then confirm that the last thing done was finish a backup.

This solution does have its potential pitfalls. It’s still physically possible to get the data out of sync and lose data as a result. And in those cases where something does get messed up, the procedure for fixing it can be a bit complex. I went over it all with Edi and she seemed to understand it.

Unfortunately, I wasn’t able to actually finish the project because of yet another irritation…

One of the requirements for this system to work is that the OS on both machines is the same. Because of the problems I had with KDE4 and 2009.0, I decided it’d be better to use 2008.1 on her laptop and the office machine. The office machine was running 2007.1, so I needed to upgrade it. I should never say never, but after my experience I’d like to think I’ll never start an OS install without reading the errata first.

The office machine has a two-drive RAID array that mirrors both drives. It uses the linux kernel raid capability. I’ve had it set up that way for years and it works great. Unfortunately, unbeknownst to me, 2008.1 had bugs in it that affected RAID (particularly during install) which ultimately made it impossible for me to install it. I was able to get around some of them, but the result simply couldn’t be relied upon.

What should have taken less than 30 minutes (install a new OS), turned into a 12+ hour ordeal as a result of the RAID problems. I wound up having to break out a fresh drive (I had a couple extras sitting around) just to be safe, and starting from scratch. After tinkering with 2008.1 and finally deciding that wasn’t going to work, I had to decide if I’d go back to 2008.0 or up to 2009.0. Could I install 2009.0 in a way that left Edi (and me, for that matter) happy with the results and not too buggy?

After several hours of time wasted with 2008.1, I decided to at least try installing 2009.0. That’s when I ran into my next challenge. Somehow, my 2009.0 DVD (from which I installed the OS on this laptop I’m using right now) wouldn’t boot. In fact, the whole DVD became unreadable both on the office machine and on my laptop. Unfortunately, I had only burned one of those DVD’s. And, unfortuantely, the ISO from which I burned that DVD was on the office machine, which was now not bootable.

Now, take a few steps back. Here I am, having worked on this project all together for more than a week. It’s the night before I go to work. I’m almost done and then I run into this. Not only am I not done, but now I’ve rendered her office machine not usable. My frustration, needless to say, was apparent.

At this point, I really didn’t know what I was going to do. All I had was a copy of the data on a non-bootable hard drive. I pondered installing 2008.0 or even 2007.1 again (I still had those DVD’s handy) and trying to get the data off that way. But enter my new laptop. I’ll spare the whole story but it turns out this laptop comes with an eSATA port / USB combo port. You can plug a USB cable into it, or you can plug an eSATA cable into it, which is a different form factor but otherwise identical to SATA. As luck would have it, I had ordered an eSATA cable which arrived that very day.

So, I decided to give that a shot. I grabbed the old studio machine (no longer in use), opened it up, disconnected the power cable from that hard drive and plugged it into the good drive from the office machine, then connected the eSATA data cable to it. Drat, the drive didn’t ‘pop up’. I looked, though, and the OS had picked up that the drive was physically present, but wasn’t mounting it. When I dug deeper, it was because the laptop didn’t have the RAID related tools to do it (duh). After installing that, I was able to mount it up. Whew! I copied the ISO over to my laptop and burned a new DVD. Thankfully, all that worked, and at that point, I really needed something to go right.

Finally, I got back to the office machine and started the install of 2009.0. For some reason, with the combination of drives I had in there, there were some hiccups (not related to the problems in 2008.1) in the RAID setup. I haven’t read all the manuals but it looks like they changed the way RAID is handled (at least by default) in a way that made it confusing for me. After I ironed that out, it was smooth(ish) sailing. I finally got it all running and managed to still get 4 hours of sleep.

The next morning (yesterday morning), I got up to get ready to go to work. My flight had been cancelled and I was rebooked several hours later. I thought that might be God’s way of giving me enough time to finish the project. I got close but just wasn’t able to get it done. Finally, I had to give in and go pack. I made my flight (though not with much time to spare) and at least Edi has her office machine working (now with 2009.0) and her old laptop to limp along until I get back home…

at which point I *will* succeed at getting this system running……

V-