The past several weeks have been very frustrating ones at work, not just for me but for one of my clients as well. What with one thing and another I’ve been averaging six long days every week. That’s a lot of time in front of computers.

It started with a client’s move to a new office space. From the client’s point of view it went well enough, but I had to work my ass off to make it so. Due to some organizational changes on their side my input on their IT structure was cut off and I had to work with the result of decisions made by non-technical people, who used criteria that I have not yet been able to decipher.

At any rate, the move itself went well, but not long afterwards they began to have odd networking issues, mainly people intermittently not being able to print to certain network printers. While I was still trying to figure that one out, their Exchange email server began to go offline.

The problem with the email server was that the database size had grown to the limit Microsoft had set for that particular server license, and it was going offline periodically as punishment. Part of the networking problem was that they were out of user licenses. (They use a lot of seasonal workers, swelling the user list, which then shrinks down again, so we typically only have to add users there about once a year.)

A big part of the blame for those things falls on me. I screwed up. Partly through distraction from the several other projects I was working on at the time, partly through disgust with the way decisions were being made (or not made) at that organization (and quite specific indications that any input from me would be ignored anyway), I wasn’t paying close enough attention to those things.

The email problem was quickly diagnosed and it was easy enough to keep things moving along while waiting for a new mail server to come in (which had originally been planned for a month earlier, right after their move). The network oddities, however, had me pulling out my hair and took much longer than I would have liked to track down. It turned out to not be just a matter of the user licenses, but also the switch that one of the network printers was attached to, a hardware failure with the printer itself, a bad network switch, and a bad cable connecting two switches back in the server room. Six different intermittent problems, in other words, all showing similar symptoms, and it took a while to track them all down.

Shortly after getting all those problems finally exorcised, the new mail server came in. I started setting it up and the new software told me that I had to apply a patch to the old server before setting up the new one. OK, no problem, it’s a routine patch.

Except this time. That patch blew the old server from Hell to Christmas. I worked furiously all night to try and get it back up and running before people came in to work the next morning, and was only partially successful. I started installing the patch at 9pm and it was 3pm the following day before I had everything fully up and running again.

(Then the next day I had to go in and try to install the patch again. A process I faced with some trepidation, after my ordeal of the days before, but I did it.)

The point of all this is that the client has had several days recently when their computer infrastructure wasn’t working worth a damn. There are some lessons to be learned here.

First, the client is not, so far as I know, particularly unhappy with me. I am sure they are not thrilled about the problems, but I haven’t taken too much of a hit because of it. The key here is that I’ve been working with them for a number of years now and the systems have generally worked quite well, and I worked my ass off trying to resolve the various problems as quickly as possible. Also, because of the way I had things set up, the impact of some of the problems was minimized. During the hours in which their mail server wasn’t talking to the outside world, for example, they could still use the Internet and the rest of the network, and inbound mail was queueing up on the email gateway so nothing was lost.

Everyone screws up now and then, and things break sometimes. If you consistently do a good job for your clients (or your boss, though that’s more problematical), you’re building up credit that will serve you well when something goes wrong. Relatively major problems at rare intervals are more tolerable than constant annoyances.

Second, I had to go outside the organization’s chain of command and bother some people whose job it wasn’t (but who were willing, much to their credit, to make some noise and make things happen) in order to get some decisions made. As a result, we’ve done some reorganizing and clarified the lines of communication and generally made it a lot easier — I hope — to get things done in the future.

When everything is going well, there’s no motivation to reorganize and try to make things work more smoothly. If you bust your tail holding everything together with chewing gum and duct tape, no one is going to notice, because you are holding it together. Things that work well are invisible, and eventually taken for granted. That’s not a bad thing, usually. A company’s computer network, for example, shouldn’t call attention to itself. But, like an economic downturn that shakes out weak companies and makes the economy stronger in the long run, sometimes it takes a failure somewhere to bring problems to light and provide the motivation for a proper fix. Every failure is a source of lessons to be learned.

Looking at all of this work from another angle (and all that fuss and bother with that one client was only part of it; there were lightning-blown computers, server migrations, and so on and so forth), I’m bloody exhausted. My wife asked me if I was going to bill the client for all of the many hours I spent fighting that crashed mail server. “Most of them,” I told her. I’ll probably give them a bit of a break, but for the most part, they’re going to get billed what I worked. I didn’t make any mistakes on that one; it was just a bad break and unfortunately the nature of computers is such that every now and then you’re going to have to deal with a crash. The big wad of hours I’m going to bill as a result is, to me, like collecting on life insurance. I wish it hadn’t come to pass, but since it did I’m not going to turn down the money.

Speaking of which, the money has been pretty good these past couple of month, but there comes a point when I just want some free time to spend with my family and to get some rest. As I believe I’ve said before, no on ever lay on their death bed wishing they’d spent more time at the office.

Minor disasters in the workplace can sometimes be made to turn out well, even an improvement over the pre-disaster state. The price can be high, though, for everyone involved. Sometimes there’s nothing for it but to throw yourself into the breach and do what needs to be done, close up the wall with the bodies of your English dead, and afterwards collect your pay. But no amount of money is worth not being there while your child grows up.