House, Lies and Sysadmin
In the TV show House, the eponymous protagonist often repeats the idea that patients lie.
"I don't ask why patients lie, I just assume they all do."
If you're in support, you could do worse than transpose that dictum to your users. Of course, I'm using "lie" very loosely. Often the user does not set out to deceive; and yet, sometimes, I can't escape the feeling that they could not have done a better job of deception had they tried.
Take a case that occurred recently. A user wrote to me. I'll call him Gordon.
Hi Henry, I'm embarrassed to say that I seem to have lost (presumably deleted) my record of email for the AQERA association while re-organizing email folders earlier this year. The folder was called Assoc/AQERA, and I deleted it around May 18 of this year. Would it be possible to recover the folder from a time point shortly before that date? Thanks a lot Gordon
This was written in early September. That's Gordon's first bit of bad luck. We keep online indexes for 3 months. Do the math. A couple of weeks earlier, and I might have got back his emails with little trouble - and this article would never have been born.
Hi Gordon, The difficult we do today. The impossible takes a little longer. Your request falls into the second category. Cheers, Henry
Some customers have unrealistic expectations. Gordon is remarkably understanding:
Err, does that mean not possible, or that it will take a while? I understand it might not be not recoverable, and it's my own fault anyway, but I need to know.
Who knows what might have happened under normal circumstances? In this case, there were some unusual elements. I'd started work at this organisation May 10. Less than 2 weeks later, our mail server had crashed. When we rebooted, one of the disk drives showed inconsistencies which an fsck was not able to resolve (long story). We had to go back to tapes and recover to a different disk on a different machine and then serve the mail back to the first machine over NFS.
The disk drive which had experienced the inconsistencies was still attached to the mail server. I would often mount it read-only to check something or other. In this case, I checked to see if the emails he wanted were on that drive. They weren't. The folder wasn't there.
There were other folders in the Assoc/AQERA family (Assoc/AQERA-2010, Assoc/AQERA-20009, etc). But no Assoc/AQERA.
There was something else that was fishy. I can't remember the exact date of the mail server crash. Let's say it was May 24. When we recovered from backup, we would have taken the last full backup from the first week in May. In other words, we would have restored his account back to how it was before May 18. Without meaning to, we would have "undeleted" the deleted folder his email claimed.
For many reasons, after the email system was restored, users did not get access to their historical emails until early June.
So something he's telling me is misleading. Either he deleted the folder before the beginning of May or after the middle of June. Or, perhaps, it never existed. If I had online indexes going back far enough, I could answer all questions in a few minutes. Without them, I'm up for really long tape reads. My job is not about determining where the truth lies, it's about recovering the emails.
I wrote to Gordon, explaining all this and added:
So, before I embark on a huge waste of time for both of us, I suggest we have a little chat about exactly when you think you deleted the folder, and exactly what its name is.
I suppose we had the discussion. A day or two later he wrote again:
Here's another thought, that I hope will save work rather than making more. I didn't notice that I was missing any mail until about 1 September. At that time I went to my mailbox Assoc/AQERA-admin, which should have in principle contained hundreds of emails, but actually contained only one. If there any recent backup in which Assoc/AQERA-admin contains more than one or two emails, then those are the emails I'm after.
And that was all I needed!
Stupidly, I followed the suggestion in his last paragraph - and drew a blank. But then I went back to the failed disk and searched for Assoc/AQERA-admin (rather than the elusive Assoc/AQERA).
I have found a place with the above directory which contains 843 emails dating back to 2003. I have restored it to your mailbox as Assoc/AQERA-admin-May21 (in your notation). I hope this helps.
And this is how I know my customers love me:
Fantastic!! That's everything I was after. Recovering this mail will make my job as AQERA president much easier over the next year. Many thanks Gordon
It's why I put up with the "lies" and inadvertent misdirections. I have an overwhelming need to be appreciated. At heart, I'm still at school craving any sign of the teacher's approval. It may have made me a superb student. But what does it say for my self-esteem? Who's the more together human being? Me? Or my son, whom I tried to shame into doing better? He didn't care tuppence for his teacher's approval. Or mine. And yet, here he is, doing third-year Mechatronics Engineering at uni.
Henry has spent his days working with computers, mostly for computer manufacturers or software developers. His early computer experience includes relics such as punch cards, paper tape and mag tape. It is his darkest secret that he has been paid to do the sorts of things he would have paid money to be allowed to do. Just don't tell any of his employers.
He has used Linux as his personal home desktop since the family got its first PC in 1996. Back then, when the family shared the one PC, it was a dual-boot Windows/Slackware setup. Now that each member has his/her own computer, Henry somehow survives in a purely Linux world.
He lives in a suburb of Melbourne, Australia.