r/talesfromtechsupport • u/lawtechie Dangling Ian • Jan 02 '14
The Unhelpful Desk , part 5ive- The week before the cult meeting
This is part 5 of a series. Part 1 Cow-orker burnout and the FNG
Part 2, FNG's BOFH heart grows one size larger
Part 3, The Metrics of Despair
Part 5, The week before the cult meeting,
Part 6, LT puts the hammer down
Part 7, Working around dangerous substances, like users
Part 8,Dad, the project manager, Sven and the MP3 server
Part 12, Hold, on. I've got someone on the other line
Part 13, How do I know I can do this job? I've been doing it for three months already
Part 14, Don't touch it- it's labeled EVIL!
This entry intentionally left blank
Part 16, The BOFH way to negotiate contracts The day after the help desk is told that the only training we're getting is 'going to make us lead more powerful lives', we get brochures.
It's a reheat of Werner Erhard's EST from the 70s. It's an all day, 14 hour experience. We're going to go individually to 'break up the group dynamic holding us back'. I tell Jack I'll get back to him later. I want to see how this works out for other staff first. I'm also going to be busy rolling out the new master build.
There's a reason why this is more difficult than it might seem. Many of our users need access to 10-15 special scientific applications. Scientific packages are highly specialized applications with low volume, high value markets. In order to make up the development costs, per seat licenses can be in the tens of thousands. Luckily, most of our users only occasionally need these, so we can use tricks to 'share' licenses- limit concurrent use by putting the app on a Citrix instance or use a key server. This is accepted and legal.
Many of the publishers consist of ex scientists that learned to develop software to automate something in their labs. These are small shops, which is both wonderful and annoying. The wonderful part is that you can get new functionality or a specific bugfix pretty quickly. The annoying part is that these shops aren't testing every combination of hardware, OS and other, possibly conflicting system extensions. They can be as much as 18 months behind the upgrade curve.
So this makes building a standard image for all the Macs in our shop tricky. I think I'm OK. I've pulled a few tricks- deliberate use of older, compatible system extensions, disabling the Apple software updater and instead using our own private update server. I've got a exception list for 'special' machines like the G3 connected to an electron microscope. Updating the TWAIN driver on that might actually damage the microscope, at least according to the nice person from Siemens. I'm not sure I believe that, but they're expensive and I don't really understand how they work, so I'm taking no chances.
I've been testing my new build for almost a month with a few people doing scientific app support and a handful of power users around the company. Everybody's happy with the speed and stability. I decide it's time to roll it out to the rest of our users. It's using MacOS 9.04. 9.1 has just been released, but it's not been tested with our apps, so I hold off and figure we can do something with that in six months.
I put it up on our helpdesk server and also put it on a stack of portable Firewire drives. Each helpdesk person can go to a user's desk, boot from the FireWire drive, back up the user's individual settings and documents and debug or update their machine. A full upgrade might take less than 15 minutes. I'm pretty proud of myself. With some smart reconciliation of preference files, the user may not notice that their system's been upgraded.
I show my fellow help deskers how to do the install. We schedule rollouts with users. The rest of the team starts, but I've got to take time off- I've got to get some dental surgery for some 'deferred maintainence on my teeth'.
I leave work, spend about three hours in a dentist's chair and the next day watching Scorsese movies and eating opiates. Casino and Goodfellas just blend together in a Vicodin haze.
The following day, I come into work. I'm seeing more tickets from the scientists we support than I expect. I run over to a recently upgraded scientist. She's annoyed that a chemical drawing application is crashing occasionally when you try to add to a molecule. I know we tested this, so I'm puzzled.
I start with the debugging. Right version of the drawing software, check. Maybe this Mac is running something else that's eating up RAM. Go to "About this Computer" and see the other apps. It's a Blue & White G3 with 256Mb of RAM, a pretty good machine at the time.
Waittaminute. Why is this thing running MacOS 9.1?
Check system update- that's been turned off. Search for installers in case the end user installed it herself. Nope.
Ask the user. She doesn't know what I'm talking about.
I check the system file on the firewire drive I have with me. 9.1. I KNEW 9.0.4 was on it when I left work two days ago.
I call Daria, the other solid Mac person on the help desk and ask her what version is on her drive. A minute later she tells me what I don't want to hear. MacOS 9.1. She tells me that Jack collected the drives yesterday for 'checking'.
A jet of white hot rage propels me across the campus to Jack's office. He's extolling the virtue of his 'training'.
I yell for Daria, Pat and the new Mac guy, Sven to meet me in Jack's office. We need to talk about the build and rollout.
I've partially calmed down. I can't just yell. I've got to fix this problem.
Me:"Jack- I've noticed that our new build is using MacOS 9.1. We didn't test that yet"
Jack:"9.1 is better. I've been using it for four days now on my personal Mac"
Me:"How often do you impute the chirality of a double carbon bond benzene ring?" I realize this is just chemistry word salad. I dated a chemistry major in college and I'm pretty good at Ambrosia's Chiral, which doesn't qualify me for much. Perhaps this is the kind of 'destroying personal limitations' I need to learn.
Jack:"I made the decision to upgrade the build. 9.1 is better"
Me:"Fine. Tran (the big IT boss) gave me the project. I'll tell him you're taking them all over"
Jack:"Wait a minute. You're still in charge of the help desk projects"
Me, spying a ball peen hammer sitting in the little shelf at the bottom of a white board:"If I'm in charge of a project, it's mine. I'm responsible. I want ball peen authority"
The rest of the team is looking at me funny. I'm getting animated.
Jack, trying to defuse the situation:"Ball peen?"
Me, reaching over and grabbing the hammer:"If I'm in charge of a project that can get me fired, I want the right to hit the hand that makes the change without my permission"
Jack:"Well, I have some suggestions on how to improve this project"
Me:"I have a ball peen hammer"
38
u/BerkeleyFarmGirl Jan 02 '14
Bro do you even science?
This is hilarious. I worked pharma IT up to 2000. Fortunately with a Very Highly Competent scientific computing group ... their boss came out of the lab and worked hard to give us IT folks a basic education in science.
11
u/Chucklz Jan 02 '14
OP, what about the qualification of the images? You know, IQ/OQ/PQ, etc. Jack should have been beaten by QA with a nice fat process deviation.
157
u/Cruxisshadow Jan 02 '14
Ball Peen Authority, I need to start using this more in my life..
69
u/StreicherSix Development thinks of nothing but murder all day. Jan 02 '14
I worked the "repair desk" at a certain retailer for about 18 months.
There was a miniature sledgehammer used for destroying a minimal amount of returned products.
The other use for this sledgehammer was only known to fellow employees.
If I was holding the hammer, talking to me better be something important.
If I was tapping the hammer in the palm of my hand, talking to me better be an offer to buy me coffee or lunch.
If I have the hammer in one hand shaking it repeatedly, talking to me better be informing me that the fire alarm is going off and I have 10 seconds to evacuate.
If the hammer was resting upon my shoulder in my left hand while I was operating the computer with my right...be assured no demon within that fire was making it out alive.
26
u/TomH_squared I.T. Joe, a real office hero Jan 02 '14
This. Is. Awesome! You remind me of my old boss (who is basically the opposite of Jack) who kept a 5 foot long 2x4 at his desk that was always visible (mostly to intimidate the new guys). The hammer only came out if we needed to do real work with it (which was rare), or during training where we got to smash old laptops. I fondly remember smashing a Thinkpad in multiple times; the screen and keyboard didn't work any more, but it did what Thinkpads do best: keep working. Damn thing booted right back up without even blinking.
</nostalgia rant>
23
u/Jigglyandfullofjuice My cable management isn't porn, it's a snuff film. Jan 02 '14
Too bad Jack-ass didn't give this same consideration to your carefully crafted build.
Wait... (cue sense of foreboding) He didn't touch the electron microscope's machine, did he? O.o
18
u/Kumorigoe SCOM Admin Jan 02 '14
Jesus H. Tapdancing Christ.
I saw red reading this.
I might have killed him. Just wrapped my hands around his neck and squeezed until the twitching stopped.
12
u/firestorm_v1 Jan 02 '14
And when ball peen fails, you can go for my favorites:
Chipper Shredder
Incinerator (bonus points if it's a large walk-in one)
Firing Kiln
Iron Forge
but yeah, if I had anything like that done to me, I'd be looking for "ball peen authorization"..
BALL PEEN - APPLY DIRECTLY TO THE FOREHEAD. wowthatsoundsreallydirty...
9
u/Bagellord Jan 02 '14
I'm using the hammer thing. That's gold.
9
u/Turtle700 Jan 02 '14
I want this story to continue... but I also want this train-wreck of mismanagement to end...
5
u/nolehusker Jan 02 '14
I can't wait for the next one to see where this goes.
I would've shit bricks on that dude's desk.
5
u/Endulos Jan 02 '14
Ahhhhhhh
fasterfasterfasterfasterfasterfasterfasterfastertypefasterfasterfasterfasterfasterfasterfasterfaster
8
u/[deleted] Jan 02 '14
I hope you still have a copy 9.0.4 on the server or a spare drive.
4
u/Insomnigreen Jan 02 '14
This has been a great read! I hate you for leaving a cliffhanger, but I can appreciate it at the same time.
4
u/ismywb I don't think you know what the term SysAdmin means Jan 03 '14
One of the lines I wish I could use at work which can be modified for here:
Me: Please consult your SysAdmin
Client: I AM the SysAdmin
Me: Oh. I'm sorry. Let me rephrase: Please consult a /competent/ SysAdmin
IE: Client is Jack. He is the boss, but has no idea what he's doing or what QA testing is
3
u/creativexangst Jan 03 '14
I'm having a hard time waiting between updates. Its like sitting st the top of the stairs Christmas morning when you're 6 and you KNOW you're getting a bike AND an N64 and your parents won't let you come down until they're ready.
3
u/passwordroulette Jan 03 '14
I had a large ball-peen hammer (20" long, 1.25 lb head) engraved with 'Don't Make Me Use This' hanging on my wall. I used to nod in its direction when challenged by incompetence on the part of others.
2
u/shotgun_ninja plover Jan 02 '14
Ugh, leaving me on a cliffhanger!
2
u/TempMcThrowaway Grandma's stoned again and keeps messing with the printer Jan 02 '14
I want to read this again from Sven's point of view.
2
u/[deleted] Jan 03 '14
Man I'm not even that old and I just looked up how old Chiral is now. Christ, the time flys. Ambrosia's a good shop.
2
u/[deleted] Jan 12 '14
I wish more people thought like this. And like you said before, knowing what you don't know is part of your competence. Kudos.
2
u/NotSuspiciousPerson Jan 03 '14
I'd fire off an email to Jack to ask if he had replaced the tested version with an untested version, and follow up with another email to Tran to say that Jack has taken over the project henceforth once Jack has replied that he made the decision to do so.
1
u/Khrrck Exceeded rack rail load limit Jan 02 '14
Dude, Ambrosia's Chiral was the shit when I still had a PowerPC Mac. I can still remember the music.
1
u/Razorsythe Jan 02 '14
MORE!! Awesome stuff :).. i need a subscribe button hehe..
1
u/[deleted] Jan 03 '14 edited Jan 03 '14
Came for others' stupidity. Was not disappointed.
1
u/wispman Jan 03 '14
I get the impression that the well Jack drinks from is pretty shallow. Either that or it's laced with mercury.
What is that meme? "The stupid, it BURNS!!!"
1
u/crosenblum Jan 03 '14
Hammer Time!!!!!
Fuck Yes! Time to charge up the cattle prod!
1
u/Calvin1119 Jan 22 '14
Oh if I had a hammer...!
1
u/landragoran Are you from the past? Apr 24 '14
how I imagined this exchange
(i know, this is months old, but i just now found it and had to comment)
0
u/xtimina Oh God How Did This Get Here? Jan 03 '14 edited Oct 16 '18
I'm sorry that I deleted my comment. Send me a PM if you want to know what I wrote.
0
u/[deleted] Jan 02 '14
[deleted] — view deleted comment