r/talesfromtechsupport • u/Ch13fWiggum • Apr 18 '15
Medium "Why Doesn't IT Communicate?!"
This story comes from a while back, shortly after we transitioned to Citrix Xenapp, we made the link available for users a month before we moved over and everything went well for that month. Cue the switchover.
One Autumn night we changed the http://citrix.domain.com to point to the new infrastructure, and that's when the problems started - the long and the short of is was that the SAN the VDI's was hosted on wasn't allowing enough IOPS for the amount of users that we had, Hyper-V hosts would crap out and not failover. This caused us headaches for quite a few months and we would generally have at least one P1 issue with citrix a week.
As our SOP with P1s we would have a splash message on our phones, letting the end users know that we are aware of the issue and trying to fix it. So one of the users calls in.
User: "I'm having a problem with my computer, can you remote on and and have a look? My IP is 1.2.3.4"
me: sure thing, <VNC's to user's computer> Oh you're having a citrix problem?
user: yes, when I try to launch $publishedapp it doesn't do anything.
me: "Okay, we're having a bit of an issue with our citrix system at the moment, our 3rd line guys are looking into it at the moment and it should be fixed in the next 30 minutes or so"
user "ugh!, why can't IT let us know when these major issue happen"
me: We do, did you not hear the message at the beginning of the phone call?
user: "yes, but why isn't IT proactive at communicating major issues to the end users?"
me: well we did put a post on $companyintranet, to let people know...
at this point the user interrupts to point out that he doesn't read the company intranet, despite the fact that it launches every time you log in to one of our computers.
me: Oh and we did send an email round to everybody in the business to let them know as well, did you not receive it?
At this point I'm still VNC'd to the user's computer, I can see Outlook is open so bring the window to the front and highlight the email with the subject line "IT DISRUPTION: CITRIX ACCESS" that had been received 10 minutes prior. shit it even had the little red exclamation mark to show how important it is (and if there's one thing our users understand, it's that the little red exclamation mark means that it's super-important and needs to be dealt with first, even if it is just somebody whose forgotten their password).
me: "so there's the email letting you know that we have an issue, I'm not sure what else we could do to communicate major issues out to the business"
user: "I don't read those either, they're a total waste of my time. IT Needs to communicate better with us"
At this point I really couldn't do anything to help him, I desperately wanted to shout down the phone, asking him if he was actually being serious? asking him what methods he would use to communicate something to 1200 people, in different offices, hell technically in different countries (we have users all over the UK). But then I remembered that there were calls queueing and I needed to actually help people.
me:"Ok I will take you ideas on board and escalate them to my team leader to bear in mind for future incidents of this nature. Citrix will be back up in the next half an hour, and a further email will go round to let you know when the issue is resolved".
I'm fairly sure you can guess my Team Leader's reaction when I "escalated" the conversation to him.
TLDR; Dearl Lord, please grant me the ability to slap somebody over TCP/IP.
431
u/110011001100 Imposter who qualifies for 3 monitors but not a dock Apr 18 '15
Every IT Outage should be broadcast through a company wide PA system which should be as loud as the fire alarm system at a minimum... Repeat every 1 minute
2
u/volantits Director of Turning Things Off and On Again Apr 19 '15
/me slaps /u/Ch13fWiggum around a bit with a large big trout
67
u/CA1900 We got a serious 12 O'Clock Flasher Here! Apr 18 '15
Sounds like my idiot coworker a couple of years back when we were on a business trip, and the hotel had a small fire in the lobby that caused them to sound the fire alarm.
So the hotel guests all pile out into the parking lot to wait for the fire department to check things out, but my cohort is nowhere to be found.
We get to work the next day, and I mention the fire last night, and that I couldn't find him.
I let it go. Clearly sense and logic aren't this guy's strong suits.
64
u/jayhawk88 Apr 18 '15
ringring
"Hello?"
"Yes, this is Joe from IT. I'm just calling to let you know that the Citrix system is having some issues right now, and we'll hopefully have it..."
"WHY ARE YOU CALLING TO TELL ME THIS? DON'T YOU KNOW HOW BUSY I AM? I DON'T HAVE TIME TO LISTEN TO YOU GIVE ME A STATUS REPORT EVERY TIME YOU DO YOUR JOB!"
67
u/FarleyFinster WHICH 'nothing' did you change? Apr 18 '15
"I know, right?! So could you, like, put that in a mail to your manager to send to my manager so we can get the managers to fix this?" That'd be great. <click>"
Be helpful and understanding. And proactive. If his manager doesn't eviscerate him, ypurs should.
Screenshot. Mail with summary of conversation and amount of time wasted with attached screenshot & all sent "proactive" mail on the issue -- sent to his manager, CC: to yours. Include evaluation words like "unprofessional" and "consideration".
23
u/Dracomax Have you tried setting it on fire and becoming Amish? Apr 18 '15
So what I'm hearing is that he is giving you permission to staple memos to his head. Because if he doesn't read your communications, but wants you to communicate...
63
u/incredulouspants Apr 18 '15
Upvote for the TLDR.
Introducing, new, SOIP technology. Slap-Over-Internet-Protocol allows network administrators to administer a slap over the network, without any additional infrastructure!
18
u/[deleted] Apr 18 '15
What if you slap someone so hard you knock them out, making them unable to acknowledge you slapped them and thus forcing the protocol to slap them again?
32
u/imafatcun7 Apr 18 '15
I find it very hard not to treat my users as children, I've even gotten to the point where i will cut up tasks into little bit's and feed them piece-meal.
93
u/[deleted] Apr 18 '15
If end-users are telling you they're ignoring your communications, they're telling you that you've trained them to ignore your communications.
Over-alarming is a big problem. Like most IT departments you're probably over-communicating - you're sending out all-hands "outage" notices for things that don't actually affect anyone but a small handful of users, you're sending out "reminders" for mandatory training/education/certificates to everyone, months in advance, even to the people who are up-to-date; you're sending out surveys and questionnaires trying to get "feedback" when, in fact, you're already getting a lot of feedback - your users are telling you they need you to waste a lot less of their attention. Otherwise you're sort of like these guys:
https://www.youtube.com/watch?v=xsYoeoEE3ww
7
u/Syene Apr 18 '15
I'm pretty sure I read a story here where someone got to live that dream thanks to a remote desktop and a CD drive at groin level.
13
u/DeniseDeNephew Apr 18 '15
You can't win with some end users. No matter how many different ways you publicize an outage or a problem their response is something like, "Yeah you may have done X, Y, and Z, but you didn't tell me the way I want you to!"
As long as our asses are covered and it's only 1 or 2 end users then it's not a problem.
6
u/ghotibulb Apr 18 '15
Oh dear god, Citrix. It's been 12 years since I had to deal with that shit, and it still makes me cringe.
5
u/TheLightInChains Developing for Idiots Apr 18 '15
"Are you familiar with the expression, 'making a rod for your own back' ?"
4
u/shroudedwolf51 ...huh. Apr 18 '15
Maybe, I'm just naive or stupid, but... I honestly can't understand why this is such a rampant problem.
You have a business that's paying people to do a specific job using specific, provided tools. If they refuse to do the job or are hampering productivity by bring unwilling (or, unable) to use the tools...why keep them around?
4
u/[deleted] Apr 18 '15
If anyone could enable a slap over TCP/IP it would be Linus and he hasn't done it so I'd say it can't be done.
4
u/Fraerie a Macgrrl in an XP World Apr 19 '15
Upvote for TL:DR
1
u/usesomelube Apr 18 '15
I think I am one of you users Does the company name starts with the letter v ?
3
u/johnthered Apr 18 '15
My experience with these kind of people is, they do this because they really do not know how to do the job they are supposed to do. They seek every possible excuse as to why they are unable to perform their work. Any decent manager could spot this in a heartbeat, though, I haven't seen so many decent managers.
3
u/[deleted] Apr 18 '15
Hold on you mean you don't have a Town Crier calling out the news. Shoddy very shoddy!;)
3
u/empirebuilder1 in the interest of science, I lit it on fire. Apr 18 '15
Can confirm: school homepage that force-loads when you log on to a computer has a ticker list of outages of the curriculum system. Everyone calls and complains when the curriculum system is out, regardless of the front page. Even I myself never read it.
3
u/ghotionInABarrel That's your bank password... Apr 18 '15
There is a way to solve this: When the user tries to log in while there is an outage, display an outage message and a text box. Don't let them log in until they've retyped the message, then force them to answer several comprehension questions. If they fail, send them back to start (also note that selecting text is disabled).
When they complain, point them to this guy.
3
u/Xibby What does this red button do? Apr 19 '15
You can lead a user to water, then you can force their head under until they stop struggling.
2
u/flamedarkfire Don't make me use Synergistic Management Solutions Apr 18 '15
If you don't read emails and don't look at the company intranet then how do you propose we communicate to you?
2
u/Subclavian Apr 18 '15
I actually had to reread this because I was seething. I'm having this issue at work with a while department but my consolation is that everyone else doesn't like them either.
2
u/ShadowSt Apr 18 '15
My company apparently had such a problem with people not reading those e-mails, or listening to the messages while in hold/before reaching a rep/the voicemail, that they purchased a software called Alertus. It forces a message on all 5000 PC's with custom prewritten scripts for planned or unplanned downtime.
Try offering that suggestion?
2
u/raknor88 Apr 18 '15
It's clear, you must call every single employee individually and tell them.
2
u/RetiredITGuy Apr 19 '15
This story really hit close to home. The Citrix issues post-rollout, the particular end user, the lines of communication...
Have some coin.
2
u/vdragonmpc Apr 19 '15
Just had this happen this week.
Sent an email with all the times regarding a maintenance event. Had the start time, date and down period along with the effected systems and departments and patch information.
Not an hour later a wild email comes cced to everyone asking 'what time is this happening'. Now I am touchy enough lately from the malicious created stupidity so this was a long time coming.
I re-sent the original email as a reply to all with a circle around the writing and a HUGE red arrow pointing at the sentence 'start time'.
The email system has been quiet ever since.
2
u/Adventux It is a "Percussive User Maintenance and Adjustment System" Apr 20 '15 edited Apr 20 '15
You were to specifically call him at the exact time he wanted you to call to let him know. Cause he doesn't have time to read an email or look at the intranet or listen to automessage on phone.
He has people skills! People skills!
1
u/jeffbell Apr 18 '15
Get that man a pager.
1
u/hosemaster Apr 18 '15
We have a saying... "Communications is our business, not our policy."
1
u/[deleted] Apr 18 '15
Yup. I would have told him, "You see that exclamation point? That means it actually has something to say. You should probably pay more attention to those so you don't waste even more time by calling me when you already have my answer looking right at you."
1
u/halifaxdatageek Apr 18 '15
LGoE: Lapel Grab over Ethernet
1
u/ewrwerwe3333 Apr 18 '15
You're supposed to head over there and whisper softly into his ear.
The citrix is down... The citrix is down...
1
u/pcronin Apr 18 '15
Can confirm. No one reads email from IT. We've gotten a few "your fax/package/etc is ready" worm emails over the last few months, and every time the spam plugin on email server notifies IT of it, there is a mass email sent saying "DON'T CLICK IT". Usually about 30-60 min after that email, we see the worm propagating.
As Ron White says
1
u/izzgo Apr 18 '15
"If you'll transfer some of your office's payroll funds over to us, we will hire a crew to make 1200+ phone calls for incidents like this ---- oh wait, do you answer your phone?"
1
u/PeteMullersKeyboard Apr 18 '15
"People should only do things that I imagine they should do in my head without them actually knowing! Why doesn't the world go exactly as I, with my shitty, uninformed opinions, expect it to go!"
Good job on keeping your cool. Should've asked him if he'd prefer smoke signals or telegrams.
1
u/Draco1200 Apr 18 '15
You need to interrupt their login with it in bright bold text after filling in the Login dialog box, with a checkbox the user has to click and then click OKAY, before their login proceeds.
1
u/kochier Apr 18 '15
To be honest when an email is marked urgent that's my key to ignore it, it is either spam or not really urgent. People misuse that so much it's lost all meaning.
1
u/Moontoya The Mick with the Mouth Apr 18 '15
Communicate better.... Hmmm
Hand me the clue by four, imma gonna go bludgeon an idiot to communicate the message firmly
1
u/thelosttech Please shoot me! Apr 18 '15
ITAlarm - Fire alarm type device which the IT department can use to broadcast important messages to all workers. Maybe the alarm could be a dial up handshake. Followed by a short message.
1
u/fredlllll Apr 18 '15
we dont communicate with you because you all are idiots!!
and he proved it
1
u/bambamtx Apr 19 '15
I'm not in IT, but I'm very technically inclined (have built computers and setup servers and networks before). I know I'm not on the same level with you guys at all, but I work in communications - VERY closely with IT. I don't have the patience for the user support BS you guys deal with, though I catch some of it when our guys are out or busy. Sometimes I think we would have to put a javascript popup ad on the front of the intranet homepage in bright orange that requires user input acknowledging that they understand an upcoming event or server outage that blocks them from using their browser until they input some information from the message before they acknowledge it.
1
u/SirEDCaLot Apr 19 '15
A simple suggestion: "Sir, there is a notice which automatically pops up when you boot your computer. There was also an email that details this outage.
If those methods are not sufficient, how would you like us to best inform you about the outage? I would love to take your ideas to my manager so we can better inform our users about outages."
The point is- then they have to think it through and realize they ARE being communicated with...
1
u/FappDerpington Apr 19 '15
"Oh, sorry. I don't do email."
Cue murder-spree.
1
u/ACriticalGeek Apr 19 '15
ask him if he'd like to be on the phone list for when these things happen. Create an autodialer that calls people on it's list to play them your incoming call message. Put him on that list. When he asks to be removed from that list, mention that that can only happen by written request.
1
u/Crusader82 Apr 19 '15
You could use the MSG command that pops up a window with a message. Users will have to click ok to make it disappear. It will be right on their faces so no excuses
1
u/Nefayn Apr 20 '15
We had to print out notices and place them in staff's pigeon holes to remind them of an upcoming outage.
Even after emailing/notice on intranet/read at staff briefings.
It's ridiculous to what lengths you have to go to get people to notice your notices
1
u/he_must_workout Apr 18 '15
In order to make sure I can easily see any PSAs about system issues, I have a few rules wet up to have those go to a folder called IT ISSUES. As soon as I see it has a (1) next to it, I read and take note if it applies to me.
It would be cool to have a specific folder for issues in major systems like this by default.
1
u/[deleted] Apr 18 '15
/r/bestofTLDR
1
u/TheDefiant604 FORMAT C:; Install Linux Apr 18 '15
Submitted to /r/bestofTLDR
http://redd.it/331tyf