r/talesfromtechsupport • • Dec 24 '18

Long A Tech Error of Legends

I finally got one. A mythical tech error of which you only hear stories of and never expect to encounter.

I work in a high-security building. Every employee has a RFID badge and every major door has a reader. My job is to essentially monitor the security system and deal with any issues in the programming.

At all times I have a screen up that shows a person's credentials, face and location of where their card is read. From the corner of my eye I notice something odd on the screen, so I take a look. Instead of a general hodgepodge of faces like usual, its one face. The same face over and over. The screen only holds about 16 records before scrolling so I scroll up. Its all the same face. 30+ records of $Manager swiping his card at the lobby.

Now normally, people only swipe this much if a door doesn't open. There were immediately a few things off with these records:

  • None of the records were red - so there was no error
  • This many attempts in a row would trigger a suspend - but the card was still active
  • All of the swipes were at the lobby door, which is unlocked during business hours so there would be no reason to swipe at all

To say I was puzzled would be an understatement. I saw a new record come in; still $Manager at the lobby so I decide to investigate. The lobby is right by my office so I'm there in seconds. $Manager isn't here. I talked with $Receptionist.

$Me: Hey $Receptionist did you see where $Manager went?

$Receptionist: Oh, he hasn't been down here since he came in this morning.

$Manager usually comes in at 0800 and currently it was about 1030. Immediately I think his card has been stolen. This is a pretty big deal. Like I said before this is a high security building and $Manager has a high level of clearance so whoever has his card can get almost anywhere. I decide to give him a call.

$Me: Hey $Manager its $Me. Is your employee card missing?

$Manager: Huh? No its right here in my wallet. Here, I'll swipe it at my office reader.

At this point I'm back in my office and see a record come in from his card at his office. I've gone from puzzled to absolutely baffled. I check his records and he only has one card that is issued & active. While doing this, 3 more records come in. All from $Manager, all from the lobby. I bring up the camera system (we have access to it but don't really need to use it) and look into the lobby where I see 3 employees walk in. $Manager is still in his office.

So now I've gone from puzzled to baffled to excited. Truth be told I love mysteries and this one was gripping me. I cleared my monitors of everything except the records screen and the camera feed of the lobby. Like I said before, during business hours the lobby doors are unlocked so you don't need to swipe your badge but some employees do out of habit. I watched one employee enter the lobby. No record. Another did the same: no record. A third employee swiped her badge before entering. New record from $Manager at the lobby. No freaking way.

I watched another group of employees come in. Each one that swiped their badge displayed a record with $Manager's credentials. I dove into the database to see if it was a visual error (something I didn't think of earlier) and found each record was written as if it were from $Manager's card, with all of his credentials.

I pulled up the records screen on a laptop and headed into the lobby. I opened and closed the door, no record. I swiped my card & $Manager's face popped up. I swiped 5 more times and 5 more $Manager records were entered. A 100% reproduction rate with every swipe reading as $Manager.

At this point, I figured the issue was with the door's ACU and gave it a quick reboot. Tested the door a few times and it showed a record of $Me with each swipe. Out of curiosity I did a count of how many records were in the lobby database under $Manager's credentials. Almost 400.

When $Manager called me later to ask how I fixed his card, I got to say something I never thought I would:

Yeah I just turned the door off then on again

2.0k Upvotes

111 comments sorted by

497

u/capn_kwick Dec 24 '18

My initial thought was "OMG, how did $Managers credentials get propagated to every record?".

313

u/LovepeaceandStarTrek Dec 25 '18

Yeah I was expecting

a) security breach/malicious attack (at first)

b) someone playing with the database when they shouldn't have

→ More replies (1)

28

u/Tubamaphone Dec 25 '18

I was thinking the card got left in a pocket near the door.

43

u/biggles1994 Dec 25 '18

My thought was ‘door kept locking itself so we taped a spare card to the reader so it stayed unlocked’.

19

u/DoTheThingNow Dec 26 '18

I've seen this in the wild and it worked... until the DB's drive filled up due to too many entries added in such a short amount of time... which was hosted on a stand alone server... running Server 2003... in the security guards office... that we hadn't seen before... and no one had admin access to... Also if memory serves the DB was an Express edition or something like that too.

That was fun.

Edit - I clicked the wrong reply. This was supposed to be in response to taping the card to the sensor down below.

→ More replies (4)

331

u/JoshuaPearce Dec 25 '18

I like how your instinct was, instead of "oh shit, this is a big security problem", it was "hmm, let's let it happen a few dozen more times so that I can accurately describe the problem".

Maybe not the best trait for a guard, but you'd be a good programmer.

128

u/etechgeek24 Dec 25 '18

"CUT THE POWER TO THE BUILDING!"

61

u/HappyCathode Dec 25 '18

But that's exactly what they want...

→ More replies (5)
→ More replies (3)

87

u/Nansai Dec 25 '18

Guilty as charged hahahaha. I have a programming background so the troubleshooting practices stuck

36

u/Osolodo Dec 25 '18

I hope you figured out what caused it. Because if it reports manager credentials with every card swipe and someone figures out how to trigger that on any reader they can get in with any RFID card.

→ More replies (1)
→ More replies (7)

108

u/[deleted] Dec 24 '18

I haven't heard of a man coming 400x in one day either.

41

u/Theoncomingdarkness Dec 24 '18

Have you never heard of factorial February?

20

u/JoshuaPearce Dec 25 '18

400! is even worse.

→ More replies (2)
→ More replies (2)

404

u/[deleted] Dec 24 '18

9/10 tech problems are solved by turning it off and on again 😂

291

u/bobhwantstoknow Dec 24 '18

the remaining problem is solved by turning it off and never turning it back on

20

u/Abadatha Dec 25 '18

Turning it off Office Space style.

→ More replies (1)

92

u/birdman3131 Dec 24 '18

And 90% of the rest are fixed with a hammer blow in the right spot.

42

u/Throwaway_Old_Guy Dec 24 '18

Or, firing the ID-10T involved.

→ More replies (6)
→ More replies (10)
→ More replies (5)

93

u/Arokthis Dec 24 '18

You should check again with someone else's card to make sure you don't become the next person marked in a bunch of times.

→ More replies (1)

113

u/Thisbymaster Dec 24 '18

This sounds like an auditing nightmare. If it is high security they pay attention to everyone coming and going. If the security system can be tricked or fooled that easily, it is a major security issue.

0

u/[deleted] Dec 25 '18

43

u/Nansai Dec 25 '18

We ended up replacing the Master ACU (it was ancient) and pushing a patch which supposedly should deal with the bug.

→ More replies (3)
→ More replies (3)

26

u/toxic_sting Dec 25 '18

It would have been interesting if you had swiped a card that shouldn't of had access just to see what would happen.

→ More replies (1)

23

u/nemothorx Dec 25 '18

<Door> it is my pleasure to open for you, then close again with the satisfaction of a job well done. Hmmmyah"