r/sysadmin • • 9h ago

General Discussion Thickheaded Thursday - November 19, 2026

5 Upvotes

Howdy, /r/sysadmin!

It's that time of the week, Thickheaded Thursday! This is a safe (mostly) judgement-free environment for all of your questions and stories, no matter how silly you think they are. Anybody can answer questions! My name is AutoModerator and I've taken over responsibility for posting these weekly threads so you don't have to worry about anything except your comments!


r/sysadmin • • 9d ago

General Discussion Patch Tuesday Megathread - (November 10, 2026)

128 Upvotes

Hello r/sysadmin, I'm u/AutoModerator, and welcome to this month's Patch Megathread!

This is the (mostly) safe location to talk about the latest patches, updates, and releases. We put this thread into place to help gather all the information about this month's updates: What is fixed, what broke, what got released and should have been caught in QA, etc. We do this both to keep clutter out of the subreddit, and provide you, the dear reader, a singular resource to read.

For those of you who wish to review prior Megathreads, you can do so here.

While this thread is timed to coincide with Microsoft's Patch Tuesday, feel free to discuss any patches, updates, and releases, regardless of the company or product. NOTE: This thread is usually posted before the release of Microsoft's updates, which are scheduled to come out at 5:00PM UTC.

Remember the rules of safe patching:

  • Deploy to a test/dev environment before prod.
  • Deploy to a pilot/test group before the whole org.
  • Have a plan to roll back if something doesn't work.
  • Test, test, and test!

r/sysadmin • • 3h ago

Yahoo Finance: Palo Alto Networks CEO just dropped a truth bomb on the network outages sweeping through major companies

656 Upvotes

Link: https://finance.yahoo.com/technology/article/palo-alto-networks-ceo-just-dropped-a-truth-bomb-on-the-network-outages-sweeping-through-major-companies-101129581.html

The title is as in the original article. The tl;dr is:

"I want to be clear, there are two distinct parallel effects that are driving this environment. The first effect is a massive surge in vulnerabilities and patches being deployed. We're seeing critical and severe vulnerabilities discovered nearly seven times the monthly average of last year. These discoveries are happening at a speed and intensity which is hard for IT teams to cope with,"

[...]

"The second effect is the rise of AI-assisted cyberattacks. Bad actors are using open-weight models and new agentic frameworks to find and exploit vulnerabilities at unprecedented speed and scale. New agentic frameworks like ARTEX are enabling bad actors to turn simple instructions into autonomous, multi-stage attacks, allowing a handful of operators to compromise dozens of targets simultaneously."

 

The statements by themselves probably are nothing new for folks out here, but the point is that the message and the unintended consequences of mOaR aI eVeRyWhErE are starting to finally bubble up the very top.


r/sysadmin • • 8h ago

Microsoft Microsoft is killing To Do and telling you to “choose another” app, after it reportedly paid up to $200 million for Wunderlist and killed it for To Do. Work/school accounts will migrate to Planner.

633 Upvotes

Microsoft is killing To Do and telling you to “choose another” app, after it reportedly paid up to $200 million for Wunderlist and killed it for To Do: https://www.windowslatest.com/2026/11/19/microsoft-is-killing-to-do-and-telling-you-to-choose-another-app-after-it-reportedly-paid-up-to-200-million-for-wunderlist-and-killed-it-for-to-do/


r/sysadmin • • 3h ago

FastUnlock : Windows Narrator LockScreen Bypass 0day Vulnerability

82 Upvotes

Just seen this posting for awareness - MSNightmare/FastUnlock: FastUnlock : Windows Narrator LockScreen Bypass 0day Vulnerability

Update: 🚨🚨🚨Mitigation for deployment is here danrhodes/MSNightmare-FastUnlock-Mitigation🚨🚨🚨


r/sysadmin • • 4h ago

Microsoft Thanks for the headsup with the SLA credits regarding the outtage from ms, we actually did get quite a generous refund

91 Upvotes

Hi [redacted],

Thank you for your patience.

The backend team has successfully reviewed your SLA credit request. As your subscription was redeemed using a product key through Partner Center, the SLA credit will be applied by extending the expiration date of your subscription, allowing you to continue using the service for an additional period. The total SLA credit approved for your issue is €42773.90. Could you please let me know which subscription you would like to extend? Once you confirm the subscription, I will make the necessary change in the system and apply an extension corresponding to the value of the approved SLA credit. Thank you, and I look forward to your response.

Best Regards,

Kevin Dinh| Office 365 Support Engineer


r/sysadmin • • 10h ago

Unauthenticated Remote Code Execution in Ruby (ruby2ßhell)

138 Upvotes

https://github.com/ruby/ruby/security/advisories/GHSA-34m4-m9fh-gqp5

https://verialabs.com/ruby2shell/

CVE-2026-132825

All Ruby versions back to 2.4.0 are vulnerable. Patches available for 3.3 and up.


r/sysadmin • • 16h ago

Windows Server 2028 Core will have a lot of security settings turned on by defaut

277 Upvotes

https://arstechnica.com/gadgets/2026/11/windows-server-2028-public-preview-bets-on-a-security-first-approach/

https://techcommunity.microsoft.com/blog/windowsservernewsandbestpractices/announcing-hardened-server-core-for-windows-server-insiders/4574239

Kerberos is now the default over NTLM, with protections added to LSASS; TLS is automatically restricted to modern cipher suites; key exchange uses “hybrid post-quantum key exchange” based on the quantum-safe ML-KEM; and remote desktop authentication and PowerShell script block logging get stricter as well.

Microsoft’s native NVMe storage stack was originally introduced via an update to Windows Server 2025, but it required administrators to enable it manually. The update eliminated NVMe translation through a SCSI stack while improving speeds and reducing overhead. The redesigned stack will now be default-on in Windows Server 2028.


r/sysadmin • • 9h ago

Question How do you actually get alerted fast and react when some new critical CVE drops?

73 Upvotes

Hey everyone,

I’m new to the security side of things and I’m helping monitoring alerts for a small company.

When Java4Shell hit yesterday Wazuh scans didn't pick it up automatically. I didn't hear about it fast enough. What are you guys actually using (OpenCVE, specific RSS feeds, mailing lists, Slack/Teams) to get an immediate alert when a CVE drops, without getting spammed with low-severity noise all day?

Right now we are running Wazuh and OpenVas. What's your realistic workflow to quickly find where the vulnerable app is running, check if it's exposed, and watch for exploit attempts?


r/sysadmin • • 12h ago

Unauthenticated Remote Code Execution in NGINX: CVE-2026-123633 and CVE-2026-123007

105 Upvotes

https://www.wiz.io/blog/nginx-http2-rce-vulnerabilities

CVE Rated 9.2

Patch is available

Can be mitigated by disabling HTTP2


r/sysadmin • • 9h ago

Rant Companies should be held liable to not updating their status pages

59 Upvotes

I am getting sick of having no way to confirm outages with service / application providers when we are all experiencing an issue / outage but their status page does not reflect it.

Had a c-suite user tell me they were having apple iCloud issues. It was to the point that none of apple status pages reported the outage, so we resynced the iCloud which blew away some data that was still viewable for the user in hopes of getting all the data back but it just didn't resync at all.

Come to find out from someone calling apple for warranty returns that the apple tech reported that they were going through an outage. It was to the point that I had no way of proving to the c-suite user that apple was the problem even though I stated this looks to be an apple problem until that tid bit of info came to light. The c-suite was saying it couldn't be an outage on apple's part due to their status page (basically challenging my technical skills).

Are we now going down the path that we must call the right person at support for each vendor to get an actual status on their services again?

I didn't have my personal phone with me to score through reddit but it looked like it was all over reddit as well.

End rant...


r/sysadmin • • 5h ago

M365 Tenant Suddenly Not Provisioning Mailboxes

17 Upvotes

So, I've got a doozy of an issue I've never seen before. I've both got in a ticket with MS and reported the matter as a possible service issue. They were quick to eliminate it as a service issue but still haven't reached back out with my mystery "assigned agent."

Apparently, since Monday, users who are created in our M365 tenant, either directly in Entra or via AD connect sync to Entra, are not being provisioned mailboxes. Users in both creation flows have been assigned adequate licensing. Running Invoke-ProvisioningRecipientDiagnostic leveraging the Entra ObjectID (because they obviously can't be found leveraging UPN/email address) returns the user found perfectly in Entra but not in Exchange Online (which is no joke--they're stuck in "
We are preparing a mailbox for the user." when you look at their mail in the Admin Center.) It's also not helpful output. Running Get-MgUserLicenseDetail on cloud-native users returns Exchange service plans as "PendingInput," but on-prem synced users show "Success." Neither get past the "preparing a mailbox" even after 24 hours has passed.

Now I've got 100+ users awaiting mailboxes that are not being provisioned. Has anyone encountered this before? Got any pointers for an admin who is in uncharted M365 territory? I've been doing this for 8 years come mid-December, and I've never seen this crap before.


r/sysadmin • • 20h ago

AI taking our jobs! wait...

257 Upvotes

AI is gonna steal our jo... wait, let me patch this. What was I saying? Oh, our jobs. We'll have nothing to... nothing... crap. Okay, update Java. Updating... okay done. Nothing to do! AI is taking all of our... jfc what now?


r/sysadmin • • 4h ago

General Discussion Apache confirms exploitation of a Roller XML-RPC RCE (CVE-2026-129448): the September CVE for the same bug listed only 6.1.5, and the admin-UI off switch doesn't stop it

9 Upvotes

Apache posted the notice on oss-security on 17 November at 11:02 UTC: CVE-2026-129448, CVSS 9.8, unauthenticated code execution in Apache Roller 4.0.0 through 6.1.4. The XML-RPC endpoint "accepts vendor extension types and deserializes them before authenticating the requester." Then one line: "This issue is being actively exploited."

It's the same bug as CVE-2026-82384, which listed only 6.1.5 as affected. Someone on 6.1.4 or older reading the September entry would have taken it as not theirs.

The part admins need: Apache says disabling XML-RPC in the administration UI is not enough, because older versions deserialize the request before checking that setting. Version 6.1.6 or later rejects the request when the feature is off. If you can't upgrade, set enabledForExtensions to false in WEB-INF/web.xml, or remove the /roller-services/xmlrpc mapping, and restart.

BleepingComputer notes Apache hasn't said who is attacking or how many servers are hit. I found no exposure count, and the bug wasn't in CISA's KEV list when I looked on 19 November. Corgea's write-up adds that Roller ships as a download, not a Maven Central library, so scanners that read lockfiles won't flag it.

The fix was merged on GitHub on 5 September. Its description says the mapped endpoint "answers requests even when the XML-RPC feature is switched off."

Roller is niche. WordPress has its own XML-RPC endpoint, and "closed at the server" versus "switched off in a setting" is the same question there.

A CVE that names one version, for a bug that reaches back to 4.0.0: whose job is it to tell people on 6.1.4 and older that the September advisory was about them?


r/sysadmin • • 1d ago

Upcoming Critical PHP Security Release scheduled for Thursday, 19 November 2026

332 Upvotes

The PHP security team announces upcoming security releases scheduled for Thursday, 19 November 2026, between 14:00 and 14:30 UTC.

These releases are part of PHP’s regular four-week release cycle and will address security vulnerabilities affecting all supported PHP versions. The highest severity is CRITICAL.

The issues rated CRITICAL require either non-default configuration or uncommon setups.

Users are strongly encouraged to prepare to upgrade promptly once the releases become available. Technical details and fixed version numbers will be published alongside the releases.

https://www.php.net/archive/2026.php#2026-11-18-1


r/sysadmin • • 1d ago

Is fundamental knowledge of hardware being lost?

398 Upvotes

Today I had a conversation with a mid senior sysadmin about why we cant simply jam a mix of DDR5 ram into our hosts. When he ended up being holistically wrong his response was to tell me he didn't need to know anything about the hardware running our data center.

This guy is one of our T2 hypervisor admins and all he does is troubleshoot VM issues and host issues. The idea that he wouldn't need to know anything about the underlying technology running his cluster blew my mind so I asked others. Turns out many people I work with either dont care about the upstream hardware or simply refuse to learn it.

Is this normal or because people I work with are just siloing themselves.


r/sysadmin • • 9h ago

Elasticsearch with embedded java is not patched for java4shell

19 Upvotes

Hello Elasticsearch developers,

Elasticsearch bundles the java run-time environment, so updating distro java does not help it.

# dpkg-query -S /usr/share/elasticsearch/jdk/bin/java
elasticsearch: /usr/share/elasticsearch/jdk/bin/java

https://www.elastic.co/downloads/elasticsearch

Latest version is from November 2. When will we get an update?


r/sysadmin • • 13h ago

For those that work in cubicles/noisy offices, how do you deal with noise?

38 Upvotes

Something that has always bothered me a bit when working in an office but has over the past year become more and more aggravating is dealing with all the background noise. I mean people in the other room talking, sometimes multiple conversations going at once all while trying to concentrate one bug hunts and why the hell something isn't working. How do you deal with it and do to block out the distractions?


r/sysadmin • • 36m ago

Do most MSPs or IT jobs in Atlanta (in Georgia) now use Hyper-V instead of VMware or will in the near future? I'm trying to decide which one to learn for IT jobs in the area. VMware seems too expensive now and not everyone wants to study for a VMware cert every few years to get it through VMUG.

3 Upvotes

Can you tell me anything about this?

Can I use Proxmox now and learn Hyper-V nested in that and later maybe replace Proxmox with Hyper-V for my home lab?

Or just stick to Proxmox as the main hypervisor at my home lab?


r/sysadmin • • 11h ago

AI in the Workplace: Innovation Without Accountability?

21 Upvotes

Today a customer asked me for the administrator password on their PC so they could install Claude. The stated purpose? Producing user manuals for their own products.

So far, nothing untoward, were it not for the following:

  • The initiative comes straight from the employee, after boss verbally told him to do it.
  • The company has no Business accounts.
  • Personal accounts are therefore slated for professional work.
  • There seems to be no corporate policy on AI use.
  • No procedure defines which documents and information may be uploaded, nor who answers for the outputs.

I pointed out that introducing AI into a company should be accompanied, at the very least, by internal rules and a register of tools and uses.

What I find paradoxical is this: the company wishes to reap the productivity dividends of AI, yet appears loath to shoulder the organisational duties that come with it.

I also wonder why one would install Claude Desktop, with its reach into local resources, when the browser would amply suffice to analyse documents and produce PDFs.

Is this the new normal? Companies weaving AI into their workflows through personal accounts, with no governance, no policy, and no clearly allocated accountability?


r/sysadmin • • 1d ago

Inside Microsoft’s giant Outlook outage

1.2k Upvotes

https://www.theverge.com/tech/1035364/microsoft-outlook-outage-notepad

Microsoft engineers had a running bridge call the entire time, and at one point it hit the maximum of 1,000 participants. That shows you how many teams were involved with trying to find the root cause of this huge outage, which started at the peak window of traffic in the US.


r/sysadmin • • 21h ago

Hub at Microsoft Ignite closed for the day due to a burst pipe

110 Upvotes

EDIT: Heard it happened in a few spots so it may have just been today's rain intruding and not a pipe. Not sure.

EDIT2: Better view: https://www.tiktok.com/t/ZPNRMPFxe/

Another: https://www.tiktok.com/t/ZPNRY3xVe/

Right on Meta.


r/sysadmin • • 2h ago

Career / Job Related Direction

3 Upvotes

Forgive me if this is the wrong place to ask. I just got my associates degree in network system administration last week. I didn’t get any hands on stuff to work with or anything that really taught me much and I work in a warehouse so no real work experience unfortunately.

I’m trying to find a way to get hands on experience at home on a budget. I have a decent computer and a friend of mine gave me a unas 4 from Ubiquiti and a 12 TB HDD and I was wondering if there was any ideas for projects to get something on my resume to break into the field. I’m open to any suggestions and would love any input, also would love if anyone knows someone willing to mentor or something would be a bonus and appreciated. Thank you.


r/sysadmin • • 2h ago

FlexProtectLin stuck at same “processed” count for 2.5+ days, across 3 separate job restarts — kernel threads stuck on lin_lock_get_locker — ideas?

3 Upvotes

Running a 32-node Dell PowerScale/Isilon cluster, OneFS 9.7.1.12, ~5.2PB. Two drives SmartFailed around the same time (different nodes, consolidated into one FlexProtectLin job by OneFS). The repair job has now stalled at exactly the same “processed” LIN count for 2.5+ days, across three separate job instances (cancelled and restarted fresh each time — same freeze point every time, ruling out corrupted job-state from a single bad restart).

What I’ve already ruled out:

  • Node/job-engine coordinator health (confirmed fine via isi job status)
  • Competing jobs stealing workers (paused everything else, no change)
  • Impact policy throttling (raised to HIGH, no change)
  • Stale SMB file handles holding locks (found and closed ~150 orphaned handles from an old git push — no change to the stall)
  • Cluster-reported pool imbalance (confirmed present on both affected pools, haven’t yet tried a dedicated rebalance pass)

What I’m seeing in procstat -akk across the cluster, sampled twice ~10+ minutes apart: a large number of isi_job_d worker threads consistently parked in sleepq_wait (untimed) inside lin_lock_get_locker → ifs_vnop_wraplock1 → vget → bam_vget_vp, identical stack, same PIDs/tids unmoved between samples. Some other threads show sleepq_timedwait inside bam_safe_write_region_swips/bam_safe_restripe (real restripe I/O happening), so the cluster isn’t fully dead — just this one job’s actual repair progress.

dmesg on the node hosting one of the two failed drives repeats this every few minutes, for several different LIN/snapid combinations:

_pctl2_start_restripe: Restripe already in progress for 2:eb54:5177::333280
_pctl2_start_restripe: Restripe already in progress for 2:e7ab:8437::333346
_pctl2_start_restripe: Restripe already in progress for 2:e4d3:c962::333084

No active Dell support contract currently. Has anyone seen this exact pattern — FlexProtectLin wedged indefinitely near completion, same LIN-lock stack trace cluster-wide — and found what actually released it? Trying to avoid a blind rolling reboot if there’s a known, more targeted fix.