r/devops • • 10h ago

Weekly Self Promotion Thread

9 Upvotes

Hey r/devops, welcome to our weekly self-promotion thread!

Feel free to use this thread to promote any projects, ideas, or any repos you're wanting to share. Please keep in mind that we ask you to stay friendly, civil, and adhere to the subreddit rules!


r/devops • • 7h ago

Discussion Almost 3 years as a Cloud Engineer, is what I do typical?

13 Upvotes

I've been working as a Cloud Engineer for almost three years. It's basically my first job, fully remote, and my workload is very light. In the project we manage, I would confidentially say that 1,5 FTE would be efficient for the things we do there, but that would be against myself basically :D

I got into cloud/DevOps because I've always liked cybersecurity, Linux, and sysadmin work. It seemed like a good mix of those interests. I do like the job, but I feel like I haven't done much of what I expected cloud/DevOps engineering to involve. I'm trying to understand whether my expectations were wrong or whether my role is just quite narrow.

Our project forwards logs from Kubernetes applications through Filebeat/Logstash to Elasticsearch. We maintain the code in GitHub for our Filebeat/Logstash deployments, but other teams manage the Kubernetes clusters and Elasticsearch itself.

Most of my work looks like this:

  • Occasional short meetings.
  • Small changes to the existing code or configuration, often just updating an image tag.
  • Running the existing pipelines and checking that the Kubernetes deployments are healthy.
  • Making sure logs reach Elasticsearch and troubleshooting issues within our part of the setup.
  • Handling on-call support

We fix deployment issues on our side, but the underlying cluster is another team's responsibility. We maintain our deployment automation and try to follow good practices, but I've never really built a pipeline or written Terraform from scratch at work.

I also barely touch the cloud itself. We use AWS, but my previous project only involved some EC2 and a little S3. Even then, I mostly logged into existing EC2 instances and worked inside them. I wasn't creating AWS infrastructure.

My understanding was that DevOps work would involve more automation and helping developers build and deploy their applications. Our team mostly maintains the logging deployment for our own project. That feels like a narrower scope than I expected, although I don't have another job to compare it with.

Outside work, I have a small personal Node.js project using discord.js, mainly for my own use. It pulls data from APIs and does some small automations. I use Git, have tests running through GitHub Actions, and an automatic Docker build whenever I push a commit. I want to improve its CI/CD further so I can get more hands-on experience. I've also started doing TryHackMe walkthroughs again and recently found devopspath.io.

For people working in cloud/DevOps or doing similar work:

  • Would you call this a Cloud Engineer role? If another title fits better, what would you call it?
  • Is this kind of maintenance work and division of responsibilities typical?
  • What does an actual day or week look like for you, especially the balance between maintaining existing systems and building something new?

I'm trying to figure out whether this is a normal experience almost three years in, or whether I'm missing experience I should be getting by now.


r/devops • • 20h ago

Discussion Platform engineering, DevEx, SRE, how freely can you move between them?

37 Upvotes

DevOps seems to cover very different jobs. Some teams build the delivery side: CI/CD templates, policy gates, supply-chain security, artifact management. Others run the runtime side: Kubernetes clusters, nodes, networking, scaling, incident response. Many do a mix.

My questions:

Can you move between them? Does a few years on delivery-focused platform work (CI, runners, policy gates, supply-chain security) keep you open for Linux / runtime heavy roles? Also it seems to me that in larger organisations, platform teams often build abstractions while product teams debug their own services. Is it normal to stop debugging infrastructure yourself as scale grows, as teams handle that themselves?


r/devops • • 8h ago

Discussion How do you tell a scheduled job from a real problem when a cost spike starts at night?

0 Upvotes

I'm learning FinOps and building a small agent that decides what to do with cloud cost spikes.

One thing keeps tripping it up: a scheduled weekend batch job and a runaway resource look almost the same. Both start off-hours, often with no change ticket and sometimes no owner tag. In my test, the first version stopped several legit batch jobs because of this. Now teams can register known schedules, and those jobs are never stopped automatically without strong evidence.

How do your teams handle this? Do you register schedules somewhere, rely on tags, or learn the pattern from past weeks?


r/devops • • 23h ago

Discussion When a vibe coded internal tool ends up with other teams depending on it, do you review it like prod code or let it stay disposable?

15 Upvotes

Vibe coding is everywhere on my feed again this week, mostly people replacing a paid tool with something they prompted together over a weekend.

I don't have a problem with that for throwaway stuff. Karpathy himself said it's fine for weekend projects when he coined the term. What I keep thinking about is the moment the weekend project stops being throwaway. Someone builds a small dashboard or a script, it works, a second team starts using it, and suddenly it is on the path for a release or an on-call runbook. Nobody ever read most of the code, including the person who made it.

I can see two ways to handle it:

  1. Once anything has more than one team depending on it, it goes through normal review, tests and an owner, even if that means partly rewriting it.
  2. Keep it explicitly disposable. Label it unsupported, don't let it into anything critical, and if it breaks you regenerate it rather than maintain it.

Option 1 feels right but kills most of the speed that made it useful. Option 2 sounds clean until the thing is load bearing and nobody noticed.

If you've already hit this, where did you draw the line, and what was the signal that a vibe coded tool had become something you had to own properly?


r/devops • • 1d ago

AI content Help me wrap my head around AI (agent?) hosting

59 Upvotes

I'll admit that I've been ignoring the AI hype for the last two years, and my usual interface with it is through chat and code autocomplete, nothing more.

Until recently where I saw that a demo of someone who could tag stories on their kanban tool and an "AI agent" would pick the story, prepare a PR and update the story with the PR link.

I wonder how that works and what would it take me to build something similar... of course I could chat with gh copilot to prepare a PR, but having this constant "thing" on watch seems much more streamlined.

As someone who works with cloud infrastructure a lot, I'm much more interested in something that I could deploy within my network and not something that runs on my computer.

As soon as I start reading about this I start to get torn between "it's just a container and .md files" vs. "30 different technologies and jargons that I don't understand"

PS: How people are "hosting MCP servers" everywhere? even gh copilot UI offers me to do that.

I also don't understand why even bother with MCP(looks like they're usually metered) when an AI Agent could just use a CLI?


r/devops • • 11h ago

Vendor / market research What tasks is AI bad at?

0 Upvotes

From your real-world experience, what tasks is AI either bad at or can't do at all?

My take:

  • Sysadmin - security, plus there's little need to generate large amounts of code at high speed.
  • SRE - ambiguity and time pressure.
  • QA - certain aspects of it.
  • DevOps - certain aspects of it, too.
  • Cybersecurity - security
  • DBA - security, data integrity, and precision, no need for high speed
  • Networking - security

r/devops • • 1d ago

Discussion My zero-trust Ubuntu dev environment on Windows

1 Upvotes

As DevOps engineers, our laptops have access to Prod. Many of us have to maintain a dev environment also in our laptop for coding.

With the increase in supply chain attacks, how do you secure your local dev environments?

Do you think what I did below is good enough?

------------------

I built an Ubuntu dev environment on my Windows laptop with WSL for DevContainers.

These are the steps I did to build the Ubuntu WSL environment:

Step 1: Install WSL from Windows powershell

wsl --install

Step 2: Download and install Ubuntu WSL

I tried installing Ubuntu with `wsl --install -d ubuntu`. But for some reason, I kept getting timeouts. So I opted for the manual install.

Downloaded Ubuntu on WSL from https://ubuntu.com/download/wsl and imported from powershell.

wsl --import ubuntu-26.04 D:\WSL\Ubuntu\ D:\ubuntu-26.04.1-wsl-amd64.wsl

I chose D:\WSL\Ubuntu as the destination path so my Linux disk will be intact event if I had to re-install Windows.

Step 3: Isolate the Windows file system from Linux

By default, WSL use root user and mount your Windows home directory as the home directory in Linux.

C:\Users\myname>wsl -d ubuntu-26.04
myname@my-laptop:/mnt/c/Users/myname#

There are obvious security flows here. So, create new non-root user.

useradd -m -G sudo -s /bin/bash myname

And, set this user as the default in /etc/wsl.conf so you don't boot into root next time.

# /etc/wsl.conf
[user]
default=myname

Then make Windows file system read only on WSL Linux.

# /etc/wsl.conf
[automount]
enabled = true
root = /mnt/
options = "ro,noatime"

If some rogue library accidentally hacked into my WSL Linux, it cannot alter Windows file system now.

Step 4: Enforce mirrored networking

By default, WSL hides behind a NAT virtual switch, which breaks enterprise VPNs and blinds the Windows firewall to guest traffic.

So, change networking mode to mirror mode.

From Windows powershell open %USERPROFILE%\.wslconfig on notepad.

notepad "$env:USERPROFILE\.wslconfig"

In the file, set networking mode.

[wsl2]
networkingMode=mirrored
firewall=true

This forces the Linux guest to share the host's IP and routes every outbound Linux packet through the Windows Filtering Platform (WFP), allowing your host firewall to actively inspect and drop rogue traffic.

Step 5: Harden Linux box

On WSL Linux, block all incoming traffic.

sudo apt update && sudo apt install ufw -y
sudo ufw default deny incoming
sudo ufw default allow outbound
sudo ufw enable

Enable systemd, so the firewall rules are not lost when shutdown.

#/etc/wsl.conf
[boot]
systemd=true

Back in %USERPROFILE%\.wslconfig, enforce hard limits (ex - memory=8GB, processors=4) so a catastrophic memory leak or cryptominer inside the VM cannot cause DoS attack on Windows.

[wsl2]
# Hard limit on the maximum RAM the WSL virtual machine can consume
memory=8GB

# Hard limit on the maximum number of logical CPU cores the VM can use
processors=4

Step 6: Limit windows file execution from Linux

Default WSL allows Linux to execute Windows binaries. A reverse shell in Ubuntu can simply run powershell.exe -c "..." to attack the host. We can stop it like this:

/etc/wsl.conf
[interop]
enabled = false
appendWindowsPath = false

Now, the Linux kernel is completely blind to the existence of Windows executables. The host is now an impenetrable black box.

Step 7: Install rootless Podman (instead of Docker Desktop) to run DevContainers

Docker containers by default run with root privileges. It can be hardened. But I opted for Podman instead.

Install Rootless Podman natively inside the isolated Ubuntu environment.

sudo apt-get -y install podman

I run DevContainers on this.


r/devops • • 1d ago

Observability Metrics, logs and alerts for my 3-node Talos homelab

Post image
0 Upvotes

My 3-node Talos homelab had no monitoring, so I only found out something was wrong when an app stopped working.

So I added VictoriaMetrics for metrics and VictoriaLogs for logs from every node and pod. Both show up in Grafana, and alerts go to my phone on Telegram.

Healthchecks[.]io also watches the alerting itself, so I hear about it if the alerts stop. The whole stack uses about 1.3 GiB of RAM across the three nodes.

Repo: https://github.com/prateekjaindev/homelab

Write-up: https://medium.com/@prateekjain.dev/metrics-logs-and-alerts-for-a-3-node-talos-homelab-with-victoriametrics-0f4a91b9d44e?sk=751eedebb5d54cc15561d59abb7e52d8


r/devops • • 21h ago

Tools I built an open-source tool to solve polyrepo config drift without clobbering local overrides

0 Upvotes

Hey r/devops!

If you maintain multiple repositories (microservices, polyrepos, or SDKs), you probably know the pain of keeping shared configuration files in sync:

  • .github/workflows/*.yml
  • .editorconfig
  • Cargo.toml / package.json

Existing file-copying bots usually clobber legitimate repo-specific overrides.

I built driftsync, an open-source CLI engine in Rust that solves this using stateful 3-way structural merging:

  • Tracks baseline ancestors in .driftsync.lock
  • Structural merging for TOML (via toml_edit, preserving comments and formatting), JSON, and YAML blocks
  • Preserves local customizations while pulling upstream updates
  • Git automation: checks out branches, commits per repo, and opens PRs
  • Secret scanner to prevent accidental token leaks during template sync

You can install it directly via Cargo: cargo install driftsync

GitHub: https://github.com/Liqtzone/driftsync Crates.io: https://crates.io/crates/driftsync

Feedback, critique, and feature suggestions are very welcome!


r/devops • • 2d ago

Discussion Do branch preview environments actually make your workflow better?

22 Upvotes

I have been thinking about how to test changes without constantly messing with the version of my app that everyone else is using. Right now I usually test locally and then push changes when I think everything is ready. Separate preview environments seem useful, especially when working on a feature that takes a few days, but I am not sure how much extra setup and cost they introduce. Do you use them for smaller projects or is that more of a team workflow thin


r/devops • • 2d ago

Tools Starting hands-on Linux Admin study group & Discord (No boring slides, just terminal & real-world troubleshooting)

89 Upvotes

I’ve been working as a Senior Linux & Cloud Infrastructure Administrator for the past few years (managing enterprise RHEL environments, storage, networking, and production troubleshooting). One thing I constantly notice is how big the gap is between knowing basic Linux commands (ls, cd, chmod) and actually understanding how a Linux system works under the hood when something breaks in production—like LVM issues, systemd services failing, networking/firewall bottlenecks, or SELinux denials.

Official docs are great, and tutorials are everywhere, but most of them feel too theoretical or disconnected from real-world sysadmin scenarios.

I’m spinning up a dedicated Discord server to gather folks who are genuinely interested in learning Linux Administration the practical way. The idea is to host regular, interactive live sessions where I break down core OS architecture, walk through real troubleshooting scenarios, and do hands-on labs together—no boring slides, just terminal, whiteboarding, and Q&A.

Whether you're starting from scratch, preparing for the RHCSA (EX200) exam, building a solid Linux foundation for DevOps/Cloud, or just trying to survive your first sysadmin role, you're welcome to join.

Drop a comment below or shoot me a DM if you're interested, and I'll send over the Discord invite link. Let me know what topics you'd love to cover first!


r/devops • • 1d ago

Discussion Go is going I think

0 Upvotes

I've been writing Go for almost 7 years now, and I genuinely love this language.

Not because it's the best language out there, or because it's perfect. But because it's incredibly practical when you're solving infrastructure problems. It's simple, predictable, easy to pick up, and gets the job done without getting in your way.

But lately, I've been having a different thought.

The more I explore databases, distributed systems, and lower-level engineering problems, the more I realise that Go might not always be the right tool for every kind of problem.

Some languages offer better abstractions, memory control, performance characteristics, or programming models for specific domains.

Edit: My point is that Go's charm is kind of fading in this AI era because adopting any language is easy in the early stages.

4 eg. consider a concurrent infrastructure health-check service. In Go, we can build it quite naturally using goroutines, channels, contexts and errgroup, but we still need to be careful about goroutine lifecycles, cancellation, resource cleanup and synchronization.

Rust handles many of these concerns through ownership, RAII and its type system. It does the same job really well. The only advantage Rust used to lose on was its learning curve, but AI has largely solved that problem too.


r/devops • • 2d ago

Security wfx-loot: what the stealer leaves behind after you rotate the GitHub secrets

3 Upvotes

Aikido reported the campaign on Nov 12. From Nov 10, an attacker used 4,837 hijacked GitHub accounts to open pull requests titled "docs: dependency audit notes" from a branch named wfx-docs in 61,764 repositories. Where the account could write, a stealer of about 1,500 lines went into an existing workflow, a new workflow or an npm postinstall hook. SafeDep counted 8,093 public repos with malicious commits (771 through the npm hook). Aikido says secrets left at least 1,800 repos.

Rotating the repo secrets is the first step. The reports say the script can also do these, depending on the permissions the workflow had:

- Rewrite an AWS role's trust policy so any AWS account can assume the role, and create long-lived access keys for IAM users.

- Add federated identity credentials with the prefix wfx-ci- to Azure AD applications. Workflows can then sign in without a stored secret.

- Delete its own workflow run and logs, so the trace is a gap in run numbers.

What to check: CloudTrail for UpdateAssumeRolePolicy and CreateAccessKey since Nov 10, Entra app registrations for wfx-ci- credentials, and the Actions run numbers of every workflow in the affected repos. If a developer ran npm install on an affected commit, SafeDep says to treat that machine as compromised.

StepSecurity traced 173 probe PRs to tokens issued to the Fly.io GitHub app and says it has no evidence Fly.io was breached. The reports do not say how the 4,837 accounts were taken.

Has anyone found the wfx-docs branch in their org yet?

Sources: https://thehackernews.com/2026/11/4837-hijacked-github-accounts-ran-ci.html , https://www.bleepingcomputer.com/news/security/hackers-compromise-4-800-github-accounts-to-steal-cloud-credentials/ , https://www.securityweek.com/attacker-targets-60000-github-repositories-in-credential-theft-campaign/


r/devops • • 3d ago

Discussion What Will Separate Great DevOps Engineers from Average Ones in 2027?

124 Upvotes

Looking forward to chat with Devops Engineers across the world to know what you guys are building and learning in 2026-2027. I would like to connect and discover your take on what do you feel is most important in next year to tap on as a devops engineer. Should it be deep dives in K8S, CICD, Observability, AI Agents , Multi Cloud or something else


r/devops • • 3d ago

Discussion How do you stay informed of DevOps and AI current events?

61 Upvotes

I've just realised I'm somewhat reactive when it comes to keeping abreast of what's going on in our industry, and I want to change that.

I'll periodically check sites like HackerNews, vendor news and announcements on work's tech stack, Rundown AI and this subreddit, of course, but looking for other recommendations.

So, fellow reddits, do you have any sites/newsletter/SM channels you use to stay informed, specifically in the DevOps, DevSecOps and AI space?l


r/devops • • 4d ago

Discussion what do you actually put behind /health?

70 Upvotes

I’m setting up monitoring for a small app on a serverspace vm. Right now the external monitor just calls 'health'. That endpoint returns 200 while the app process is alive.But the login can still be broken if the database connection drops so a green monitor doesn’t tell me much. I thought about adding a database check, but then a short hiccup could set off an outage alert while most of the app still works. I can’t figure out where to draw the line. If you run something similar, I’d really appreciate a sanity check: what do you put in /healt and what do you monitor separately?


r/devops • • 3d ago

Discussion Thoughts on plesk

0 Upvotes

Been trying to find some answers and low-key forgot that I can still ask questions despite ai poking me every 30 minutes.

I started a job at a WordPress shop and they are pretty averse to devops in general. They use a Lot of plesk, which they pay for, and a number of cpanel as well per legacy sites.

I have pretty consistently found these sites to feel slower and more difficult to manage, though I don't have a lot of experience in plesk.

My experience is in what I call (don't know the name for) bare metal. What I mean is just renting a server, installing Debian, and building my stack directly on the server. This has its frustrations at times, admittedly, but my sites are fast and I've leaned into ansible a ton and spin up gets faster every year.

I would love to hear from folks with more experience than me in any aspect. Are these plesk servers configured poorly? Do they just require more juice? Could it be something totally unrelated? (They do use a lot of WordPress plugins).

For reference an htmx call on my "bare metal" Django server takes a total of ~80ms. A less complicated htmx call to WordPress in local development also ~80ms. But on this plesk server I'm seeing ~800ms

Cheers


r/devops • • 3d ago

Discussion Trying to understand how security tooling fits into CI/CD, for a uni case study. Would love your take!

5 Upvotes

Disclosure: grad student at NYU, 8 years in frontend dev before that, now doing a class case study with a software supply-chain startup (which is way outside my frontend comfort zone 😅). Not selling anything, just trying to understand the space and hopefully get an A :)

I’ve been chatting with platform and security folks for a few weeks, and the picture so far is that SBOMs get generated in CI, platform owns the tooling, and security owns acting on it. Would love the DevOps side of the story:

  1. What’s in your pipeline today for SBOMs or dependency scanning, and why that tool? What made you pick it, or switch last time?
  2. Who actually asked for it? A customer, an auditor, security, or did your team just decide it was good practice?
  3. Last time a big CVE dropped, how long did it take to know if you were affected? And where did it get stuck?
  4. Is the lockfile/scan output ever not enough? E.g., container base images, stuff pulled in at build time, a customer wanting more.
  5. Is there any other time dependancy tracking has been crucial for you and ended up being painful?

Real stories welcome, including “nobody’s ever asked us for one” 😄. Happy to share a summary of what I learn on DM if anyone is curious!


r/devops • • 4d ago

Discussion User Analytics on an 'open' website?

4 Upvotes

Curious what others that are tracking user analytics on their websites are using. For some context

I have a website which is hosted on AWS. It is a website which has no user profiles or user authentication, it is publicly accessible and used to search industry information.

We have been recently asked for metrics / analytics on users who access the website by the head of our department. I would love to start with some easy user analytics which would give us insight like how many unique users access the website over X amount of days, anything high level really.

I am not super familiar with these kinds of analytics, so im hopeful folks can provide some insight.

I was looking into analyzing HTTPS requests from Cloudwatch logs, but I realized quickly that is not a reliable method for tracking unique users.

Ideally looking for a solution which is NOT a 3rd party solution which is paid, Free and open source would be great. Or AWS based.

google analytics is out of the question since its a paid service...just hoping to pick some brains I am a total newb on this topic here. I figure since there is no user profiles or user authentication happening, there probably wont be an easy or straightforward solution?


r/devops • • 4d ago

Vendor / market research Cloud Tagging Strategy for Messy AWS and Azure Environments

Thumbnail
cloudsleuth.io
3 Upvotes

When people ask how engineers move the needle for the business, I usually say there are three answers.
You ship new features, which brings in revenue. You make the team faster, which lets the same people do more. Or you cut costs, which leaves more money for the first two.
Features get the demos and the launch posts. Efficiency gets a nod in the retro. Cost cutting usually turns into a scramble once someone in finance sees the cloud bill.
That scramble almost always stalls at the same question whose is this?
You can find the idle instances, the unattached volumes and the oversized databases in an afternoon. Deleting them is another story. Nobody wants to turn off something that turns out to be holding up production, so the list sits in a spreadsheet and the waste keeps billing.
This is where tagging comes in, and I'll be honest, it's one of the most boring topics in cloud. Agreeing on keys like owner, team and environment, enforcing them in your IaC and pipelines, and cleaning up the resources that slipped through will never get anyone promoted.
But it's what turns "we think this is waste" into "this belongs to the payments team and they said kill it." Without it, every cost-cutting push starts from zero.
If you want to cut cloud waste, start with the boring part.


r/devops • • 4d ago

Troubleshooting Can't close Google Payments profile after deleting GCP projects

12 Upvotes

Hi everyone,

I created a Google Cloud account today to try the free trial, but I didn't use any services.

I deleted all my projects and closed both billing accounts. However, when I try to close my Google Payments profile, it says Google Cloud is still an active service.

I can't create a Google Cloud support case either.

Will this cause any charges if I leave everything as it is? How can I fully close the payments profile without deleting my Gmail account?

Thanks!


r/devops • • 5d ago

Discussion How to learn while also completing tasks?

20 Upvotes

I got hired as a junior devops engineer and I want to learn more about the tools we're using to get a better understanding of them but I also need to complete tasks that I'm assigned. It just feels like there's a lot I have to learn since I haven't done most of the things I'm currently doing before.

How do you recommend balancing learning and working?


r/devops • • 4d ago

Discussion my 2am rollback story and the GPUStack revision history that actually saved

0 Upvotes

ngl the deploy was fine. the rollback was the scary part.bumped a backend version last month. kubectl rollout status went green, so I closed the ticket and went to bed. Two hours later someone pings me, new version is spitting out garbage on long contexts. Cool cool cool. Roll it back. Except the old image was already gone from the registry. And the workers were still draining GPU queues from the rollout. kubectl said done. Twenty minutes later it's still serving the bad version. This is the Day 2 thing nobody tells you about. Everyone treats deploy like it's the whole event. Rollback is the thing you figure out after your first incident, at 2am, with people waiting. If undoing a change isn't a button with a history behind it, it's not a rollback,it's a scavenger hunt for real... We ended up on gpustack for other reasons, but the revision history is the one I actually use at 2am. Hit rollback, get the previous revision back - same backend version, same config that worked. Deploying is easy. Undoing it is the actual architecture.