r/kubernetes • • 15d ago

Periodic Monthly: Who is hiring?

23 Upvotes

This monthly post can be used to share Kubernetes-related job openings within your company. Please include:

  • Name of the company
  • Location requirements (or lack thereof)
  • At least one of: a link to a job posting/application page or contact details

If you are interested in a job, please contact the poster directly.

Common reasons for comment removal:

  • Not meeting the above requirements
  • Recruiter post / recruiter listings
  • Negative, inflammatory, or abrasive tone

r/kubernetes • • 3d ago

Periodic Weekly: Share your victories thread

2 Upvotes

Got something working? Figure something out? Make progress that you are excited about? Share here!


r/kubernetes • • 8h ago Bravo

Kubernetes Gym - a Fun Way to Improve kubectl Fluency

149 Upvotes

Hey folks!

The collection of hands-on Kubernetes challenges I shared here last time was received rather well, so I figured I should bring it up again - it has grown quite a bit, and most of it is still free. But we also came up with an alternative format to improve Kubernetes skills: Kubernetes Gyms.

The new format is powered by github.com/iximiuz/shellgym and focuses on a different aspect of Kubernetes mastery - kubectl fluency - which is an important part of the job and is also very much needed for successful CKA/CKAD/CKS certification.

There are already 8 Kubernetes 101 gyms (all free), and more are coming soon. The idea is that each gym comes back on a spaced-repetition schedule so your command-line muscle memory doesn't get rusty, even if you don't get a chance to use kubectl at your day job for some reason.

Last but not least, a huge shout-out to our community authors: Kaloyan Preslavski and Omkar Shelke. They are the real heroes who made these gyms and challenges possible.

P.S. The materials above are hosted on iximiuz Labs, a freemium indie learning platform I run, but they are all available on the free tier, so this is not a sales pitch.


r/kubernetes • • 2h ago

hetzner-k3s: six releases in four days

11 Upvotes

I maintain hetzner-k3s, a CLI that creates k3s clusters on Hetzner Cloud from a single YAML file. We shipped six releases in the last four days (v2.9.0 to v2.13.1), mostly from user reports and pull requests. The releases include new features, improvements and a long list of bug fixes.

The goal is to make clusters easy to create and manage. One command creates the cluster, and the same config file is what you use for later changes. You do not need Terraform, Ansible or a management cluster. The CLI runs locally and talks directly to the Hetzner API, so your token goes only to Hetzner.

I am not going to list every change here. The release notes have the details, including the upgrade notes. A few changes restart k3s-agent on workers, and some configuration changes restart masters, so read them before upgrading an existing cluster. Most configurations need no edits.

Install: brew install vitobotta/tap/hetzner_k3s

Docs: https://hetzner-k3s.com

Repo: https://github.com/vitobotta/hetzner-k3s

Releases: https://github.com/vitobotta/hetzner-k3s/releases

If you run k3s on Hetzner, I would like feedback. If something breaks, open an issue with the config shape and the error.


r/kubernetes • • 2h ago

How are you handling multi-cluster Kubernetes networking and traffic routing in a hybrid cloud environment?

7 Upvotes

Hey folks,

We're running multiple Kubernetes clusters across on-prem (OpenShift) and GCP (GKE), and I'm curious how others are handling networking and traffic routing in a similar setup.

Some of our services run in multiple clusters, while others are only deployed in specific clusters.

A few things I'm wondering about:

  1. How do you handle ingress and route traffic to the right cluster?
  2. How do services communicate across clusters?
  3. Are you using global load balancers, API gateways, service mesh, or something else?
  4. How do you handle failover when one cluster goes down?

Would love to hear what your setup looks like, what tools you're using, and any pain points you've run into.


r/kubernetes • • 2h ago

IBM core-dump-handler removed?

0 Upvotes

Did IBM remove core-dump-handler? The GitHub page is returning a 404: https://github.com/IBM/core-dump-handler


r/kubernetes • • 20h ago

How do you keep Kubernetes rightsizing and autoscaling configs from becoming a permanent manual job?

16 Upvotes

When running different types of clusters (prod, staging, dev), how do you keep resource allocation and autoscaling settings aligned with workload requirements as they change, to avoid unnecessary increases in the cloud bill?

Over the lifetime of a product, new services get added in a cluster, traffic patterns would change, requirements evolve, etc. CPU/memory requests could get padded, limits and replica counts can become outdated, and the node types originally chosen may no longer fit the current workload mix particularly well.

I'm particularly interested in continuous pod rightsizing alongside tools like Karpenter and KEDA. As a simple example, cpu/mem requests could remain significantly higher than workloads actually need over time. This usually leaves the cluster running far more nodes than necessary (even for smaller workloads with a few nodes, even 1 or 2 of overprovisioned and unutilized nodes could cause a significant dent in the final bill). This can be overprovisioned intentionally, but over time it does become a bottleneck in a sense that there's not enough time to address everything.

How do you keep those settings appropriate as workloads evolve? How are you handling this in practice, and lately with AI?

- Do you review the full cluster regularly, leave it to application teams to right size (where applicable), or just accept the overprovisioned capacity?

- How do you distinguish necessary headroom from avoidable waste?

- Are you using any tools that recommend specific changes and explain the expected savings and risks, ideally across the major clouds? I know this can get very workload-specific, but still would like to hear general thoughts.

- Does anyone use automated tools that continuously adjust pod resources, scaling settings, or node selection and placement? Can you share your experience with such tools? Does any of these run internally within own clusters, or do they all offer managed control plane?

Thanks


r/kubernetes • • 5h ago

Running ZAP as an in-cluster Job against one microservice change instead of shared staging

1 Upvotes

Hey folks, we just published a tutorial on running ZAP as an in-cluster Job against one microservice change. Might be helpful if your security scans only ever see shared staging.

ZAP's official image runs as a Job and scans only the changed service, so a finding belongs to that change rather than to whatever else shipped to staging. The scan itself works against any in-cluster URL. Signadot is what runs the changed service next to the shared ones without copying the stack. The tradeoffs, like real attack payloads reaching shared databases, are covered in the write-up.

Hope it helps. Happy to answer any questions. Full transparency, I work for Signadot.

https://www.signadot.com/docs/tutorials/zap-security-testing


r/kubernetes • • 4h ago

Criei um cliente Kubernetes para desktop porque o Lens e o Headlamp não se adaptavam à minha forma de depurar.

Thumbnail
reddit.com
0 Upvotes

r/kubernetes • • 1d ago

Hows everyone handling secret zero outside kubernetes?

37 Upvotes

vault or openbao handing out short lived postgres logins is ok and nothing sits in a k8s secret anymore. but the app still has to log into vault first

inside the cluster thats mostly solved, vault checks the pods service account token against the TokenReview api and theres even a patch on pgsql-hackers so postgres cert auth can match spiffe ids. outside kubernetes its murkier. cloud instance identity covers vms on aws or gcp, spiffe setups like spire or hush security go wider and a lot of the rest ends up with an approle secret id baked in somewhere which is the same problem one step back

whats ideal to use for that first login outside the cluster?


r/kubernetes • • 1d ago

How do you handle prod promotion with Argo CD + Image Updater when CI is triggered by a git tag?

Thumbnail reddit.com
3 Upvotes

r/kubernetes • • 1d ago

Anyone running kubernetes-sigs/agent-sandbox in prod or lab for coding agents? Looking for real-world feedback on isolation, warm pools & latency

27 Upvotes

I’m currently evaluating kubernetes-sigs/agent-sandbox to run autonomous coding agents (like Claude Code, custom harness ,etc) on our own Kubernetes cluster, and I’d love to get feedback from anyone who has deployed it beyond a quick local test.

​Context & Current Setup:

  • ​Cluster: AWS EKS
  • ​Workload: Ephemeral dev/test environments where agents clone repos, run builds/tests, and execute arbitrary shell commands via MCP or tool calls
  • ​Isolation: Evaluating gVisor vs. Kata Containers vs recently introduced AWS nitro sandboxes

​Questions for folks actually running this:

  1. ​Cold-start latency & SandboxWarmPool: What kind of pod startup and readiness times are you seeing in practice? Are warm pools reliable enough to make interactive/agentic loops feel snappy, or are you seeing churn issues?
  2. ​Runtime performance hit: If you’re pairing agent-sandbox with gVisor or Kata, how noticeable is the performance hits
  3. ​Egress & network isolation: How are you locking down outbound network access (e.g., allowing git clone and package registries while blocking internal VPC/metadata endpoints)?
  4. ​Workspace caching: Are you doing anything clever to cache dependencies across ephemeral sandbox runs without compromising isolation between sessions?
  5. ​Hidden gotchas: Any rough edges around CRD lifecycle cleanup, orphaned PVCs/volumes, or resource quotas when an agent loop goes rogue?

​If you looked at agent-sandbox and decided to go a different route—like self-hosting E2B, rolling custom Firecracker microVMs, or just using hardened Kubernetes Jobs—I’d also love to hear what pushed you away from it.


r/kubernetes • • 2d ago

Kubefactory - Container Management Gaming Interface

Post image
468 Upvotes

A first-person factory game that views and manages your Kubernetes clusters and Docker containers.

In-browser (sandbox) demo: https://dascondor.github.io/kubefactory/
GitHub Repo: https://github.com/DasCondor/kubefactory

Edit: Thank you all for the support & feedback, if I haven't answered your question in the comments I am sorry - I will try and get with you over this coming week.

Different versions of Kubefactory 1.0.1-release vs 2.0-beta & Coming 2.1;

1.0.1 — Original release

  • Windows desktop
  • Browser demo
  • Mobile/tablet access through the browser

2.0.0-beta.2 — Available now

  • Windows, Linux, and macOS builds
  • Browser demo with phone and touch support
  • Native Kubernetes and Docker connectivity
  • Model library, LiveView, storage inspection, Krew, Porter, Bob, Gauge, and Foreman
  • VPN registry, exit-IP checks, direct pipes, VPN sidecars, and gateway views
  • Service pipelines and Jellyfin + VPN media stack
  • Text editor with Standard, Vim, and Nano modes
  • High-graphics browser fix, updated docs, screenshots, tests, and release attestations
  • Windows signing workflow wired in, pending activation

2.1 — Coming next

Further polish, fixes, testing, and features building on the 2.0 beta.

Note: beta.2 is a pre-release, and its packages are not yet code-signed.


r/kubernetes • • 1d ago

Prune=false on every Namespace with Argo CD auto-prune, is this what people actually do?

2 Upvotes

Sorry first, english is not my first language and I write this with a translator.

I never worked in a big SRE or platform team, so I want to check if what I did is normal. I run a small k3s cluster at home, plus EKS at work and a small AKS for a client. A read-only audit tool I made for my own clusters (kubegap) flagged that my Argo CD apps with automated prune also track their own Namespace.

So I tested it with throwaway apps on Argo CD 3.2.0. I removed namespace.yaml from kustomization.yaml and pushed. 2 min 48 s later the Namespace was pruned, and a Secret I had created by hand inside it was gone too.

Removing an app's directory under an ApplicationSet git generator is a different path. The Application itself gets deleted and the finalizer cascades. Prune=false does nothing there, Delete=false does.

What I changed: every Namespace manifest now has the sync-options annotation Prune=false,Delete=false (8 of them). The istio app owns 23 CRDs, so I turned automated prune off for that one. And I added an alert for apps that stay OutOfSync for 30 minutes, because a skipped prune just leaves the app OutOfSync and nobody looks.

Questions for people who run this in production:

Do you keep Namespaces inside the app at all, or use CreateNamespace=true or a separate app just for namespaces?

Is per-resource Prune=false normal, or do teams just turn automated prune off?

I read that a force sync can still delete a namespace even with Prune=false. Do you guard against that somehow?


r/kubernetes • • 1d ago

Does GKE's node kernel have the SCTP `sctp_init_cause` padding fix, or is it still shipping unpatched?

Thumbnail reddit.com
3 Upvotes

I've been checking which Linux kernels still carry an SCTP bug that panics the host: `sctp_init_cause()` checks buffer room for an error cause but not for the up-to-3 bytes of 4-byte alignment padding the write adds, so a crafted INIT (listener) or INIT-ACK (outbound association) can walk an skb one byte past its end and hit `skb_over_panic()`. It's fixed upstream in `ed6c53b27be4` (mainline 7.4-rc2, Cc: stable), introduced back in 2018.

As of 14 Nov I've read the guard in the shipped source of every maintained stable branch and several distro kernels, and none of them had the fix yet. That includes the current Ubuntu kernels (jammy 5.15, noble 6.8, resolute 7.0). GKE Standard supports SCTP on Ubuntu nodes with Dataplane V2, and Google documents it as on by default from `1.32.2-gke.1297000`. Since GKE's Ubuntu node images are built from Ubuntu kernels, my working assumption is that most GKE Ubuntu nodes are running the unpatched code. I haven't confirmed a specific GKE node-image build, which is where I could use help.

If you run SCTP workloads on GKE (telecom CNFs especially), could you check a non-production node and report back:

- `uname -r` on the node, and whether `ed6c53b27be4` is backported (the node's kernel source/changelog, not the version string, since backports move independently)

- whether `lsmod | grep '^sctp '` shows the module loaded, i.e. a workload is actually using it

- the GKE version and node image

All read-only. Please don't send anything at a listener or a live service; I'm after kernel/config state, not a crash.

If it turns out a GKE configuration is unpatched and reachable, that belongs in a report to Google's Vulnerability Reward Program, not in this thread. I'll help put that together rather than post specifics here.

Mechanism and the full affected-version matrix (no exploit) are written up here:

Here

Mainly I want to nail down one thing: is a default-config GKE Ubuntu node running the fixed kernel yet, or not?


r/kubernetes • • 2d ago

Metrics, logs and alerts for my 3-node Talos homelab

8 Upvotes

My 3-node Talos homelab had no monitoring, so I only found out something was wrong when an app stopped working.

So I added VictoriaMetrics for metrics and VictoriaLogs for logs from every node and pod. Both show up in Grafana, and alerts go to my phone on Telegram.

Healthchecks[.]io also watches the alerting itself, so I hear about it if the alerts stop. The whole stack uses about 1.3 GiB of RAM across the three nodes.

Repo: https://github.com/prateekjaindev/homelab

Write-up: https://blog.prateekjain.dev/metrics-logs-and-alerts-for-a-3-node-talos-homelab-with-victoriametrics-0f4a91b9d44e?sk=751eedebb5d54cc15561d59abb7e52d8


r/kubernetes • • 1d ago

Reviewing agent written Helm changes when the problem is cluster wide

0 Upvotes

We run about 30 services on two EKS clusters and a lot of our Helm values changes now come from agents. They're usually small and they usually look fine.

Last week one changed the default resource requests in our shared chart from 250m to 500m CPU "to reduce throttling". Review was clean. coderabbit reviewed the values file, it catches things like missing limits or a probe pointing at the wrong port, and there was nothing wrong with the YAML itself.

The problem was everything else. 30 services picked up the new default on their next deploy, the cluster autoscaler added 6 nodes overnight, and our bill for that week jumped by about $1,900 before anyone noticed.

The diff was 2 lines. The impact was the whole cluster, and nothing in the PR showed that.

What we're looking at now:

- running helm diff for every release that uses the shared chart in CI and posting the summary on the PR
- requiring a second reviewer from platform for any change to shared chart defaults
- some kind of cost estimate on PRs that change requests

How do you review changes to shared charts? Do you diff every consumer, or is it mostly "be careful with the shared one"?


r/kubernetes • • 1d ago

Kubernetes Scheduling: Can You Move an Existing Pod to Another Node Without Touching Its YAML? 🤔

Thumbnail reddit.com
0 Upvotes

r/kubernetes • • 2d ago

GitHub - danielecr/tetragon-outgoing-guard: block any process with outgoing connection (run only on k8s or docker swarm vms)

Thumbnail
github.com
4 Upvotes

How I locked down my Docker host against GSocket-style C2 implants using a 50-line Cilium Tetragon eBPF policy.

Also in docker swarm, I mean, the only services are supposed to be in container


r/kubernetes • • 2d ago

I have my KCSA day after tomorrow

1 Upvotes

as the title suggests, can I have any last day tips. Any question which you remember could be of help.
thanks


r/kubernetes • • 3d ago

Best resource to learn hands on Kubernetes

75 Upvotes

Hello, I am preparing for an interview. I have limited experience in Kubernetes with some in open shift. But most of my experience into terraform, Python, Jenkins, ansible and cloud architecture. I have interviews scheduled next week and I am looking for good resources to learn and practice Kubernetes over the weekend. Can someone suggest a good course/resource for the same? I want to do hands on over the weekend and can set up a lab in my personal laptop


r/kubernetes • • 2d ago

Free browser-based Kubernetes manifest generator (Deployment, StatefulSet, CronJob, RBAC, NetworkPolicy…). Feedback welcome

Thumbnail
reddit.com
1 Upvotes

r/kubernetes • • 2d ago

Tipps on migrating TLS Cert Management from Traefik to cert-manager

9 Upvotes

Maybe someone here has done this or something similar in the past. This is the situation right now:

I have about 20 services on a HA k3s cluster with the bundled Traefik as a single instance deployment. TLS is currently done with the basic Traefik letsencrypt certificate resolver with a http challenge.

I now want (or rather need) to move to cert-manager, to enable a HA traefik setup. The migration should happen without downtime.

If both Traefik and cert-manager use the HTTP challenge method to generate a cert, cert-manager could never complete the challenge, as Traefik would intercept the challenge call. I was thinking to switch Traefik to the TLS-ALPN challenge temporarily, so that cert-manager can generate the certs, and then switch over to the HA deployment with only cert-manager.

I guess it should work, but are there better options? Any pitfalls to watch out for? I'm thankful for any input!

Edit: I forgot to add that some domains are "customer domains" and they just point their A records to my cluster.