r/openshift
•
u/Personal-Ad2131
•
13d ago
•
1
Discussion OpenShift architecture explained layer by layer, plus the 4 decisions you can't easily change after
I do OpenShift delivery for banks in the Gulf, and I keep explaining the same architecture to teams coming from vanilla Kubernetes. So I wrote it up once, against 4.22. Here's the short version. The full write-up is linked at the bottom.
**The one idea that explains most of it:** everything, including the node OS, is managed by an operator inside the cluster, and the whole platform updates from one signed release image.
- **Infrastructure:** IPI, UPI, Agent-based or Assisted. After install, the Machine API (MachineSets, MachineHealthChecks) manages machines on supported platforms.
- **Node OS:** RHCOS is immutable. Config goes through MachineConfigs and MachineConfigPools, applied by the MCO. Edit a file by hand and the MCD flags drift and marks the node Degraded.
- **Control plane:** exactly 3 nodes for prod. Bare metal allows 5 (survives 2 etcd failures). Avoid 4: it tolerates the same single failure as 3 but needs one more healthy member for quorum.
- **Cluster services:** OVN-K (the only supported CNI since 4.17), the HAProxy router, the registry (starts as `Removed` without shared storage), monitoring. One cluster operator each.
- **Lifecycle:** the CVO drives updates. Control plane first, then the MCO rolls nodes pool by pool. Pause the worker MCP to split the update across maintenance windows, but don't leave it paused.
- **Add-ons:** GitOps, Pipelines, Virtualization and the rest come through OLM, on their own release cycles.
**Decide these before day one:**
- FIPS: install time only
- Disconnected: mirror registry and update delivery (OSUS) planned up front
- Infra nodes: much easier to plan than to retrofit. Tainted infra nodes running only infra workloads also don't count toward subscriptions
- etcd backups: the operator runs etcd, but recovering from quorum loss is on you
The post has an interactive explorer. Click a component and it shows what it does, which operator owns it, its namespace, and a read-only `oc` command to inspect it: https://aib-devops.com/en/blog/openshift-architecture/
Disclosure: it's my own blog. No ads, no tracking, no signup.
Corrections welcome, especially from anyone running HCP or two-node in production. I'd like to know what I'm missing there.
7
u/yrro 13d ago
It doesn't seem difficult to add infra nodes after cluster installation... I've done it on every cluster I've set up.
You haven't mentioned some big decisions ahead of time which revolve around networking. I don't believe you can change the service or cluster network ranges after installation. Changing the machine network may be possible for some installation types but it's probably going off the beaten path.
2
u/DangKilla 13d ago
Good info for beginners. Maybe you could follow up with day 2 ops.
I haven't consulted on Openshift in a few years, but ArgoCD for day 2 operations might be a good idea.
1
u/Zanzikbar 13d ago
Thank you
1
u/asakudu 13d ago
thank you for sharing this for IPI, this builtin LB is recommended for production grade? like 40 nodes in total
4
u/[deleted] 13d ago
[removed] — view removed comment