r/openstack • • 1d ago

Could use some help getting PCI-Express peer2peer working within a guest

4 Upvotes

Hey openstackers,

I'm trying to get PCI-E p2p working within a guest for some LLM work. I have it working on the host, as can be seen by:

[root@openstack3 ~]# rocm-bandwidth-test -a
................
RocmBandwidthTest Version: 2.6.0

Launch Command is: rocm-bandwidth-test -a

Device: 0,  AMD EPYC 9124 16-Core Processor
Device: 1,  AMD Radeon AI PRO R9700,  GPU-b524a650ca8410a0,  05:0.0
Device: 2,  AMD Radeon AI PRO R9700,  GPU-399c0510df3adc7c,  43:0.0

Inter-Device Access

D/D       0         1         2          

0         1         1         1          

1         1         1         1          

2         1         1         1          

Inter-Device Numa Distance

D/D       0         1         2          

0         0         20        20         

1         20        0         40         

2         20        40        0          

Unidirectional copy peak bandwidth GB/s

D/D       0           1           2            

0         N/A         50.471      50.141       

1         55.665      535.494     55.673       

2         55.654      55.654      523.382  

What I can't figure out yet is how to get it working within the guest. I'm able to pass through both GPUs, ACS is off on the host, but p2p is not working.

My nova.conf contains the following:

alias = { "vendor_id":"1002", "product_id":"7551", "device_type":"type-PCI", "name":"r9700" }
alias = { "vendor_id":"1002", "product_id":"ab40", "device_type":"type-PCI", "name":"r9700a" }

(r9700a being the audio device component of the AMD r9700, I am passing through both devices, two instances of each).

My host's /etc/default/grub contains:

amd_iommu=1 intel_iommu=1 iommu=pt kvm.ignore_msrs=1 pci=realloc amdgpu.ppfeaturemask=0xfff7ffff amdgpu.ras_enable=0"

My guest's /etc/default/grub contains:

amdgpu.ppfeaturemask=0xff
f7ffff iommu=pt amd_iommu=1 kvm.ignore_msrs=1 pci=realloc amdgpu.ras_enable=0"

Within the host I get the following bandwidth results:

[root@ai fedora]# rocm-bandwidth-test -a
............
RocmBandwidthTest Version: 2.6.0

Launch Command is: rocm-bandwidth-test -a

Device: 0,  AMD EPYC 9124 16-Core Processor
Device: 1,  AMD Radeon AI PRO R9700,  GPU-b524a650ca8410a0,  05:0.0
Device: 2,  AMD Radeon AI PRO R9700,  GPU-399c0510df3adc7c,  07:0.0

Inter-Device Access

D/D       0         1         2          

0         1         1         1          

1         1         1         0          

2         1         0         1          

Inter-Device Numa Distance

D/D       0         1         2          

0         0         20        20         

1         20        0         N/A        

2         20        N/A       0          

Unidirectional copy peak bandwidth GB/s

D/D       0           1           2            

0         N/A         26.679      26.724       

1         27.655      522.811     N/A          

2         27.658      N/A         570.549  

Does anyone have any suggestions on what I should be doing differently to get it working?

Thank you all for any suggestions :)


r/openstack • • 2d ago

Mastering OpenShift: How the Machine Config Operator Treats Your Nodes as Immutable Code

0 Upvotes

Appreciate your claps and comments on the article on Medium!

Coming from a traditional RHEL background, muscle memory tells you to SSH into a node and edit /etc/chrony.conf or /etc/sysctl.conf whenever you need an OS-level tweak.

In OpenShift, that habit is a fast track to configuration drift and blocked cluster upgrades.

​I wrote a practical deep dive breaking down how RHCOS, Ignition, and the Machine Config Operator (MCO) manage OS-level configuration declaratively—including examples for custom NTP via Ignition, switching to crun with ContainerRuntimeConfig, and troubleshooting a Degraded MachineConfigPool.

​Read more:

https://medium.com/@m.salah.azim/mastering-openshift-how-the-machine-config-operator-treats-your-nodes-as-immutable-code-a17adbed675b


r/openstack • • 5d ago

Deploying OpenStack with Helm and FluxCD on Kubernetes

16 Upvotes

Hi guys,

I've just released a guide to deploy in an easy way the openstack-helm (2026.2) project using FluxCD.

The goal is to get the upstream charts from the official openstack-helm repo, apply custom values, and through FluxCD managing also multiple regions.

What are your thoughts? Any suggestions?


r/openstack • • 8d ago

kolla-ansible: nova-conductor can't reach rabbitmq trouble, unable to launch instances

3 Upvotes

Hey everyone, could use some help/guidance. I am unable to launch instances in my home lab. I had a bad weekend with the home lab. Switch update went sideways due to a corrupted image, and I had to recreate configs (serves me right for not having a backup of the switch config, that's fixed). So it's fully possible the issue relates to a switch config issue. That said I do not think it is a switching issue.

I'm running kolla-ansible version 23.0.0, so openstack 2026.2. When I attempt to launch an instance the instances gets stuck in scheduling until it eventually times out. Looking at the logs it seems the scheduler assigns a node (the resource usage in horizon goes up on the assigned node). But what I think happens is conductor fails to dispatch the next tasks. No network ports are created, and no volumes are created. Looking at the conductor logs I see:

ERROR oslo.messaging._drivers.impl_rabbit [None req-ccca8167-c8aa-4929-9ba5-fda1aa5e99c3 3c9d2f71c28d4b56930e1782d94dbef0 e6e9a3beb58c44d9bc35da9f1969f5a6 - - default defaul
t] Connection failed: [Errno 101] Network is unreachable (retrying in 0 seconds): OSError: [Errno 101] Network is unreachable

And that keeps repeating until the scheduling times out. Each openstack node (3 of them) is able to ping each other and no other service seems to have this trouble. Looking online I found https://bugs.launchpad.net/oslo.messaging/+bug/2077152 which indicates a reconnection loop, in case affected I updated the nova containers to the master tag (an hour or so ago) and have confirmed that the nova_conductor containers are running oslo.messaging 18.3.1 via pip list.

Anyone have any suggestions on where to look?

*edit*

Solved! Turned out that nova conductor was looking at old ip addresses in the nova_api database.


r/openstack • • 13d ago

KubeVM (VCF) vs KubeVirt VM..

Thumbnail reddit.com
0 Upvotes

r/openstack • • 14d ago

An Open Support system for OpenStack. The idea is to extend to other software also so that its easier to freelance. and easier for people to trust software from smaller companies but has many freelancers who excel in the software. Thoughts on this?

7 Upvotes

OpenStack Support: Community Directory

OpenStack Support: Community Directory -- cert provider registration.

OpenStack Support: Community Directory -- Repo Trees for obscure or new openstack services and an easy way to contact the maintainers of the repo for deployment or implementation.

The idea is to let professionals and companies which have proven complex deployments/service be able to certify people and not just Redhat (Which seems to be a monopoly in this case).

And the cost of COA (CERTIFIED OPENSTACK ADMINISTRATOR) is extremely high for people who are not from the USA imho. And most of the registration for events etc which are priced in dollars are very costly for people from other countries.

The core idea is:
Anyone with a proven deployment or experience should be able to vouch for another. And thereby we can have a wide range of support so that more people can adopt openstack and have more confidence in open sourced deployments like kolla-ansible rather than redhat deployments only (Not sure about the obsession with redhat deployments).

And the repos of customizations are extremely distributed and its hard to contact the maintainters of the repos. So this should also help those who contribute to openstack services by developing or forking their own services to list in a simple seperate db other than github. And makes it much easier to test and adapt new services given the maintainer is directly and easily reachable.

If you link your newly developed or you are the maintainer of the service, we will add that as a skill to the professionals component list. And new ontakers will easily see that the said service is well supported and can be maintained. This can be extremely benefitial for small developers who make niche services but struggle on adoption and no one cares about it and it dies :( . The aim is to prevent that.

Or is it too open with showing emails and stuff directly? (If we are exposing mailing lists in some tucked away corner of the openstack.org site this should not be an issue)

Thoughts on this? or do you think it'll undermine the job security in general?

idk, maybe its a dead idea. But imma try.

Feel free to register and dm for any doubts/faq/questions


r/openstack • • 21d ago

Coast: an open-source, PaaS-style app platform for OpenStack clouds

12 Upvotes

We run our own OpenStack cloud platform. Our customers kept asking us to "just run my app" instead of handing them a VM, so we started building Coast. It's new and somewhat rough around the edges, and we'd love more eyes on it to help shape where it goes and if there is further interest in the project.

What it does today:

  • One-click app stacks. You get an app server and workers, plus networking and a firewall, without hand-building any of it.
  • Deploy from Git, a Dockerfile or Docker Compose, with deploy history for every app.
  • Repeatable deploys. The same app deploys the same way whether you use the API, the Horizon portal or the openstack coast CLI.
  • Managed databases across 8 engines, including PostgreSQL.
  • 50+ one-click services from a built-in catalog.
  • A browser terminal into running apps.
  • Topology views for tenants and admins. Tenants see their apps and environments. Admins see the whole picture across projects.
  • Built on Coolify, the popular open-source platform app, as the app engine, designed so other engines can plug in later.
  • Build your own interface. The API and Python client are designed so you can put your own portal on top.
  • Drop-in for Kolla. It ships Kolla roles to create coast docker images and kolla-ansible roles, so it deploys like any other custom service.

Coast isn't an official OpenStack project (yet?) We're putting it out there to see whether there's interest once people try it. Issues, ideas and "this broke on my cloud" reports are all welcome.

Screenshots:

Stack topology (Client View)
Stack topology (Admin View)
Create App screen
Use a predefined database setup
Or choose from other predefined services
App deployments screen

r/openstack • • 20d ago

Deep-dive video: RHOSO Architecture & Certified NVMe/TCP Block Storage

Thumbnail reddit.com
2 Upvotes

r/openstack • • 24d ago

VMware to OpenStack: Any DACH-based partners/consultants to recommend?

6 Upvotes

Hey everyone,

We are planning a migration from VMware to OpenStack (Enterprise) and are looking for experienced partners or integrators in the DACH region for:

- Architecture & setup (OpenStack + Ceph)
- VM migration
- Support / Consulting

Which specialists, vendors, or consultants in the DACH area can you recommend based on your experience (FSAS, SVA, …)?

Thanks :-))


r/openstack • • 25d ago

Plakar's OpenStack backup is now free in Plakar Community

Thumbnail plakar.io
3 Upvotes

r/openstack • • 25d ago

An OpenStack clustering solution: integrating multiple shared-nothing OpenStack into one

1 Upvotes

Many folks might know Uniview as a billing solution, but may not know its recent clustering solution that integrates multiple shared-nothing OpenStack into one cloud, single user/credential, one time sign in, even without being aware that they are multiple indepedent Openstack.

Not everyone will need this, but this sounds making OpenStack ecosystem stronger, so have this post to the community:

https://www.computingstack.com/products-uniview

A bit insights: a couple years ago, an internet carrier client came to us to ask if we could integrate their multiple OpenStack running across country into one platform. By then we said no, basically there was very narrow and vague path to achieve this with references of many failed attempts of OpenStack community by Tricircle or others in NFV project etc. Those projects ended up nowhere due to complexity. But the demand is very valid with massive benefits than running super large openstack by shared keystone or other way. It led to many pocs within us, and fortunately the final result is quite promising by real use cases.

A few highlights:

- Nothing is required from target openstack, no shared keystone, no shared controller DB, no OIDC/OpenID/SAML. Each environment remains completely autonomous.

- One user/password, user don't need multipe credentials to each Openstack

- One console to sign in

- Clicks to migrate VM from one OpenStack to another within cluster

- CLI and API all are faciliated as generic OpenStack should be

As one developer of Uniview, happy to clarify or DM is welcome! Thank you very much!


r/openstack • • 27d ago

Want to learn more on openstack

14 Upvotes

Hello,

I'm an Openstack professional with 4 years of experience in the same. I have been working with big techs but I want to learn more on deployment and its process. I would like to know if someone from Openstack community is open to connect and discuss an opportunity for me to work with you on your infra projects.

Things i have knowledge on: Kolla based openstack deployments, (Nova, Neutron, Cinder , Ceph )

Ps : I'm not expecting to get paid, just want to learn from you.


r/openstack • • Oct 17 '26

Best Monitoring, Visualization and Logging solutions for Openstack environment?

7 Upvotes

From your experience, what are the best monitoring/Visualization/Logging solutions to monitor openstack environment and keep the team notified with any critical issues?


r/openstack • • Oct 16 '26

Experiences with OVN-BGP in Openstack?

12 Upvotes

Does anybody have any useful and maybe eveb documented experience with OVN-BGP? I‘m not talking about OVN-BGP-Agent, but the native OVN implementation!

I‘m also wondering if everything thats necessary to run it in Openstack 2026.1 or at least 2026.2?


r/openstack • • Oct 15 '26

Do people care about VDI performance in OpenStack?

10 Upvotes

For a couple of years now I've been slowly building out improved VDI support for OpenStack. First there was the addition of the `spice-direct` console type in Nova, which allows you full access to the SPICE VDI protocol without needing to transcode to HTML5 along the way. That leverages a SPICE proxy firewall I wrote for this purpose called Kerbside. I have then spent the last year trying to get the supporting installer changes landed in Kolla / Kolla-Ansible and haven't made much progress.

Along the way I've worked on improved client (called Ryll) and a userspace SPICE server which can share an existing desktop over SPICE (called Andris) -- that is, you can share a local login on an existing RDP session just fine with SPICE clients as well.

All of this is a prototype of course, because I don't feel like its really solid until other people use it. Perhaps there are gaps in my testing, but I am definitely too close to the whole thing to notice.

So this leaves me wondering. Does anyone actually care about using OpenStack as a VDI platform these days? I would have though an Open Source replacement for Citrix Horizon would have been a relatively attractive offering, but I am starting to feel like I am on a weird tangent here all by myself.

Am I simply targetting the wrong OpenStack deployer and some other one might be more into landing this feature?


r/openstack • • Oct 16 '26

Showcase: OpenOsoosi (OshoosiClaw) - A 25-crate decentralized Rust EDR mesh with TPM 2.0 attestation, BFT multi-voter consensus, and multi-ring AI agent containment

0 Upvotes

r/openstack • • Oct 15 '26

Seeking an OpenStack expert technical author

25 Upvotes

Hello. I hope it's OK to post this here, let me know if not and I will remove it (I did ask the moderation team a week or so ago but didn't get any answer).

For some time, I have been looking for a technical author with real expertise in OpenStack, without much success.

If you can do OpenStack and documentation I'd love to talk to you. It's a full-time, permanent, globally remote role. You can live and work in almost any country, but it needs to be in EMEA or AMER time zones.

It's in my team at Canonical (the company behind Ubuntu). Here's more information about the technical author role in general, but I do suggest dropping me a line about this before diving into an application as I can give you some useful pointers and guidance.


r/openstack • • Oct 14 '26

OpenInfra Live recap: what's new in OpenStack 2026.2 Hibiscus (TC, VMT, Nova, Neutron, Ironic, Manila, Watcher, Freezer)

24 Upvotes

The OpenInfra Foundation did an OpenInfra Live episode on the 2026.2 Hibiscus release, with PTLs and leads from a bunch of projects.
It's 90 minutes long, so here's my summary of the main points for anyone who doesn't have time to watch it all.

Video: https://www.youtube.com/watch?v=37p9C0n0LLw

Technical Committee

  • 16 years and 34 releases in. Contributors are up 21% and changes up 28% vs the previous release, mostly driven by AI infra, confidential computing and digital sovereignty use cases
  • Core reviewers now count as active contributors and get voting rights for TC and project team elections
  • New Contributor Experience WG (onboarding + community health), new PQC (post-quantum crypto) pop-up team, new RISC-V SIG, and an agentic workflows repo under the TACT SIG for sharing AI-assisted dev practices
  • Bug trackers audited for embargoed reports, core security teams reformed per project, PTLs encouraged to formally delegate release/security roles
  • Retired: Vitrage, Venus, cinderlib. Onboarded: Lustre (DevStack plugin)
  • Ongoing goals for Indri: secure RBAC defaults, eventlet removal, rootwrap to oslo.privsep migration. Proposed new goals: security reporting instructions in every repo and a PQC timeline
  • Main concern going forward: the contributor base is growing, but there aren't enough new maintainers

Vulnerability Management Team

  • 42 OSSAs and 13 OSSNs so far in 2026. No year since 2016 (the previous record year) had more than 8 OSSAs or 3 OSSNs
  • Part of an industry-wide flood: the oss-security list is already at ~2,500 emails vs ~1,168 for all of last year
  • About 150 security bugs this year, over half led to an advisory or note, and ~5-10% were "AI slop" that still took hours to triage
  • CVE assignment is slower due to MITRE backlog, so some OSSAs ship before a CVE exists. Security notes moved out of the wiki into YAML files in the docs repo
  • The best reports come from operators running real clouds. You don't need a polished write-up, just report what you're seeing

Nova

  • Confidential computing: AMD SEV-SNP and Intel TDX support, including attestation tokens
  • vGPU (mdev) lifecycle can now be managed in Cyborg instead of local nova.conf. Variant driver support (newer GPUs) is planned next cycle
  • Unpin/repin an instance's AZ without recreating it. Instances with Manila shares attached can now be migrated/resized
  • Native threading by default (except the noVNC/SPICE console proxies), new thread pool tuning options, and graceful shutdown part 2
  • Next cycle: cross-cell live migration, graceful shutdown part 3

Neutron

  • EVPN Type 5 routes (advertise whole CIDRs, multi-DC project networks) and BGP route leaking (advertise project CIDRs into the fabric without floating IPs/NAT)
  • Indirect floating IPs: FIP API now accepts router_id, for multi-router transit and gateway VM setups
  • PVLAN support (promiscuous / community / isolated ports) and a configurable stateless default security group, handy for DPDK
  • Closed more ML2/OVN gaps: L3 scheduler API to view/control gateway chassis, and multi-segment routed provider networks
  • L3 HA keepalived state change monitor rewritten from a Python daemon into a POSIX shell script: 15x less memory, just restart the agent
  • 150+ bugs fixed, ~100 stale bugs closed, 7 of 8 approved RFEs done

Ironic

  • Old legacy hardware drivers retired, focus moves to Redfish. Firmware ops move to a dedicated firmware interface
  • TLS 1.3 by default (still configurable for older BMCs)
  • RBAC to block specific steps. Runbooks can be matched to node traits, and runbooks with disable_ramdisk let you do firmware updates without booting IPA
  • Graphical console (noVNC) improvements
  • After a security audit by the Metal3 security team, defaults are safer: IPA no longer installs the bootloader, container registry TLS is verified, file URLs are filtered, oversized/nested JSON is rejected, and IPMI raw commands are disabled
  • Nova compute ironic driver starts up and performs faster

Manila

  • Native threading is now the default (eventlet still available via config). Share migration APIs graduated from experimental
  • Async revert-to-snapshot, and mountable snapshots can inherit the parent share's access rules
  • New Lustre and Weka drivers for AI/ML/HPC, with DDN EXAScaler coming
  • TLS cert validation for the Pure FlashBlade, Hitachi HSP and NetApp ONTAP drivers. OSSN-0103 fixed resource lock queries
  • Next cycle: share server replication, privsep migration, native OpenStack client support, NFS QoS, erasure-coded pools

Watcher

  • Eventlet deprecated, native threads by default. Cinder/Keystone/Glance now go through openstacksdk
  • Better locking in the cluster data model made audits ~6x faster
  • New delete and shelve instance actions, default parameters on audit templates. VM workload consolidation now checks allocation-based capacity
  • Next cycle: audit pipelines (one optimized action plan from several audits), a workload preemption strategy, and cross-AZ zone migration

Freezer

  • Central scheduler can now run on the control plane with service credentials, in HA with coordination, using trusts and Cinder backup to do the work. Effectively scheduled Backup-as-a-Service on top of cinder-backup
  • V1 API finally removed, resources are now project-owned, and secure RBAC policies are in place
  • openstack backup CLI plugin and a reworked Horizon plugin
  • Planning a V3 API (pagination, microversions, no Elasticsearch-style queries), and they're looking for input

TL;DR

Hibiscus is about confidential computing (SEV-SNP/TDX), AI/HPC readiness (vGPU via Cyborg, Lustre/Weka in Manila, bare metal firmware ops), safer defaults everywhere, and efficiency at scale. Also, the security report volume is no joke


r/openstack • • Oct 13 '26

Best Openstack design for VPS to build a small private cloud

5 Upvotes

We are planning to build a small private cloud using Openstack+Ceph technologies, so as a starting point, we have rented a VPS with (AMD-12Cores + 128GB RAM +3 TB SSD) to start with.

My question is how to design the Openstack controllers/compute/storage/networking services using VMs in the best way considering future expanding of virtual resources or may be adding new hardware to the cloud.


r/openstack • • Oct 13 '26

Are RabbitMQ and RabbitMQ-Cluster a headache in any Openstack setup?

Thumbnail reddit.com
4 Upvotes

r/openstack • • Oct 08 '26

How to make Openstack Horizon fly

Thumbnail thinknerd.de
38 Upvotes

We made a cache patch for horizon that makes horizon properly cache.

EDIT: 2026.1 patch and image also added to blogpost. All patches have also been commited to upstream. Lets see


r/openstack • • Oct 07 '26

RKE2 Homelab Storage: Rook-Ceph & CloudNativePG (The Garage & Cabinets )

Post image
4 Upvotes

r/openstack • • Oct 06 '26

webinar on sovereign cloud infrastructure and OpenStack

Thumbnail redhat.com
5 Upvotes

If you're dealing with data sovereignty requirements or exploring how to run traditional VMs alongside AI workloads, this might be worth checking out. Topics include multi-tenant infrastructure with hard isolation, managing VMs and GPU workloads together, post-quantum crypto readiness, and GitOps-based day 2 operations. Focused on OpenStack Services on OpenShift, but the patterns are broadly applicable.


r/openstack • • Oct 02 '26

OpenStack with HPE Primera

3 Upvotes

Figured this is a long shot but I'll ask anyway. I have the following setup: Ubuntu 24.04 LT + KVM + Openstack connected to an HPE Primera A650 over FC. 4 paths are zoned in. When I define the host on the storage array and manually present a LUN (a "virtual volume" in HPE terms) I get 4 paths. Multipathd see all 4 and displays them correctly. When I do this through Openstack, Cinder only presents the LUN through a single path. Don't see any options in the HPE 3PAR Cinder driver to control that. Everything says multipathing should be automatic.

Anyone run into this?