I'm trying to set up a SET team on Windows Server 2022. I am having issues getting RDMA…
- User
I'm trying to set up a SET team on Windows Server 2022. I am having issues getting RDMA enabled. On the underlying physical NICs it is enabled and functional; but the vSwitch reports the RDMA status as unknown.
- Assistant
It sounds like you're trying to configure Switch Embedded Teaming (SET) on Windows Server 2022 with RDMA (Remote Direct Memory Access) support, but the vSwitch reports RDMA status as "Unknown", even though it's enabled and functional on the underlying physical NICs.
This is a common issue when setting up RDMA-enabled SET for use with SDN (Software Defined Networking) or high-performance computing scenarios such as SMB Direct.
Let’s walk through the key configuration steps and troubleshooting points to help resolve the "RDMA Unknown" state on your vSwitch.
✅ Prerequisites
- Supported Hardware: Make sure your NICs are RDMA-capable (RoCE or iWARP).
- Driver Compatibility: Ensure that the physical NIC drivers support VPort RDMA, which is required for RDMA over a Hyper-V virtual switch.
- Firmware and Driver Updates: Update both firmware and drivers of the physical NICs to the latest version from the vendor.
- Teaming Mode: SET only supports
LACPorStaticteaming modes. Make sure this is configured correctly.
🔧 Step-by-Step Configuration
1. Create the SET Team and vSwitch with RDMA Enabled
New-VMSwitch -Name "SET-Switch" ` -NetAdapterName "NIC1", "NIC2" ` -EnableEmbeddedTeaming $True ` -EnableRdma $TrueReplace
"NIC1"and"NIC2"with the actual names of your physical NICs (as seen inGet-NetAdapter).
2. Verify Physical NIC RDMA Capabilities
Run:
Get-NetAdapterRdmaYou should see something like:
Name Status RunningStatus InterfaceDescription ---- ------ ------------- -------------------- NIC1 Enabled Active Mellanox ConnectX-5... NIC2 Enabled Active Mellanox ConnectX-5...If they’re not active or not supported, that’s a driver/firmware issue.
3. Check vSwitch RDMA Status
Get-VMSwitch | Select Name, EnableRdma, RrdqSupportEnabledMake sure
EnableRdmaisTrue.
4. Check VMQ and VPort RDMA Support
Sometimes VMQ needs to be disabled for VPort RDMA to work properly.
Check VMQ status:
Get-NetAdapterVmqDisable VMQ if needed:
Disable-NetAdapterVmq -Name "NIC1", "NIC2"Reboot may be required after changing VMQ settings.
5. Confirm VPort RDMA Support
Use the following command to check if VPort RDMA is supported and enabled:
Get-VmNetworkAdapterRdma -VMName <YourVMName>Or for the management OS:
Get-VMNetworkAdapterRdma -ManagementOSEnsure it shows:
Status : Success RdmaCapable : True Active : True
6. Use Core Isolation (Optional)
Some RDMA configurations require isolating the RDMA-capable NICs from general traffic using core isolation.
Set-VMSwitch -Name "SET-Switch" -RrdqSupportEnabled $true
🛠️ Common Issues & Fixes
Issue Fix Physical NIC RDMA is enabled, but vSwitch shows "Unknown" This usually indicates that VPort RDMA isn't working. Check NIC driver supports VPort RDMA. RDMA not supported in guest VMs Make sure VM has "Virtual Machine Queue (VMQ)" disabled and SR-IOV is off. Mellanox / Intel / Broadcom specific quirks Refer to your NIC vendor documentation. Some cards require special registry tweaks or advanced properties. Hyper-V Role Not Fully Installed Make sure all Hyper-V components are installed ( Get-WindowsFeature -Name Hyper-V).
📌 Example: Full Setup Script
# Disable VMQ Get-NetAdapterVmq | Disable-NetAdapterVmq # Create SET Switch with RDMA New-VMSwitch -Name "SET-Switch" ` -NetAdapterName "NIC1", "NIC2" ` -EnableEmbeddedTeaming $True ` -EnableRdma $True # Verify RDMA status on switch Get-VMSwitch | Select Name, EnableRdma, RrdqSupportEnabled # Verify RDMA capability on Management OS Get-VMNetworkAdapterRdma -ManagementOS
📚 Additional Resources
- Microsoft Docs – RDMA over Converged Ethernet
- Microsoft Docs – Switch Embedded Teaming (SET)
- Mellanox RDMA Configuration Guide
❓Still Having Issues?
Please provide the output of these commands:
Get-NetAdapterRdma Get-VMSwitch | Select Name, EnableRdma, RrdqSupportEnabled Get-VMNetworkAdapterRdma -ManagementOSAlso, share:
- Your NIC make/model
- Driver version
- Whether this is for SMB Direct, SDN, etc.
I’ll help you dig deeper into logs or vendor-specific settings.
Let me know how it goes!
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.