Skip to content
The Chat Archive
qwen3-235b-a22b-no-thinking· Alibaba (Qwen) LMArena Human Preference 140K

I'm trying to set up a SET team on Windows Server 2022. I am having issues getting RDMA…

  1. User

    I'm trying to set up a SET team on Windows Server 2022. I am having issues getting RDMA enabled. On the underlying physical NICs it is enabled and functional; but the vSwitch reports the RDMA status as unknown.

  2. Assistant

    It sounds like you're trying to configure Switch Embedded Teaming (SET) on Windows Server 2022 with RDMA (Remote Direct Memory Access) support, but the vSwitch reports RDMA status as "Unknown", even though it's enabled and functional on the underlying physical NICs.

    This is a common issue when setting up RDMA-enabled SET for use with SDN (Software Defined Networking) or high-performance computing scenarios such as SMB Direct.

    Let’s walk through the key configuration steps and troubleshooting points to help resolve the "RDMA Unknown" state on your vSwitch.


    ✅ Prerequisites

    1. Supported Hardware: Make sure your NICs are RDMA-capable (RoCE or iWARP).
    2. Driver Compatibility: Ensure that the physical NIC drivers support VPort RDMA, which is required for RDMA over a Hyper-V virtual switch.
    3. Firmware and Driver Updates: Update both firmware and drivers of the physical NICs to the latest version from the vendor.
    4. Teaming Mode: SET only supports LACP or Static teaming modes. Make sure this is configured correctly.

    🔧 Step-by-Step Configuration

    1. Create the SET Team and vSwitch with RDMA Enabled

    New-VMSwitch -Name "SET-Switch" `
                 -NetAdapterName "NIC1", "NIC2" `
                 -EnableEmbeddedTeaming $True `
                 -EnableRdma $True
    

    Replace "NIC1" and "NIC2" with the actual names of your physical NICs (as seen in Get-NetAdapter).


    2. Verify Physical NIC RDMA Capabilities

    Run:

    Get-NetAdapterRdma
    

    You should see something like:

    Name    Status   RunningStatus   InterfaceDescription
    ----    ------   -------------   --------------------
    NIC1    Enabled  Active          Mellanox ConnectX-5...
    NIC2    Enabled  Active          Mellanox ConnectX-5...
    

    If they’re not active or not supported, that’s a driver/firmware issue.


    3. Check vSwitch RDMA Status

    Get-VMSwitch | Select Name, EnableRdma, RrdqSupportEnabled
    

    Make sure EnableRdma is True.


    4. Check VMQ and VPort RDMA Support

    Sometimes VMQ needs to be disabled for VPort RDMA to work properly.

    Check VMQ status:

    Get-NetAdapterVmq
    

    Disable VMQ if needed:

    Disable-NetAdapterVmq -Name "NIC1", "NIC2"
    

    Reboot may be required after changing VMQ settings.


    5. Confirm VPort RDMA Support

    Use the following command to check if VPort RDMA is supported and enabled:

    Get-VmNetworkAdapterRdma -VMName <YourVMName>
    

    Or for the management OS:

    Get-VMNetworkAdapterRdma -ManagementOS
    

    Ensure it shows:

    Status        : Success
    RdmaCapable   : True
    Active        : True
    

    6. Use Core Isolation (Optional)

    Some RDMA configurations require isolating the RDMA-capable NICs from general traffic using core isolation.

    Set-VMSwitch -Name "SET-Switch" -RrdqSupportEnabled $true
    

    🛠️ Common Issues & Fixes

    IssueFix
    Physical NIC RDMA is enabled, but vSwitch shows "Unknown"This usually indicates that VPort RDMA isn't working. Check NIC driver supports VPort RDMA.
    RDMA not supported in guest VMsMake sure VM has "Virtual Machine Queue (VMQ)" disabled and SR-IOV is off.
    Mellanox / Intel / Broadcom specific quirksRefer to your NIC vendor documentation. Some cards require special registry tweaks or advanced properties.
    Hyper-V Role Not Fully InstalledMake sure all Hyper-V components are installed (Get-WindowsFeature -Name Hyper-V).

    📌 Example: Full Setup Script

    # Disable VMQ
    Get-NetAdapterVmq | Disable-NetAdapterVmq
    
    # Create SET Switch with RDMA
    New-VMSwitch -Name "SET-Switch" `
                 -NetAdapterName "NIC1", "NIC2" `
                 -EnableEmbeddedTeaming $True `
                 -EnableRdma $True
    
    # Verify RDMA status on switch
    Get-VMSwitch | Select Name, EnableRdma, RrdqSupportEnabled
    
    # Verify RDMA capability on Management OS
    Get-VMNetworkAdapterRdma -ManagementOS
    

    📚 Additional Resources


    ❓Still Having Issues?

    Please provide the output of these commands:

    Get-NetAdapterRdma
    Get-VMSwitch | Select Name, EnableRdma, RrdqSupportEnabled
    Get-VMNetworkAdapterRdma -ManagementOS
    

    Also, share:

    • Your NIC make/model
    • Driver version
    • Whether this is for SMB Direct, SDN, etc.

    I’ll help you dig deeper into logs or vendor-specific settings.


    Let me know how it goes!

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.