Skip to content
TheChatArchiveThe Chat Archive
gpt-4.1-2025-04-14· OpenAIHugging Face LMArena Human Preference 140K

Besides CUDA, what other libraries or frameworks can be used to write code that runs on…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Claudeclaude-3-5-haiku-20241022’s answer.
  1. User

    Besides CUDA, what other libraries or frameworks can be used to write code that runs on Nvidia GPUs? Note, that I am not talking about libraries that compiles to CUDA like triton, I am talking about straight GPUs execution without CUDA on Nvidia GPUs. Please only limit the scope for libraries that can be used for writing code for deep learning applications not for graphics.

  2. Assistant

    Great question! If you want to write code that runs on Nvidia GPUs for deep learning applications, but not using CUDA or libraries that compile to CUDA (like Triton), your options are quite limited, because CUDA is the primary low-level API for Nvidia GPUs. However, there are a few alternatives, though each comes with caveats:


    1. OpenCL

    • Description: OpenCL (Open Computing Language) is an open standard for cross-platform, parallel programming of diverse processors, including GPUs.
    • Nvidia Support: Nvidia provides OpenCL drivers for their GPUs, so you can write OpenCL code that runs directly on Nvidia hardware.
    • Deep Learning Use: There are some deep learning frameworks and libraries that support OpenCL (e.g., PlaidML), but most mainstream frameworks (like TensorFlow and PyTorch) do not have robust OpenCL backends.
    • Note: OpenCL is generally less performant and less feature-rich than CUDA on Nvidia GPUs.

    2. Vulkan Compute

    • Description: Vulkan is a low-level, cross-platform graphics and compute API. It includes compute shaders that can be used for general-purpose GPU computation.
    • Nvidia Support: Nvidia provides Vulkan drivers for their GPUs.
    • Deep Learning Use: There are very few deep learning libraries built on Vulkan, but you can write your own compute shaders for deep learning operations.
    • Note: This is a very low-level approach and requires significant effort to implement deep learning primitives.

    3. DirectX 12 / DirectML

    • Description: DirectX 12 is a Microsoft API for graphics and compute. DirectML is a high-performance, hardware-accelerated DirectX 12 library for machine learning.
    • Nvidia Support: Nvidia supports DirectX 12 and DirectML on Windows.
    • Deep Learning Use: DirectML can be used for deep learning inference (and some training), and is supported in ONNX Runtime and some other frameworks.
    • Note: This is Windows-only and not as widely used as CUDA.

    4. SYCL (via DPC++)

    • Description: SYCL is a C++ abstraction layer for heterogeneous computing, part of the Khronos Group (like OpenCL).
    • Nvidia Support: There is experimental support for Nvidia GPUs via OpenCL or CUDA backends (e.g., hipSYCL), but this often still relies on CUDA or OpenCL under the hood.
    • Deep Learning Use: Some experimental projects, but not mainstream.

    5. OpenACC

    • Description: OpenACC is a directive-based programming model for parallel computing, often used in scientific computing.
    • Nvidia Support: Nvidia supports OpenACC via their compilers.
    • Deep Learning Use: Rarely used for deep learning, but possible for custom code.

    Summary Table

    Library/APINvidia SupportDeep Learning UseNotes
    OpenCLYesLimitedCross-vendor, less performant than CUDA
    Vulkan ComputeYesRareLow-level, few deep learning libs
    DirectMLYes (Windows)SomeInference-focused, Windows-only
    SYCLExperimentalRareOften uses CUDA/OpenCL under the hood
    OpenACCYesRareDirective-based, not common for DL

    Key Takeaway

    • OpenCL is the main alternative to CUDA for writing code that runs directly on Nvidia GPUs for deep learning, but it is not as well supported or performant.
    • Vulkan Compute and DirectML are other options, but are rarely used for deep learning and require significant effort or are limited in scope.
    • Most deep learning on Nvidia hardware is done via CUDA because of its performance and ecosystem.

    If you want to avoid CUDA entirely, OpenCL is your best bet, but be prepared for limited support and performance compared to CUDA.

  3. User

    what about the MAX platform by Modular claims that you can write code with Mojo language that directly execute on GPUs without CUDA!!

  4. Assistant

    Great follow-up! The MAX platform by Modular and the Mojo language are indeed generating a lot of buzz in the AI and deep learning community. Let’s clarify what’s currently possible and what’s claimed regarding direct GPU execution without CUDA.


    Mojo, MAX, and GPU Execution: The Claims

    What Modular Claims

    • Mojo is a new programming language designed for high-performance AI workloads, aiming to combine Python’s usability with C++/CUDA-level performance.
    • MAX is Modular’s AI inference engine, which claims to run models on a variety of hardware backends, including Nvidia GPUs, without requiring CUDA.

    How Does It Work?

    • Modular claims that their stack can target Nvidia GPUs without using CUDA as a dependency.
    • Instead, they say they use their own custom GPU runtime and compiler infrastructure, which can generate code for Nvidia GPUs directly, bypassing the CUDA toolkit.

    Is This True?

    Current State (as of mid-2024)

    • MAX does not require you to install CUDA or the Nvidia CUDA toolkit. You only need the Nvidia driver.
    • Mojo code, when compiled for GPU, is translated by Modular’s toolchain into GPU machine code (PTX or SASS) that the Nvidia driver can execute.
    • This means: You can write Mojo code, and Modular’s stack will compile it to run on Nvidia GPUs, without using the CUDA toolkit or libraries.

    How Is This Possible?

    • Nvidia GPUs execute code via the driver, which accepts PTX (Parallel Thread Execution) or SASS (native assembly).
    • CUDA is just one way to generate PTX/SASS. If you have your own compiler that can generate valid PTX/SASS, you can run code on Nvidia GPUs without using CUDA libraries.
    • Modular’s stack appears to do just that: it generates PTX directly, bypassing CUDA as a software dependency.

    Caveats and Limitations

    • You still need the Nvidia driver (which is closed-source and proprietary).
    • You do not need the CUDA toolkit (the developer tools and libraries from Nvidia).
    • Ecosystem and maturity: Modular’s stack is new and not as mature as CUDA. Not all deep learning operations or models may be supported yet.
    • Performance and compatibility: Real-world performance and compatibility may vary, and some advanced features may still require CUDA.

    Summary Table

    Platform/LanguageNeeds CUDA Toolkit?Needs Nvidia Driver?Can Run on Nvidia GPU?Notes
    CUDAYesYesYesOfficial Nvidia stack
    OpenCLNoYesYesCross-vendor, less performant
    Mojo (via MAX)NoYesYesNew, generates PTX directly
    Vulkan ComputeNoYesYesLow-level, not common for DL

    Conclusion

    Yes, Modular’s MAX platform and Mojo language do allow you to write code that runs on Nvidia GPUs without using CUDA (the toolkit/libraries). They achieve this by generating PTX code directly, which the Nvidia driver can execute. This is a significant development, but the ecosystem is still young compared to CUDA.

    If you want to experiment with deep learning on Nvidia GPUs without CUDA, Modular’s MAX and Mojo are currently the most promising new options.


    References:

    If you want more technical details or code examples, let me know!

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.