Skip to content
TheChatArchiveThe Chat Archive
Human-written demonstration (OpenAssistant volunteers)· OpenAssistant (human-written)Hugging Face OpenAssistant Conversations (oasst2)

i have a model trained with the "trasformers" library, using python and pytorch. if i…

In OpenAssistant data, “assistant” turns were written by human volunteers as demonstrations. They are not model outputs.
  1. User

    i have a model trained with the "trasformers" library, using python and pytorch.

    if i use a GPU, the model scale well with batches of input data, but if I perform inference using a CPU, the model dosn't scale.

    any clues on how to do better?

  2. Assistant

    GPUs typically excel at number crunching and provide better performance than x86_64 CPU architecture. You can investigate improving your Python code's performance using native machine code (usually C/C++) implementations of Python libraries. Your Python code using underlying native machine code execution will be significantly faster than your same code executing underlying Python source via the Python interpreter.

  3. User

    But C and C++ are not native machine code, C is considered to be a high level language back in the day and a middle level language currently.

  4. Assistant

    You are correct that C and C++ are not considered native machine code. They are both high-level programming languages that allow developers to write code at a higher level of abstraction compared to low-level languages like assembly language. However, they are often referred to as "middle-level" languages due to their ability to access low-level features and directly manipulate memory, which is a characteristic of low-level languages.

    C, which was developed in the early 1970s, is often described as a high-level language for systems programming. It provides low-level access to memory through pointers and allows for efficient hardware manipulation. C++ is an extension of C and adds object-oriented programming features. It maintains compatibility with C, allowing C code to be used in C++ programs.

    The distinction between high-level, middle-level, and low-level languages can vary, and the terminology has evolved over time. Some may consider C and C++ to be high-level languages, while others may classify them as middle-level languages due to their ability to combine low-level programming with high-level abstractions.

    It's worth noting that the categorization of programming languages can be subjective, and the terms used to describe them can vary among different contexts and individuals.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.