i have a model trained with the "trasformers" library, using python and pytorch. if i…
- User
i have a model trained with the "trasformers" library, using python and pytorch.
if i use a GPU, the model scale well with batches of input data, but if I perform inference using a CPU, the model dosn't scale.
any clues on how to do better?
- Assistant
GPUs typically excel at number crunching and provide better performance than x86_64 CPU architecture. You can investigate improving your Python code's performance using native machine code (usually C/C++) implementations of Python libraries. Your Python code using underlying native machine code execution will be significantly faster than your same code executing underlying Python source via the Python interpreter.
- User
But C and C++ are not native machine code, C is considered to be a high level language back in the day and a middle level language currently.
- Assistant
You are correct that C and C++ are not considered native machine code. They are both high-level programming languages that allow developers to write code at a higher level of abstraction compared to low-level languages like assembly language. However, they are often referred to as "middle-level" languages due to their ability to access low-level features and directly manipulate memory, which is a characteristic of low-level languages.
C, which was developed in the early 1970s, is often described as a high-level language for systems programming. It provides low-level access to memory through pointers and allows for efficient hardware manipulation. C++ is an extension of C and adds object-oriented programming features. It maintains compatibility with C, allowing C code to be used in C++ programs.
The distinction between high-level, middle-level, and low-level languages can vary, and the terminology has evolved over time. Some may consider C and C++ to be high-level languages, while others may classify them as middle-level languages due to their ability to combine low-level programming with high-level abstractions.
It's worth noting that the categorization of programming languages can be subjective, and the terms used to describe them can vary among different contexts and individuals.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →