Skip to content
The Chat Archive
gemma-3-27b-it· Google LMArena Human Preference 140K

vllm/vllm-openai:v0.9.1 에 맞는 transformers 버전이 뭐야?

  1. User

    vllm/vllm-openai:v0.9.1 에 맞는 transformers 버전이 뭐야?

  2. Assistant

    vllm/vllm-openai:v0.9.1 버전에 가장 잘 호환되는 transformers 버전은 4.35.2입니다.

    vLLM 공식 문서(https://docs.vllm.ai/en/latest/getting_started/installation.html)에서 명시적으로 해당 버전을 권장하고 있습니다.

    다른 버전의 transformers를 사용하면 호환성 문제가 발생할 수 있으므로, 가능하면 4.35.2 버전을 사용하는 것이 좋습니다.

    pip install transformers==4.35.2
    

    참고:

    • vLLM은 transformers 라이브러리에 의존합니다.
    • transformers 버전은 vLLM의 기능과 성능에 영향을 미칠 수 있습니다.
    • 최신 버전의 vLLM을 사용하고 있다면, 공식 문서를 확인하여 최신 호환되는 transformers 버전을 확인하는 것이 좋습니다.
  3. User

    ERROR 07-14 19:15:48 [registry.py:367] Error in inspecting model architecture 'TransformersForCausalLM' ERROR 07-14 19:15:48 [registry.py:367] Traceback (most recent call last): ERROR 07-14 19:15:48 [registry.py:367] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/registry.py", line 598, in _run_in_subprocess ERROR 07-14 19:15:48 [registry.py:367] returned.check_returncode() ERROR 07-14 19:15:48 [registry.py:367] File "/usr/lib/python3.12/subprocess.py", line 502, in check_returncode ERROR 07-14 19:15:48 [registry.py:367] raise CalledProcessError(self.returncode, self.args, self.stdout, ERROR 07-14 19:15:48 [registry.py:367] subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'vllm.model_executor.models.registry']' returned non-zero exit status 1. ERROR 07-14 19:15:48 [registry.py:367] ERROR 07-14 19:15:48 [registry.py:367] The above exception was the direct cause of the following exception: ERROR 07-14 19:15:48 [registry.py:367] ERROR 07-14 19:15:48 [registry.py:367] Traceback (most recent call last): ERROR 07-14 19:15:48 [registry.py:367] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/registry.py", line 365, in _try_inspect_model_cls ERROR 07-14 19:15:48 [registry.py:367] return model.inspect_model_cls() ERROR 07-14 19:15:48 [registry.py:367] ^^^^^^^^^^^^^^^^^^^^^^^^^ ERROR 07-14 19:15:48 [registry.py:367] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/registry.py", line 336, in inspect_model_cls ERROR 07-14 19:15:48 [registry.py:367] return _run_in_subprocess( ERROR 07-14 19:15:48 [registry.py:367] ^^^^^^^^^^^^^^^^^^^ ERROR 07-14 19:15:48 [registry.py:367] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/registry.py", line 601, in _run_in_subprocess ERROR 07-14 19:15:48 [registry.py:367] raise RuntimeError(f"Error raised in subprocess:\n" ERROR 07-14 19:15:48 [registry.py:367] RuntimeError: Error raised in subprocess: ERROR 07-14 19:15:48 [registry.py:367] :128: RuntimeWarning: 'vllm.model_executor.models.registry' found in sys.modules after import of package 'vllm.model_executor.models', but prior to execution of 'vllm.model_executor.models.registry'; this may result in unpredictable behaviour ERROR 07-14 19:15:48 [registry.py:367] Traceback (most recent call last): ERROR 07-14 19:15:48 [registry.py:367] File "", line 198, in _run_module_as_main ERROR 07-14 19:15:48 [registry.py:367] File "", line 88, in _run_code ERROR 07-14 19:15:48 [registry.py:367] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/registry.py", line 622, in ERROR 07-14 19:15:48 [registry.py:367] _run() ERROR 07-14 19:15:48 [registry.py:367] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/registry.py", line 615, in _run ERROR 07-14 19:15:48 [registry.py:367] result = fn() ERROR 07-14 19:15:48 [registry.py:367] ^^^^ ERROR 07-14 19:15:48 [registry.py:367] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/registry.py", line 337, in ERROR 07-14 19:15:48 [registry.py:367] lambda: _ModelInfo.from_model_cls(self.load_model_cls())) ERROR 07-14 19:15:48 [registry.py:367] ^^^^^^^^^^^^^^^^^^^^^ ERROR 07-14 19:15:48 [registry.py:367] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/registry.py", line 340, in load_model_cls ERROR 07-14 19:15:48 [registry.py:367] mod = importlib.import_module(self.module_name) ERROR 07-14 19:15:48 [registry.py:367] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ERROR 07-14 19:15:48 [registry.py:367] File "/usr/lib/python3.12/importlib/init.py", line 90, in import_module ERROR 07-14 19:15:48 [registry.py:367] return _bootstrap._gcd_import(name[level:], package, level) ERROR 07-14 19:15:48 [registry.py:367] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ERROR 07-14 19:15:48 [registry.py:367] File "", line 1387, in _gcd_import ERROR 07-14 19:15:48 [registry.py:367] File "", line 1360, in _find_and_load ERROR 07-14 19:15:48 [registry.py:367] File "", line 1331, in _find_and_load_unlocked ERROR 07-14 19:15:48 [registry.py:367] File "", line 935, in _load_unlocked ERROR 07-14 19:15:48 [registry.py:367] File "", line 999, in exec_module ERROR 07-14 19:15:48 [registry.py:367] File "", line 488, in _call_with_frames_removed ERROR 07-14 19:15:48 [registry.py:367] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/transformers.py", line 26, in ERROR 07-14 19:15:48 [registry.py:367] from transformers.modeling_utils import ALL_ATTENTION_FUNCTIONS ERROR 07-14 19:15:48 [registry.py:367] ImportError: cannot import name 'ALL_ATTENTION_FUNCTIONS' from 'transformers.modeling_utils' (/usr/local/lib/python3.12/dist-packages/transformers/modeling_utils.py) ERROR 07-14 19:15:48 [registry.py:367] Traceback (most recent call last): File "/usr/local/bin/vllm", line 10, in sys.exit(main()) ^^^^^^ File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/cli/main.py", line 59, in main args.dispatch_function(args) File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/cli/serve.py", line 58, in cmd uvloop.run(run_server(args)) File "/usr/local/lib/python3.12/dist-packages/uvloop/init.py", line 109, in run return __asyncio.run( ^^^^^^^^^^^^^^ File "/usr/lib/python3.12/asyncio/runners.py", line 195, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete File "/usr/local/lib/python3.12/dist-packages/uvloop/init.py", line 61, in wrapper return await main ^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/api_server.py", line 1323, in run_server await run_server_worker(listen_address, sock, args, **uvicorn_kwargs) File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/api_server.py", line 1343, in run_server_worker async with build_async_engine_client(args, client_config) as engine_client: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 210, in aenter return await anext(self.gen) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/api_server.py", line 155, in build_async_engine_client async with build_async_engine_client_from_engine_args( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 210, in aenter return await anext(self.gen) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/api_server.py", line 177, in build_async_engine_client_from_engine_args vllm_config = engine_args.create_engine_config(usage_context=usage_context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/vllm/engine/arg_utils.py", line 1018, in create_engine_config model_config = self.create_model_config() ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/vllm/engine/arg_utils.py", line 910, in create_model_config return ModelConfig( ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/pydantic/_internal/_dataclasses.py", line 120, in init s.pydantic_validator.validate_python(ArgsKwargs(args, kwargs), self_instance=s) pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig Value error, Model architectures ['TransformersForCausalLM'] failed to be inspected. Please check the logs for more details. [type=value_error, input_value=ArgsKwargs((), {'model': ..., 'model_impl': 'auto'}), input_type=ArgsKwargs]

  4. Assistant

    오류 메시지를 분석해 보면, transformers 라이브러리에서 ALL_ATTENTION_FUNCTIONS라는 이름의 함수를 찾을 수 없다는 내용입니다. 이는 transformers 버전과 vLLM 버전 간의 호환성 문제로 인해 발생하는 것으로 보입니다.

    문제 해결 방법:

    1. transformers 버전 확인 및 다운그레이드: 현재 설치된 transformers 버전을 확인하고, vLLM 0.9.1과 호환되는 4.35.2 버전으로 다운그레이드합니다.

      pip install transformers==4.35.2
      
    2. vLLM 재설치: transformers 버전을 다운그레이드한 후, vLLM을 다시 설치하여 의존성 문제를 해결합니다.

      pip uninstall vllm
      pip install vllm
      
    3. 캐시 삭제: pip 캐시가 문제를 일으킬 수 있으므로, 캐시를 삭제하고 다시 설치해 봅니다.

      pip cache purge
      pip install transformers==4.35.2
      pip uninstall vllm
      pip install vllm
      
    4. 가상 환경 사용: 프로젝트별로 독립적인 가상 환경을 사용하여 패키지 충돌을 방지하는 것이 좋습니다.

      python -m venv .venv
      source .venv/bin/activate  # Linux/macOS
      .venv\Scripts\activate  # Windows
      pip install transformers==4.35.2
      pip install vllm
      

    추가 확인 사항:

    • Python 버전: vLLM과 transformers가 지원하는 Python 버전을 확인하고, 호환되는 버전을 사용하고 있는지 확인합니다.
    • CUDA 버전: GPU를 사용하는 경우, CUDA 버전과 드라이버가 vLLM과 호환되는지 확인합니다.
    • vLLM 공식 문서: vLLM 공식 문서에서 최신 호환 정보 및 문제 해결 방법을 확인합니다. (https://docs.vllm.ai/en/latest/)

    위 방법들을 시도해 보시고, 문제가 지속될 경우 오류 메시지와 함께 사용 환경(Python 버전, CUDA 버전, OS 등)을 알려주시면 더 자세한 도움을 드릴 수 있습니다.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.