Update whl Deps

Update whl Deps#

To ensure that Python dependencies are correctly referenced by code and test cases, and properly resolved during packaging, the following three steps need to be completed:

  • Add the required dependency libraries and their version numbers to the requirements description file

  • Generate a hardware-specific locked-version requirements_lock file based on the requirements file

  • Add the corresponding pip package dependencies to the py_library targets that use the relevant libraries in the Bazel BUILD file

1. Add the required libraries to the requirements description file#

All hardware dependencies are based on open_source/deps/requirements_base.txt as the foundational dependency configuration. If you need to add or update basic dependencies that apply to all hardware (such as fundamental libraries like transformers), you should add them in this file. For example:

# ...
transformers==4.46.2 # update transformers to 4.46.2
# ...

If you need to add specialized dependencies for specific hardware (such as torch packages, hardware computation libraries, etc.), you must add these dependencies in the corresponding hardware-specific requirements files.

2. Generate requirements_lock file#

Lock files must not be manually modified and can only be automatically generated through relevant commands from the base dependency description files.

When opening any dependency file starting with requirements_lock, you will typically find the generation command at the top of the file:

#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    bazel run //open_source/deps:requirements_torch_gpu_cuda12.update
#
--extra-index-url https://mirrors.aliyun.com/pypi/simple/

accelerate==0.25.0 \
    --hash=sha256:c7bb817eb974bba0ff3ea1ba0f24d55afb86d50e3d4fe98d6922dc69cf2ccff1 \
    --hash=sha256:ecf55b0ab278a1dac8539dde0d276977aff04683f07ede73eaf02478538576a1
    # via
    #   -r open_source/deps/requirements_base.txt
    #   auto-gptq
    #   autoawq
    #   peft
# ...

Note that in actual execution, bazel should be replaced with bazelisk.

For each lock file affected by dependency configuration changes (modifying requirements_base.txt will affect all lock files, including those in both open-source and closed-source directories), you need to run the update command specified in the lock file to refresh it.

Important note: If lock file conflicts occur during code merging, do not directly merge the conflicting content manually. Instead, re-run the lock file generation command.

3. Add the dependencies to the Bazel BUILD file#

At this point, the dependencies can be installed correctly, but Python files still cannot properly reference these Python libraries. We also need to declare the dependencies in the Bazel BUILD files.

In the rtp_llm/BUILD file, locate the requirement() function and add the new dependency to its internal list to declare the existence of the dependency.

requirement([
    "sentencepiece",
    "transformers",
    "pynvml",
    "tiktoken",
    # ...
    # add your new dependencies here
])

In the py_library targets that use the corresponding dependency libraries, add the declared dependencies, for example:

py_library(
    name = "utils",
    srcs = glob([
        "utils/**/*.py",
    ]),
    data = [":cutlass_config"],
    deps = [
        ":torch",
        ":safetensors",
        ":lru-dict",
        ":cpm_kernels",
        ":prettytable",
        "//rtp_llm/aios/kmonitor:kmonitor_py",
    ] + arch_dep + internal_deps()
)

Add the corresponding pip dependencies in the whl_reqs section of the rtp_llm/BUILD file to ensure that dependencies are included during packaging.

whl_reqs = [
    "filelock==3.13.1",
    "jinja2",
    "sympy",
    "typing-extensions",
    "importlib_metadata",
    "transformers==4.46.2",
    # ...
    # add your new dependencies here
]

Note: For hardware-related packages, you need to branch and select the appropriate packages based on hardware type in the whl_deps function in the open_source/bazel/arch_select.bzl file.

Contents