DPO算法微调实战

代码地址见文末

1. 环境配置

conda create --name florence python=3.11
conda activate florence
nvcc --version
pip3 install -i https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/ torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip3 install -i https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/ transformers
pip3 install -i https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/ peft tqdm Levenshtein
pip3 install -i https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/ pandas numpy
pip3 install -i https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/ tqdm einops timm
pip3 install -i https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/ Pillow supervision
pip3 install -i https://mirrors.tuna.tsinghua.edu.cn/pypi/web