LoRA(低秩适应,Low-Rank Adaptation)的 alpha 参数解析(54)
如果 聚焦于层本身 ,情况就会像下面这样简单直接
regular_output = base_layer(batch)
out_A = layer_A(batch)
additional_output = layer_B(out_A)
output = regular_output + additional_output
regular_output, additional_output, output
输出