tqchen commented on PR #15700: URL: https://github.com/apache/tvm/pull/15700#issuecomment-1711774724
Definitely agree that the parameters like `lora_scaling`, `temperature` needs to be passed in. These are usually parameters that needs to be decoupled from the weights themselves. In most of the cases, we would like to keep the same set of weights, change `temperature`/`lora_scaling` without recompute the weight parameter set themselves. The way to realize such decoupling is to enable mechanisms to pass in small set of parameters from controller side and not rely on weight parameter serialization for handle these cases. The disco runtime brought support for such cases -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
