Lunderberg commented on PR #15700: URL: https://github.com/apache/tvm/pull/15700#issuecomment-1711718670
@tqchen For the background PR, the slice index is only included in the saved parameters if two conditions are met. 1. The symbolic variable is required for a computation that occurs during runtime. 2. The symbolic variable cannot be inferred at runtime, either from a runtime or from tensor parameters produced by the lifted `*_transform_params` function. I don't think we should require a separate channel for passing this information. This is a more general approach that handles any symbolic variable that may need to be passed across the boundary of a segmented compute graph, not just a `rank` variable. If we have other variables that we want to enclose with the parameters, such as a `max_seq_len`, `lora_scaling` or `temperature`, this approach would allow them to be included with the associated parameter set. I think there's still room for improvement, as it may be useful to expose these parameters in an inspectable manner (e.g. a JSON dictionary), such that a user can identify which parameters were used to generate a transformed parameter set. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
