tqchen commented on PR #16955: URL: https://github.com/apache/tvm/pull/16955#issuecomment-2085436158
Just want to note here. Having view operation can in general cause problems mainly because most ops(including generated and external ones) assumes `elem_offset = 0` for performance reasons(alignment, less kernel argument). Ideally we would like to ensure such assumption to hold. This being said there are some needs in slicing out the arrays. e.g. in the case of LoRA elements. There are a few ways to go with this: - Enable special ops that handle inputs which can come a view, likely only LoRA ones, which can inline view operations into ops. - Add a `R.memory.ensure_compact` operation, which can potentially results in a copy for backends that do not have direct memory ptr access, but can potentially do ptr editing for backends that support them - `R.view` perhaps should be renamed as `R.memory.view`, this is a more advanced operator that contains certain assumptions and likely not something we want to advertise genrally for now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
