Re: [PR] [Unity][BYOC] Add support for sliding window in attention op [tvm]

2023-10-19 Thread via GitHub
vinx13 merged PR #15951: URL: https://github.com/apache/tvm/pull/15951 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.or

[PR] [Unity][BYOC] Add support for sliding window in attention op [tvm]

2023-10-18 Thread via GitHub
masahi opened a new pull request, #15951: URL: https://github.com/apache/tvm/pull/15951 This is necessary for Mistral support. A new attribute `window_size` has been added to `nn.attention(...)` op, and such sliding-window attention runs efficiently via flash attention. I updated our flash