CharlieFRuan opened a new pull request, #16910: URL: https://github.com/apache/tvm/pull/16910
There are two changes in this PR. #### Change 1: Support `string[]` in `setPackedFunc()` Prior to this PR, we cannot pass in `string[]` from typescript to a TVM PackedFunc and need to convert it to `TVMArray<TVMString>` (for instance in `getParamsFromCacheByName()`). This may not be the most convenient if the PackedFunc's caller is not internal to tvmjs. Thus, this PR moves the conversion to `setPackedFunc()` instead. #### Change 2: Support exceptionally long TVM arrays The second change is dealing with exceptionally long TVM arrays. In cases like passing in a token table, we need to pass in a long `string[]` (in Llama-3's case, of size 128000), leading to JS error `RangeError: Maximum call stack size exceeded` since we treat each string element as an argument, shown in `this.ctx.arrayMake(...inputs)`. This PR sets an empirical call stack limit of 30000 and chunks the array elements in `makeTVMArray()`, converting each chunk to its own TVMArray. Then we concatenate them with the newly implemented `runtime.ArrayConcat` that concatenates N TVMArrays. Tested end-to-end in WebLLM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
