j0h0k0i0m opened a new issue, #17715:
URL: https://github.com/apache/tvm/issues/17715
I had previously raised this issue on MLC LLM as well, but it seems that the
root cause lies in `PagedKVCache`. With the recent release of Phi-4-mini-inst,
the introduction of the `partial_rotary_factor` variable has led to a dimension
mismatch issue. While manually adjusting `rope_ext_factors` allows inference to
proceed, it results in garbage values. Therefore, I am reporting this issue
here. Is there any way to resolve this issue?
### Expected behavior
What you were expecting
### Actual behavior
```shell
Traceback (most recent call last):
File "/opt/anaconda3/envs/mlc/lib/python3.11/threading.py", line 1045, in
_bootstrap_inner
self.run()
File "/opt/anaconda3/envs/mlc/lib/python3.11/threading.py", line 982, in
run
self._target(*self._args, **self._kwargs)
File "tvm/_ffi/_cython/./packed_func.pxi", line 339, in
tvm._ffi._cy3.core.PackedFuncBase.__call__
File "tvm/_ffi/_cython/./packed_func.pxi", line 270, in
tvm._ffi._cy3.core.FuncCall
File "tvm/_ffi/_cython/./packed_func.pxi", line 259, in
tvm._ffi._cy3.core.FuncCall3
File "tvm/_ffi/_cython/./base.pxi", line 185, in
tvm._ffi._cy3.core.CHECK_CALL
File
"/opt/anaconda3/envs/mlc/lib/python3.11/site-packages/tvm/_ffi/base.py", line
465, in raise_last_ffi_error
raise py_err
tvm._ffi.base.TVMError: TVMError: Assert fail: T.Cast("int32",
fused_rope_longrope_scaling_ext_factors_handle_shape[0]) == 64, Argument
fused_rope_longrope_scaling.ext_factors_handle.shape[0] has an unsatisfied
constraint: 64 == T.Cast("int32",
fused_rope_longrope_scaling_ext_factors_handle_shape[0])
```
### Environment
Any environment details, such as: Operating System, TVM version, etc
### Steps to reproduce
Preferably a minimal script to cause the issue to occur.
### Triage
Please refer to the list of label tags
[here](https://github.com/apache/tvm/wiki/Issue-Triage-Labels) to find the
relevant tags and add them below in a bullet format (example below).
* needs-triage
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
