MasterJH5574 commented on code in PR #17837:
URL: https://github.com/apache/tvm/pull/17837#discussion_r2042818754
##########
python/tvm/relax/frontend/nn/spec.py:
##########
@@ -64,6 +65,14 @@ def __repr__(self) -> str:
return "object"
+class PagedKVCache(Object): # pylint: disable=too-few-public-methods
+ """A specialized spec that extends Object for a paged key-value cache."""
+
+ def __init__(self) -> None:
+ # Pass the type of this class to the Object constructor.
+ super().__init__(object_type=_PagedKVCache)
+
+
Review Comment:
Hi @annanyapr, we will need `PagedKVCache` to be its own standalone class
here, rather than a subclass of Object. This will also need us to support the
kv cache StructInfo first. I think a better order is to work on the StructInfo
and runtime checking (with unit tests) first, and then come back to work on the
nn frontend spec.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
