srkreddy1238 commented on PR #13868: URL: https://github.com/apache/tvm/pull/13868#issuecomment-1411536112
@echuraev thanks for this feature. I too had a similar problem statement with CLML tuning cache management at runtime which need to be generated once and reused later. In CLML, there existed multiple sub graphs related to each tvm module and for now the tuning cache is indexed by subgraph symbol and are stored under one file by serializing them through ```DMLC::Strm```. This works by maintaining one file per tvm module. This approach adds additional overhead for the user to maintain and specify the tuning cache for each tvm module. I think we have a generalized problem statement here where there is a need of cache management for the tvm runtime. Probably we could come up with an unified approach where - There will be a tvm_runtime cache specified by environment variale or graph_runtime api interface. - Each tvm compiled module will have a unique hash key generated at compile time and is accessible from tvm module interface. - A new file utility interface to load/store the binary blobs generated by any runtime as a key & value pair. The flow would be like 1: Runtime (OpenCL/CLML ..etc.) will form a key (concatenating [ tvm_module hash + runtime + purpose ...etc.] ) 2: Try to fetch from runtime cache. 3: If not exist regenerate and save into cache. 4: Use the stored cache This will simplify and minimize the end user hassle to a level of just specify a cache folder and relax. In the implementation side we have - tvm Module passing a unique key from compilation to runtime probably via grpah_json. - Cache API to serialize and load/store the key, value paired binary blobs. - Runtime specific changes to use this cache interface. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
