Hi yong, I think we may start to do it after we release our first version of tsfile project and tsfile-python should be in another separated repo.
On Fri, Jan 5, 2024 at 9:43 PM liuyong <[email protected]> wrote: > Hi, has the tsfile submodule planned to provide the native interface for > prevalent analytical frameworks or programming languages? Especially, the > Python community. The Python community consists of many packages powerful > on time series data analysis, such as the sktime for Machine Learning and > PyTorch for Deep Learning, which highly relies on training on a huge amount > of time series data. > > > As we develop the native AI engine for IoTDB (AINode). We find that the > tsfile, which takes advantage of the fast reading and dense compression of > tsfile organization, may further benefit the incorporation of the DB and AI > System with authority management and free from numerous and costly data > transfers compared with loading from the local file system. And there are > still some open problems as loading the data for training deep models in > iterations and inference of the data as the stream, which pose novel > challenges to the operator supported by the native interface. > > > > Perhaps a good starting point would be to implement a PyTorch-like > dataloader, where the data source could support the tsfile. > > > >
