Re: Outhreachy Internship: Should we enrol as IOTDB? Anyone interested in mentorship

[Xiangdong Huang]

Hi,

So any idea about this?

I may have no time these two days for handling this,
but I will try my best.

And also welcome anyone to propose tasks and register as mentor.

(Better to report the idea here.)

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Xiangdong Huang  于2020年9月24日周四 下午3:05写道：

> Hi,
>
> I have a crazy idea (but not sure whether it works), make TsFile
> cross-language.
> Now we can use Java to generate and query data from TsFile (Maybe we can
> also use C++ to do that by the effort of Giorgio).
>
> But how about Go, how about python?  It will make the Tsfile really
> available on embedded devices if we can support those languages.
>
> And, how we maintain them if TsFile's structure is changed? (It is obvious
> that the file format will evolve time by time).
>
> So, a hard-work task is, implementing TsFile with different languages.
>
> A smart task is using some DSL to describe it and generating codes into
> different languages (at least generating part of codes).
>
> Is that a good task for the internship?
>
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
>
>  黄向东
> 清华大学 软件学院
>
>
> Xiangdong Huang  于2020年9月24日周四 下午2:19写道：
>
>> Hi Giorgio,
>>
>> Thanks for raising this up. I notice the DDL is entened to 29th.
>>
>> It is a good chance.
>>
>> Any ideas?
>>
>> Best,
>> ---
>> Xiangdong Huang
>> School of Software, Tsinghua University
>>
>>  黄向东
>> 清华大学 软件学院
>>
>>
>> Giorgio Zoppi  于2020年9月24日周四 上午2:18写道：
>>
>>> Hi all,
>>>
>>> Apache Software Foundation has participated in the next round of
>>> Outreachy
>>> Internships (https://www.outreachy.org/).
>>> Can we send them some projects ?
>>> BR,
>>> Giorgio
>>>
>>

Re: Outhreachy Internship: Should we enrol as IOTDB? Anyone interested in mentorship

[Giorgio Zoppi]

Ok,
i will handle if you want to provide the complete C++ Ts file support.

Idea:
- Complete support TsFile in C++ and test in an embedded platform.
Could you loook to pointer how to register the project since you are a PMC.

Best Regards,
Giorgio

Re: Outhreachy Internship: Should we enrol as IOTDB? Anyone interested in mentorship

[Xiangdong Huang]

Hi Giorgio,

I will do that today.

By the way, I'd like to add a new task in the proposal:
- add IT to guarantee the difference between TsFiles generated by java and
by C++ can be found in CI.

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Giorgio Zoppi  于2020年9月27日周日 下午6:11写道：

> Ok,
> i will handle if you want to provide the complete C++ Ts file support.
>
> Idea:
> - Complete support TsFile in C++ and test in an embedded platform.
> Could you loook to pointer how to register the project since you are a PMC.
>
> Best Regards,
> Giorgio
>

Re: Support SDT compression

[Jialin Qiao]

Hi Haimei,

Good work! This doc is comprehensive :)

As for the implementation in IoTDB, here are some points:

(1) First, SDT could act as a new Encoding method in IoTDB. It works inside 
each Page (PageWriter and PageReader).
(2) The interface of Encoder could be changed to encode(long time, XX value). 
The interface of Decoder could be change to getTime(), getXXValue(). Which is, 
the encoder and decoder is not only responsible for one data column but a 
series. This involves some reconstruction of the Encoder and Decoder, the data 
buffers should be stored inside each encoder.
(3) For the SDT without timestamps, we need to record the count of each segment.
(4) We could offer two encodings, SDT with timestamps and SDT without 
timestamps.

Thanks,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "runhus...@foxmail.com" 
> 发送时间: 2020-09-25 11:56:16 (星期五)
> 收件人: dev 
> 抄送: 
> 主题: Re: Support SDT compression
> 
> Great work！
> 
> 
> 
> Thanks.
> 
> Chao Wang
> BONC Ltd
> 
>  
> From: Eileen Guo
> Date: 2020-09-25 11:47
> To: dev
> Subject: Support SDT compression
> Hi all,
>  
> I've completed a design draft for supporting swinging door compression.
>  
> Jira: jira SDT link
> 
> design doc: SDT design doc link
> 
>  
> The doc explains SDT algorithm, compression and decompression process,
> performance tests and SDT + IoTDB implementation and usage.
>  
> There is still some question about where to use this algorithm. If you have
> any idea, welcome to comment.
>  
> Thank you!
> Haimei Guo