Re: Outhreachy Internship: Should we enrol as IOTDB? Anyone interested in mentorship
Using an idl for tsfile similar to protobuf and we are fine. On Fri, 9 Oct 2020, 08:19 Xiangdong Huang, wrote: > > BTW, as you mentioned, if each device supports a different transport > protocol, does IoTDB need to support this protocol accordingly?(like > AMQP、CoAP、DDS、MQTT(already supported) and so on). > > IMO, letting IoTDB support more protocols does not bring disadvantages. If > someone implements any of them, it will be a good new feature of IoTDB. > > But another ambition from me is: if applications allow sending data to the > cloud (IoTDB) in batch, then sending TsFile directly, rather than data > points by the above protocols (Though there is no a good real word practice > up to now). > > Best, > --- > Xiangdong Huang > School of Software, Tsinghua University > > 黄向东 > 清华大学 软件学院 > > > Houliang Qi 于2020年10月9日周五 下午1:59写道： > > > > > > > Hi Xiangdong, > > > > > > Yeah, for those devices can not run JVM and do not support TCP are not > > appropriate for The restful API approach. > > BTW, as you mentioned, if each device supports a different transport > > protocol, does IoTDB need to support this protocol accordingly?(like > > AMQP、CoAP、DDS、MQTT(already supported) and so on). > > > > > > Thanks, > > --- > > Houliang Qi > > On 10/9/2020 11:59，Xiangdong Huang wrote： > > Hi Houliang, > > > > I am trying to consider the following scenario: > > > > > > We have a device (e.g., a robot arm), an edge gateway, and a cloud > center. > > > > Let's just consider the first two roles: the device and the edge gate. > > > > The device collects data from its PLC, and may store the data locally > for a > > short time, and then sends to the edge gateway. > > The edgeway collects data from the device and may store the data locally > > for a short time and then sends to the cloud. > > > > On both the device and the edgeway, they may have no ability to connect > to > > another service by TCP, and have to save data locally because of some > > reasons. > > Yes they can call rest API in their programs, but if they can not run > Java > > locally, then we can do nothing. > > But TsFile is a data exchange way as it stores binary data. Writing with > > C++, and sending the file to another role (who can run Java) and then we > > can run IOTDB there. > > > > Best, > > --- > > Xiangdong Huang > > School of Software, Tsinghua University > > > > 黄向东 > > 清华大学 软件学院 > > > > > > Houliang Qi 于2020年10月9日周五 上午11:20写道： > > > > Hi, Xiangdong > > > > > > |I have a crazy idea (but not sure whether it works), make TsFile > > cross-language. > > > > > > I have an immature idea. Why not turn TsFile into a restful micro-service > > that can provide read-write and other operations. Since Java is cross > > platform, as long as there is a Java virtual machine, you don't need to > > care about the language implementation of these embedded devices. > > > > > > Thanks, > > --- > > Houliang Qi > > On 09/24/2020 15:05，Xiangdong Huang wrote： > > Hi, > > > > I have a crazy idea (but not sure whether it works), make TsFile > > cross-language. > > Now we can use Java to generate and query data from TsFile (Maybe we can > > also use C++ to do that by the effort of Giorgio). > > > > But how about Go, how about python? It will make the Tsfile really > > available on embedded devices if we can support those languages. > > > > And, how we maintain them if TsFile's structure is changed? (It is > obvious > > that the file format will evolve time by time). > > > > So, a hard-work task is, implementing TsFile with different languages. > > > > A smart task is using some DSL to describe it and generating codes into > > different languages (at least generating part of codes). > > > > Is that a good task for the internship? > > > > Best, > > --- > > Xiangdong Huang > > School of Software, Tsinghua University > > > > 黄向东 > > 清华大学 软件学院 > > > > > > Xiangdong Huang 于2020年9月24日周四 下午2:19写道： > > > > Hi Giorgio, > > > > Thanks for raising this up. I notice the DDL is entened to 29th. > > > > It is a good chance. > > > > Any ideas? > > > > Best, > > --- > > Xiangdong Huang > > School of Software, Tsinghua University > > > > 黄向东 > > 清华大学 软件学院 > > > > > > Giorgio Zoppi 于2020年9月24日周四 上午2:18写道： > > > > Hi all, > > > > Apache Software Foundation has participated in the next round of > Outreachy > > Internships (https://www.outreachy.org/). > > Can we send them some projects ? > > BR, > > Giorgio > > > > > > > > >
Re: Outhreachy Internship: Should we enrol as IOTDB? Anyone interested in mentorship
> BTW, as you mentioned, if each device supports a different transport protocol, does IoTDB need to support this protocol accordingly?(like AMQP、CoAP、DDS、MQTT(already supported) and so on). IMO, letting IoTDB support more protocols does not bring disadvantages. If someone implements any of them, it will be a good new feature of IoTDB. But another ambition from me is: if applications allow sending data to the cloud (IoTDB) in batch, then sending TsFile directly, rather than data points by the above protocols (Though there is no a good real word practice up to now). Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Houliang Qi 于2020年10月9日周五 下午1:59写道： > > > Hi Xiangdong, > > > Yeah, for those devices can not run JVM and do not support TCP are not > appropriate for The restful API approach. > BTW, as you mentioned, if each device supports a different transport > protocol, does IoTDB need to support this protocol accordingly?(like > AMQP、CoAP、DDS、MQTT(already supported) and so on). > > > Thanks, > --- > Houliang Qi > On 10/9/2020 11:59，Xiangdong Huang wrote： > Hi Houliang, > > I am trying to consider the following scenario: > > > We have a device (e.g., a robot arm), an edge gateway, and a cloud center. > > Let's just consider the first two roles: the device and the edge gate. > > The device collects data from its PLC, and may store the data locally for a > short time, and then sends to the edge gateway. > The edgeway collects data from the device and may store the data locally > for a short time and then sends to the cloud. > > On both the device and the edgeway, they may have no ability to connect to > another service by TCP, and have to save data locally because of some > reasons. > Yes they can call rest API in their programs, but if they can not run Java > locally, then we can do nothing. > But TsFile is a data exchange way as it stores binary data. Writing with > C++, and sending the file to another role (who can run Java) and then we > can run IOTDB there. > > Best, > --- > Xiangdong Huang > School of Software, Tsinghua University > > 黄向东 > 清华大学 软件学院 > > > Houliang Qi 于2020年10月9日周五 上午11:20写道： > > Hi, Xiangdong > > > |I have a crazy idea (but not sure whether it works), make TsFile > cross-language. > > > I have an immature idea. Why not turn TsFile into a restful micro-service > that can provide read-write and other operations. Since Java is cross > platform, as long as there is a Java virtual machine, you don't need to > care about the language implementation of these embedded devices. > > > Thanks, > --- > Houliang Qi > On 09/24/2020 15:05，Xiangdong Huang wrote： > Hi, > > I have a crazy idea (but not sure whether it works), make TsFile > cross-language. > Now we can use Java to generate and query data from TsFile (Maybe we can > also use C++ to do that by the effort of Giorgio). > > But how about Go, how about python? It will make the Tsfile really > available on embedded devices if we can support those languages. > > And, how we maintain them if TsFile's structure is changed? (It is obvious > that the file format will evolve time by time). > > So, a hard-work task is, implementing TsFile with different languages. > > A smart task is using some DSL to describe it and generating codes into > different languages (at least generating part of codes). > > Is that a good task for the internship? > > Best, > --- > Xiangdong Huang > School of Software, Tsinghua University > > 黄向东 > 清华大学 软件学院 > > > Xiangdong Huang 于2020年9月24日周四 下午2:19写道： > > Hi Giorgio, > > Thanks for raising this up. I notice the DDL is entened to 29th. > > It is a good chance. > > Any ideas? > > Best, > --- > Xiangdong Huang > School of Software, Tsinghua University > > 黄向东 > 清华大学 软件学院 > > > Giorgio Zoppi 于2020年9月24日周四 上午2:18写道： > > Hi all, > > Apache Software Foundation has participated in the next round of Outreachy > Internships (https://www.outreachy.org/). > Can we send them some projects ? > BR, > Giorgio > > > >
Re: Outhreachy Internship: Should we enrol as IOTDB? Anyone interested in mentorship
Hi Xiangdong, Yeah, for those devices can not run JVM and do not support TCP are not appropriate for The restful API approach. BTW, as you mentioned, if each device supports a different transport protocol, does IoTDB need to support this protocol accordingly?(like AMQP、CoAP、DDS、MQTT(already supported) and so on). Thanks, --- Houliang Qi On 10/9/2020 11:59，Xiangdong Huang wrote： Hi Houliang, I am trying to consider the following scenario: We have a device (e.g., a robot arm), an edge gateway, and a cloud center. Let's just consider the first two roles: the device and the edge gate. The device collects data from its PLC, and may store the data locally for a short time, and then sends to the edge gateway. The edgeway collects data from the device and may store the data locally for a short time and then sends to the cloud. On both the device and the edgeway, they may have no ability to connect to another service by TCP, and have to save data locally because of some reasons. Yes they can call rest API in their programs, but if they can not run Java locally, then we can do nothing. But TsFile is a data exchange way as it stores binary data. Writing with C++, and sending the file to another role (who can run Java) and then we can run IOTDB there. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Houliang Qi 于2020年10月9日周五 上午11:20写道： Hi, Xiangdong |I have a crazy idea (but not sure whether it works), make TsFile cross-language. I have an immature idea. Why not turn TsFile into a restful micro-service that can provide read-write and other operations. Since Java is cross platform, as long as there is a Java virtual machine, you don't need to care about the language implementation of these embedded devices. Thanks, --- Houliang Qi On 09/24/2020 15:05，Xiangdong Huang wrote： Hi, I have a crazy idea (but not sure whether it works), make TsFile cross-language. Now we can use Java to generate and query data from TsFile (Maybe we can also use C++ to do that by the effort of Giorgio). But how about Go, how about python? It will make the Tsfile really available on embedded devices if we can support those languages. And, how we maintain them if TsFile's structure is changed? (It is obvious that the file format will evolve time by time). So, a hard-work task is, implementing TsFile with different languages. A smart task is using some DSL to describe it and generating codes into different languages (at least generating part of codes). Is that a good task for the internship? Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Xiangdong Huang 于2020年9月24日周四 下午2:19写道： Hi Giorgio, Thanks for raising this up. I notice the DDL is entened to 29th. It is a good chance. Any ideas? Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Giorgio Zoppi 于2020年9月24日周四 上午2:18写道： Hi all, Apache Software Foundation has participated in the next round of Outreachy Internships (https://www.outreachy.org/). Can we send them some projects ? BR, Giorgio
Re: Outhreachy Internship: Should we enrol as IOTDB? Anyone interested in mentorship
Hi Houliang, I am trying to consider the following scenario: We have a device (e.g., a robot arm), an edge gateway, and a cloud center. Let's just consider the first two roles: the device and the edge gate. The device collects data from its PLC, and may store the data locally for a short time, and then sends to the edge gateway. The edgeway collects data from the device and may store the data locally for a short time and then sends to the cloud. On both the device and the edgeway, they may have no ability to connect to another service by TCP, and have to save data locally because of some reasons. Yes they can call rest API in their programs, but if they can not run Java locally, then we can do nothing. But TsFile is a data exchange way as it stores binary data. Writing with C++, and sending the file to another role (who can run Java) and then we can run IOTDB there. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Houliang Qi 于2020年10月9日周五 上午11:20写道： > Hi, Xiangdong > > > |I have a crazy idea (but not sure whether it works), make TsFile > cross-language. > > > I have an immature idea. Why not turn TsFile into a restful micro-service > that can provide read-write and other operations. Since Java is cross > platform, as long as there is a Java virtual machine, you don't need to > care about the language implementation of these embedded devices. > > > Thanks, > --- > Houliang Qi > On 09/24/2020 15:05，Xiangdong Huang wrote： > Hi, > > I have a crazy idea (but not sure whether it works), make TsFile > cross-language. > Now we can use Java to generate and query data from TsFile (Maybe we can > also use C++ to do that by the effort of Giorgio). > > But how about Go, how about python? It will make the Tsfile really > available on embedded devices if we can support those languages. > > And, how we maintain them if TsFile's structure is changed? (It is obvious > that the file format will evolve time by time). > > So, a hard-work task is, implementing TsFile with different languages. > > A smart task is using some DSL to describe it and generating codes into > different languages (at least generating part of codes). > > Is that a good task for the internship? > > Best, > --- > Xiangdong Huang > School of Software, Tsinghua University > > 黄向东 > 清华大学 软件学院 > > > Xiangdong Huang 于2020年9月24日周四 下午2:19写道： > > Hi Giorgio, > > Thanks for raising this up. I notice the DDL is entened to 29th. > > It is a good chance. > > Any ideas? > > Best, > --- > Xiangdong Huang > School of Software, Tsinghua University > > 黄向东 > 清华大学 软件学院 > > > Giorgio Zoppi 于2020年9月24日周四 上午2:18写道： > > Hi all, > > Apache Software Foundation has participated in the next round of Outreachy > Internships (https://www.outreachy.org/). > Can we send them some projects ? > BR, > Giorgio > > >
Re: Outhreachy Internship: Should we enrol as IOTDB? Anyone interested in mentorship
Hi, Xiangdong |I have a crazy idea (but not sure whether it works), make TsFile cross-language. I have an immature idea. Why not turn TsFile into a restful micro-service that can provide read-write and other operations. Since Java is cross platform, as long as there is a Java virtual machine, you don't need to care about the language implementation of these embedded devices. Thanks, --- Houliang Qi On 09/24/2020 15:05，Xiangdong Huang wrote： Hi, I have a crazy idea (but not sure whether it works), make TsFile cross-language. Now we can use Java to generate and query data from TsFile (Maybe we can also use C++ to do that by the effort of Giorgio). But how about Go, how about python? It will make the Tsfile really available on embedded devices if we can support those languages. And, how we maintain them if TsFile's structure is changed? (It is obvious that the file format will evolve time by time). So, a hard-work task is, implementing TsFile with different languages. A smart task is using some DSL to describe it and generating codes into different languages (at least generating part of codes). Is that a good task for the internship? Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Xiangdong Huang 于2020年9月24日周四 下午2:19写道： Hi Giorgio, Thanks for raising this up. I notice the DDL is entened to 29th. It is a good chance. Any ideas? Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Giorgio Zoppi 于2020年9月24日周四 上午2:18写道： Hi all, Apache Software Foundation has participated in the next round of Outreachy Internships (https://www.outreachy.org/). Can we send them some projects ? BR, Giorgio
Re: Outhreachy Internship: Should we enrol as IOTDB? Anyone interested in mentorship
Hi Giorgio, I will do that today. By the way, I'd like to add a new task in the proposal: - add IT to guarantee the difference between TsFiles generated by java and by C++ can be found in CI. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Giorgio Zoppi 于2020年9月27日周日 下午6:11写道： > Ok, > i will handle if you want to provide the complete C++ Ts file support. > > Idea: > - Complete support TsFile in C++ and test in an embedded platform. > Could you loook to pointer how to register the project since you are a PMC. > > Best Regards, > Giorgio >
Re: Outhreachy Internship: Should we enrol as IOTDB? Anyone interested in mentorship
Ok, i will handle if you want to provide the complete C++ Ts file support. Idea: - Complete support TsFile in C++ and test in an embedded platform. Could you loook to pointer how to register the project since you are a PMC. Best Regards, Giorgio
Re: Outhreachy Internship: Should we enrol as IOTDB? Anyone interested in mentorship
Hi, So any idea about this? I may have no time these two days for handling this, but I will try my best. And also welcome anyone to propose tasks and register as mentor. (Better to report the idea here.) Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Xiangdong Huang 于2020年9月24日周四 下午3:05写道： > Hi, > > I have a crazy idea (but not sure whether it works), make TsFile > cross-language. > Now we can use Java to generate and query data from TsFile (Maybe we can > also use C++ to do that by the effort of Giorgio). > > But how about Go, how about python? It will make the Tsfile really > available on embedded devices if we can support those languages. > > And, how we maintain them if TsFile's structure is changed? (It is obvious > that the file format will evolve time by time). > > So, a hard-work task is, implementing TsFile with different languages. > > A smart task is using some DSL to describe it and generating codes into > different languages (at least generating part of codes). > > Is that a good task for the internship? > > Best, > --- > Xiangdong Huang > School of Software, Tsinghua University > > 黄向东 > 清华大学 软件学院 > > > Xiangdong Huang 于2020年9月24日周四 下午2:19写道： > >> Hi Giorgio, >> >> Thanks for raising this up. I notice the DDL is entened to 29th. >> >> It is a good chance. >> >> Any ideas? >> >> Best, >> --- >> Xiangdong Huang >> School of Software, Tsinghua University >> >> 黄向东 >> 清华大学 软件学院 >> >> >> Giorgio Zoppi 于2020年9月24日周四 上午2:18写道： >> >>> Hi all, >>> >>> Apache Software Foundation has participated in the next round of >>> Outreachy >>> Internships (https://www.outreachy.org/). >>> Can we send them some projects ? >>> BR, >>> Giorgio >>> >>
Re: Outhreachy Internship: Should we enrol as IOTDB? Anyone interested in mentorship
Hi, I have a crazy idea (but not sure whether it works), make TsFile cross-language. Now we can use Java to generate and query data from TsFile (Maybe we can also use C++ to do that by the effort of Giorgio). But how about Go, how about python? It will make the Tsfile really available on embedded devices if we can support those languages. And, how we maintain them if TsFile's structure is changed? (It is obvious that the file format will evolve time by time). So, a hard-work task is, implementing TsFile with different languages. A smart task is using some DSL to describe it and generating codes into different languages (at least generating part of codes). Is that a good task for the internship? Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Xiangdong Huang 于2020年9月24日周四 下午2:19写道： > Hi Giorgio, > > Thanks for raising this up. I notice the DDL is entened to 29th. > > It is a good chance. > > Any ideas? > > Best, > --- > Xiangdong Huang > School of Software, Tsinghua University > > 黄向东 > 清华大学 软件学院 > > > Giorgio Zoppi 于2020年9月24日周四 上午2:18写道： > >> Hi all, >> >> Apache Software Foundation has participated in the next round of Outreachy >> Internships (https://www.outreachy.org/). >> Can we send them some projects ? >> BR, >> Giorgio >> >
Re: Outhreachy Internship: Should we enrol as IOTDB? Anyone interested in mentorship
Hi Giorgio, Thanks for raising this up. I notice the DDL is entened to 29th. It is a good chance. Any ideas? Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Giorgio Zoppi 于2020年9月24日周四 上午2:18写道： > Hi all, > > Apache Software Foundation has participated in the next round of Outreachy > Internships (https://www.outreachy.org/). > Can we send them some projects ? > BR, > Giorgio >
Outhreachy Internship: Should we enrol as IOTDB? Anyone interested in mentorship
Hi all, Apache Software Foundation has participated in the next round of Outreachy Internships (https://www.outreachy.org/). Can we send them some projects ? BR, Giorgio
