[jira] [Created] (IOTDB-451) [Distributed]Recovery of snapshot pulling
Tian Jiang created IOTDB-451: Summary: [Distributed]Recovery of snapshot pulling Key: IOTDB-451 URL: https://issues.apache.org/jira/browse/IOTDB-451 Project: Apache IoTDB Issue Type: Improvement Reporter: Tian Jiang After the addition/removal of a node, snapshots of slots are pulled from the previous holders to the new holders. In case that the new holders are down and restarted, it would be better to restart the pulling from a breakpoint instead of starting over. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (IOTDB-335) Separate query executions of the same timeseries with different aggregate functions may be optimized
[ https://issues.apache.org/jira/browse/IOTDB-335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao closed IOTDB-335. - Fix Version/s: 0.10.0-SNAPSHOT Resolution: Fixed > Separate query executions of the same timeseries with different aggregate > functions may be optimized > > > Key: IOTDB-335 > URL: https://issues.apache.org/jira/browse/IOTDB-335 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Lei Rui >Priority: Major > Labels: pull-request-available > Fix For: 0.10.0-SNAPSHOT > > Time Spent: 20m > Remaining Estimate: 0h > > For example, > select avg(s1),sum(s1) from root.sg.d0 > select first_value(s1),first_time(s1) from root.sg.d0 > select count(s1), sum(s1) from root.sg.d0 group by ([1,100],2ms) > > The separate query executions of the same timeseries with different aggregate > functions may be a waste. Probably they can be achieved in one execution. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-452) Do all aggregations of one series at one pass in GroupBy
Jialin Qiao created IOTDB-452: - Summary: Do all aggregations of one series at one pass in GroupBy Key: IOTDB-452 URL: https://issues.apache.org/jira/browse/IOTDB-452 Project: Apache IoTDB Issue Type: Improvement Reporter: Jialin Qiao For example select avg(s1), count(s1), first(s1) from root.sg.d1 group by ([100,199],10ms) We could just construct one aggregate reader and get three aggregation results at one pass. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IOTDB-452) Do all aggregations of one series at one pass in GroupBy
[ https://issues.apache.org/jira/browse/IOTDB-452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17029806#comment-17029806 ] Xiangdong Huang commented on IOTDB-452: --- +1 . It is a basic optimization for the query process. Even if there is no group by clause, the aggregations should also only depend on one Reader for one path. e.g., select avg(s1), count(s1), first(s1) from root.sg.d1. > Do all aggregations of one series at one pass in GroupBy > > > Key: IOTDB-452 > URL: https://issues.apache.org/jira/browse/IOTDB-452 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Jialin Qiao >Priority: Minor > > For example > > select avg(s1), count(s1), first(s1) from root.sg.d1 group by ([100,199],10ms) > > We could just construct one aggregate reader and get three aggregation > results at one pass. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IOTDB-335) Separate query executions of the same timeseries with different aggregate functions may be optimized
[ https://issues.apache.org/jira/browse/IOTDB-335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17029807#comment-17029807 ] Xiangdong Huang commented on IOTDB-335: --- Hi, what is the difference with IOTDB-452? > Separate query executions of the same timeseries with different aggregate > functions may be optimized > > > Key: IOTDB-335 > URL: https://issues.apache.org/jira/browse/IOTDB-335 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Lei Rui >Priority: Major > Labels: pull-request-available > Fix For: 0.10.0-SNAPSHOT > > Time Spent: 20m > Remaining Estimate: 0h > > For example, > select avg(s1),sum(s1) from root.sg.d0 > select first_value(s1),first_time(s1) from root.sg.d0 > select count(s1), sum(s1) from root.sg.d0 group by ([1,100],2ms) > > The separate query executions of the same timeseries with different aggregate > functions may be a waste. Probably they can be achieved in one execution. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IOTDB-452) Do all aggregations of one series at one pass in GroupBy
[ https://issues.apache.org/jira/browse/IOTDB-452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17029814#comment-17029814 ] Zesong Sun commented on IOTDB-452: -- Hi [~hxd], aggregations without group by clause is implemented in [this PR|https://github.com/apache/incubator-iotdb/pull/765]. With the development and improvement of query codes refactor, "group by clause" part is optimized today, so this new JIRA issue is opened for group by clause. > Do all aggregations of one series at one pass in GroupBy > > > Key: IOTDB-452 > URL: https://issues.apache.org/jira/browse/IOTDB-452 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Jialin Qiao >Priority: Minor > > For example > > select avg(s1), count(s1), first(s1) from root.sg.d1 group by ([100,199],10ms) > > We could just construct one aggregate reader and get three aggregation > results at one pass. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IOTDB-452) Do all aggregations of one series at one pass in GroupBy
[ https://issues.apache.org/jira/browse/IOTDB-452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17029818#comment-17029818 ] Xiangdong Huang commented on IOTDB-452: --- I see. Make sense. > Do all aggregations of one series at one pass in GroupBy > > > Key: IOTDB-452 > URL: https://issues.apache.org/jira/browse/IOTDB-452 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Jialin Qiao >Priority: Minor > > For example > > select avg(s1), count(s1), first(s1) from root.sg.d1 group by ([100,199],10ms) > > We could just construct one aggregate reader and get three aggregation > results at one pass. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IOTDB-335) Separate query executions of the same timeseries with different aggregate functions may be optimized
[ https://issues.apache.org/jira/browse/IOTDB-335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17029874#comment-17029874 ] Jialin Qiao commented on IOTDB-335: --- This is the aggregation query. 452 is the group by query > Separate query executions of the same timeseries with different aggregate > functions may be optimized > > > Key: IOTDB-335 > URL: https://issues.apache.org/jira/browse/IOTDB-335 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Lei Rui >Priority: Major > Labels: pull-request-available > Fix For: 0.10.0-SNAPSHOT > > Time Spent: 20m > Remaining Estimate: 0h > > For example, > select avg(s1),sum(s1) from root.sg.d0 > select first_value(s1),first_time(s1) from root.sg.d0 > select count(s1), sum(s1) from root.sg.d0 group by ([1,100],2ms) > > The separate query executions of the same timeseries with different aggregate > functions may be a waste. Probably they can be achieved in one execution. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[ANNOUNCE] Welcome two new committers: Jincheng Sun and Haonan Hou
Hi folks, In January, two contributors became IoTDB's new committer: Jincheng Sun and Haonan Hou. They are our 29th and 30th committers. Jincheng Sun made effort to the community building, contains the JIRA management, website info etc.. Jincheng is also one of Apache Flink PMCs, which make it easier to integration IoTDB with Flink. Haonan Hou made effort to the query engine of IoTDB and the TsFile module. Welcome them and look forward to more new committers. Cheers, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院
[jira] [Created] (IOTDB-453) Parallel the alignment of timestamp in raw data query
Jialin Qiao created IOTDB-453: - Summary: Parallel the alignment of timestamp in raw data query Key: IOTDB-453 URL: https://issues.apache.org/jira/browse/IOTDB-453 Project: Apache IoTDB Issue Type: Improvement Reporter: Jialin Qiao In the default raw data query without value filter, we already parallel the reading of each series. However, the alignment of timestamp is still by one thread. It's possible to parallel the alignment to speed up the query. Suppose there are 10 series, we could use two threads, each align 5 series. Then a new thread does the final alignment. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-454) Vectorize the TimeGenerator
Jialin Qiao created IOTDB-454: - Summary: Vectorize the TimeGenerator Key: IOTDB-454 URL: https://issues.apache.org/jira/browse/IOTDB-454 Project: Apache IoTDB Issue Type: Improvement Reporter: Jialin Qiao For all query types with value filter, such as where s1=1 or s2=1, we will construct a TimeGenerator to get the timestamp in the result. In the above case, the generator tree is OrNode(LeafNode(s1=1), LeafNode(s2=1)). In the engine, each leafnode corresponds to a SeriesRawDataPointReader. It's possible to use SeriesRawDataBatchReader in each leafnode and internal nodes to speed up the time generator. It could also be optimized in the TsFile module. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[Discussion] Unify metrics system with rest api.
I will implement the rest APIs for IoTDB by using jetty. I want to move all web resources into an independent package and the rest APIs make web development easier.
Re: [ANNOUNCE] Welcome two new committers: Jincheng Sun and Haonan Hou
Thanks Xiangdong, Really happy to be onboard! It is really a pleasure working with you and the community. Congratulations Haonan! Best, Jincheng Xiangdong Huang 于2020年2月5日周三 上午9:00写道： > Hi folks, > > In January, two contributors became IoTDB's new committer: Jincheng Sun and > Haonan Hou. > > They are our 29th and 30th committers. > > Jincheng Sun made effort to the community building, contains the JIRA > management, website info etc.. > > Jincheng is also one of Apache Flink PMCs, which make it easier to > integration IoTDB with Flink. > > Haonan Hou made effort to the query engine of IoTDB and the TsFile module. > > Welcome them and look forward to more new committers. > > Cheers, > --- > Xiangdong Huang > School of Software, Tsinghua University > > 黄向东 > 清华大学 软件学院 >
Re: [ANNOUNCE] Welcome two new committers: Jincheng Sun and Haonan Hou
Thanks Xiangdong and Jincheng, I’m also very happy to become a committer. Looking forward to contributing more to the community. Congratulations Jincheng too! Best, Haonan
[jira] [Created] (IOTDB-455) Fix the default Fill function for different datatypes
Jialin Qiao created IOTDB-455: - Summary: Fix the default Fill function for different datatypes Key: IOTDB-455 URL: https://issues.apache.org/jira/browse/IOTDB-455 Project: Apache IoTDB Issue Type: Bug Reporter: Jialin Qiao Fix For: 0.8.2, 0.8.1, 0.9.1, 0.9.0, 0.8.0 In our document, the default Fill function of int/long/float/double is Linear Fill and boolean/text is Previous. However, the actual implementation is Previous for all datatypes. The implementation should be fixed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (IOTDB-455) Fix the default Fill function for different datatypes
[ https://issues.apache.org/jira/browse/IOTDB-455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao closed IOTDB-455. - Fix Version/s: (was: 0.8.2) (was: 0.8.1) (was: 0.9.1) (was: 0.9.0) (was: 0.8.0) 0.10.0-SNAPSHOT Resolution: Fixed > Fix the default Fill function for different datatypes > - > > Key: IOTDB-455 > URL: https://issues.apache.org/jira/browse/IOTDB-455 > Project: Apache IoTDB > Issue Type: Bug >Reporter: Jialin Qiao >Assignee: Jialin Qiao >Priority: Major > Fix For: 0.10.0-SNAPSHOT > > > In our document, the default Fill function of int/long/float/double is Linear > Fill and boolean/text is Previous. However, the actual implementation is > Previous for all datatypes. > The implementation should be fixed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IOTDB-455) Fix the default Fill function for different datatypes
[ https://issues.apache.org/jira/browse/IOTDB-455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17030340#comment-17030340 ] Jialin Qiao commented on IOTDB-455: --- Hi, I have fixed this bug in the new_series_reader branch. The default Fill function of int/long/float/double is reset to LinearFill in FillQueryExecutor. > Fix the default Fill function for different datatypes > - > > Key: IOTDB-455 > URL: https://issues.apache.org/jira/browse/IOTDB-455 > Project: Apache IoTDB > Issue Type: Bug >Reporter: Jialin Qiao >Assignee: Jialin Qiao >Priority: Major > Fix For: 0.8.0, 0.9.0, 0.9.1, 0.8.1, 0.8.2 > > > In our document, the default Fill function of int/long/float/double is Linear > Fill and boolean/text is Previous. However, the actual implementation is > Previous for all datatypes. > The implementation should be fixed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-456) Support Fill function in Group by query
Jialin Qiao created IOTDB-456: - Summary: Support Fill function in Group by query Key: IOTDB-456 URL: https://issues.apache.org/jira/browse/IOTDB-456 Project: Apache IoTDB Issue Type: New Feature Reporter: Jialin Qiao Hi, we need to extend the group by query to support fill function. such as: select count(s1) from root.sg.d1 group by ([1,100], 10ms) fill (int32[previous,20ms]) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-457) Remove the constrains of Fill clause
Jialin Qiao created IOTDB-457: - Summary: Remove the constrains of Fill clause Key: IOTDB-457 URL: https://issues.apache.org/jira/browse/IOTDB-457 Project: Apache IoTDB Issue Type: Improvement Reporter: Jialin Qiao Currently, we must set at lease one fill function in the fill clause, which is not friently. For example, select count(s1) from root.sg.d1 where time = 1 Fill(int32[linear]) We'd better remove this constrains and support sql like this: select count(s1) from root.sg.d1 where time = 1 Fill() or select count(s1) from root.sg.d1 where time = 1 Fill -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [Discussion] Unify metrics system with rest api.
+1 for that. Sounds great! Boris Zhu schrieb am Mi., 5. Feb. 2020, 02:30: > I will implement the rest APIs for IoTDB by using jetty. I want to move all > web resources into an independent package and the rest APIs make web > development easier. >
Re: [Discussion] Unify metrics system with rest api.
+1 for rest API. If you have already finished the API documentation, you can share it with the community. Something like Swagger is also recommended. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Lukas Ott 于2020年2月5日周三 下午1:27写道： > +1 for that. Sounds great! > > Boris Zhu schrieb am Mi., 5. Feb. 2020, 02:30: > > > I will implement the rest APIs for IoTDB by using jetty. I want to move > all > > web resources into an independent package and the rest APIs make web > > development easier. > > >
Re: [Discussion] Unify metrics system with rest api.
By the way, I prefer jersey + jetty than spring MVC + tomcat Because jetty is more light and embedded. Do you have any other suggestions? On Wed, Feb 5, 2020 at 9:30 AM Boris Zhu wrote: > I will implement the rest APIs for IoTDB by using jetty. I want to move > all web resources into an independent package and the rest APIs make web > development easier. >
Re: [Discussion] Unify metrics system with rest api.
Hi, I do not know the performance difference between them. But, I prefer the rule of "the performance wins", i.e., we choose the one who can support high concurrency and low latency. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Boris Zhu 于2020年2月5日周三 下午3:43写道： > By the way, I prefer jersey + jetty than spring MVC + tomcat Because jetty > is more light and embedded. Do you have any other suggestions? > > On Wed, Feb 5, 2020 at 9:30 AM Boris Zhu > wrote: > > > I will implement the rest APIs for IoTDB by using jetty. I want to move > > all web resources into an independent package and the rest APIs make web > > development easier. > > >
