Re: [DISCUSS] Champion and PMC Chair of TsFile
+1 for Christofer Dutz to be the Champion. Jialin Qiao to be the pmc chair. Replied Message | From | Chao Wang | | Date | 10/26/2023 10:33 | | To | | | Subject | Re: [DISCUSS] Champion and PMC Chair of TsFile | +1 ， for Christofer Dutz to be the Champion.Jialin Qiao to be the pmc chair. On 2023/10/26 02:20:14 Xiangdong Huang wrote: Hi, I vote for Christofer Dutz to be the Champion. I think he is the best person for the role. We can feel how he is professional in another email thread (discussion about the proposal and resolution) :D As for the PMC Chair, I think Jialin Qiao is a good choice, he really did a lot on the project, knows everything about the project, including the design, the goal, and pay almost all of his time on the project. Best, --- Xiangdong Huang Jialin Qiao 于2023年10月25日周三 15:54写道： Hi, To enter the Apache, we also need to define the project champion and PMC chair. For Champion, I am honored to invite Chris to be the champion of the TsFile project. For PMC, welcome to recommend yourself and others. Thanks, — Jialin Qiao Apache IoTDB PMC
Re: Rollcall TsFile initial committers
Hi Jialin, I also would like to be the initial committer for the TsFile project。 I am familiar with the format of TsFile and have contributed features such as cross multi time partition load TsFile. Currently, I am also working on OLAP, and some OLAP databases already support reading file formats such as parquet and orc. I hope that TsFile can become a top-level project in Apache, so that other OLAP databases can support reading TsFile format and expand the usage scenarios of TsFile. I am willing to make my contribution to this. Thanks, Houliang Replied Message | From | Xiangdong Huang | | Date | 10/20/2023 22:29 | | To | | | Subject | Re: Rollcall TsFile initial committers | Hi Jialin, I would also like to be an initial committer for the TsFile project. Best, --- Xiangdong Huang Kun Liu 于2023年10月20日周五 16:28写道： Hi jialin I also would like to be the initial committer for the TsFile project, and participated the initial version of the TsFile format. I am familiar with the `write path` and Rust language, if user and community need the API for Tsfile with Rust I can help to make it. Thanks, Kun Jialin Qiao 于2023年10月18日周三 12:48写道： Hi, Glad to receive so much support! I also would like to be the initial committer of TsFile. I'm familiar with the tech part(format, read and write) of TsFile, application scenarios and solutions with TsFile and IoTDB. Looking forward to making TsFile the infrastructure of IoT. Thanks, — Jialin Qiao Apache IoTDB PMC Gaofei Cao 于2023年10月18日周三 00:20写道： Hi Jialin, I would like to be the initial committer of TsFile. I participated in the design and development of the first version of TsFile, familiar with the query process, file structure and other system integrations. And now I'm participating in the work to push down value filter. I wish I could bring more features to make TsFile a better file format. Thanks Gaofei Cao Haonan Hou 于2023年10月17日周二 14:39写道： Hi Jialin, I would like to be the initial committer of TsFile. I participated in the development of TsFile v2 for IoTDB 0.10.x and 0.11.x and the update tool for v1 to v2, v2 to v3. Thanks Haonan Hou On 2023/10/17 03:40:42 Xinyu Tan wrote: Hi Jialin, I would like to be the initial committer of TsFile. I participated in the design and development of IoTDB 0.12 cluster version combined with TsFile for incremental snapshot. Now the consensus layer is designing a consensus algorithm based on TsFile synchronization, which is expected to make TsFile more general in more scenarios involving CDC and consensus. Thanks Xinyu Tan On 2023/10/15 15:22:11 Jialin Qiao wrote: Hi, To make TsFile a TLP, we need to affirm the initial committer first. The criteria for becoming the initial committer of TsFile is prelimilary defined as meeting the following two conditions simultaneously. (1) Apache IoTDB PMC member (2) Made contributions to the TsFile module If you have made significant contributions to the TsFile module, you can also apply to become an initial committer. If you would like to be the initial committer of TsFile, please reply to this email in 1 week :-) Thanks, — Jialin Qiao Apache IoTDB PMC
Re:[VOTE] New Repo: iotdb-client-csharp
+1 And once the new repositories is created, perhaps it's best to contact the author(github id: eedalong) and see if they are willing to contribute code to iotdb-client-csharp Thanks, --- Houliang Qi BONC, Ltd Replied Message | From | gitda...@163.com | | Date | 05/20/2023 15:22 | | To | | | Subject | Re:[VOTE] New Repo: iotdb-client-csharp | +1 At 2023-05-20 14:57:56, "Jialin Qiao" wrote: Hi, Currently, there are two Repositories of IoTDB Csharp client. eedalong/Apache-IoTDB-Client-CSharp eedalong/Apache-IoTDB-Client-CSharp-UseCase I prefer to call for a vote for creating a new repo iotdb-client-csharp for the csharp client. Please vote accordingly: [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove with the reason The vote is open for the next 72 hours and passes if at least three +1 votes and more +1 votes than -1 votes. Best, — Jialin Qiao Apache IoTDB PMC
Re: [discuss] consider revert the feature of multi-tenancy
A healthy community requires different voices. It is these different voices that enable us to strictly review each feature and consider each scenario more comprehensively, so that we can better provide users with better features. I am also very grateful to the colleagues who participated in this discussion. If it is still the previous discussion point, I will stick to my previous opinion and will not reply again. If someone brings up a new discussion point, I think it is possible to open another discussion. Thanks. --- Houliang Qi BONC, Ltd Replied Message | From | Houliang Qi | | Date | 04/12/2023 20:08 | | To | dev@iotdb.apache.org | | Subject | Re: [discuss] consider revert the feature of multi-tenancy | Hi, I would like to end by stating my point of view. And I don't want to say it a second time. First of all, I agree to change the name of this feature to resource control. The reason why I think that both multi-tenancy and resource-control are suitable for us is that what we are currently doing is to limit the functions of users or databases resources. @Jinrui 2. A true multi-tenant system should also have features such as per-tenant configurations, customized tenant roles and permissions, and clear separation of tenant data. @Jinrui 3. In our case, the lack of configuration file isolation in the current implementation means that different tenants may inadvertently affect each other's configurations, which goes against the core principle of multi-tenancy. The example I raised in last mail is also used to illustrate it rather than the function scope definition. But I disagree with Jinrui said above: You can refer to this paper [1]. This document summarizes multi-tenancy very well. First of all, it does not mean that multi-tenancy must have the ability to configure isolation. For this feature, we do share configuration, but our data is logically isolated and can limit some resources such as cpu, memory, timeseries, etc. This is also a way to realize multi-tenancy. Of course, each tenant can even use a different operating system, which is also a way to realize multi-tenancy. Please don't limit the way to realize multi-tenancy. There I would like to paste a passage from the paper as a conclusion: By sharing infrastructure, middleware, or application among tenants, multi-tenancy can be achieved In a multi-tenant architecture, a single instance of the software is deployed on the server functioning on the service provider infrastructure. It provides access to multiple tenants at the same time, as describe by Karataş et al. [KCD+17]. Abdul et al. [ABG+18]describe several multi-tenant data layer models, such as dedicated, isolated, shared, and hybrid. By sharing infrastructure, middleware, or application among tenants, multi-tenancy can be achieved. Figure 2.2 visualizes the multi-tenancy strategies described by Walraven et al. [WLJ14]. However, application-level multi-tenancy with a shared everything architecture can lead to cost reduction. Chong et al. [CCW06] have outlined several viable database patterns, primarily for Microsoft SQL Server, which supports application-level multi-tenancy. In a multi-tenant situation, they present three major strategies which are: distinct databases, shared databases with distinct schemes, and shared databases with shared schemes. [1] Enabling multi-tenant scalable IoT platforms. https://elib.uni-stuttgart.de/bitstream/11682/11024/1/Enabling%20multi-tenant%20scalable%20IoT%20platforms.pdf Thanks, --- Houliang Qi BONC, Ltd Replied Message | From | Jinrui 张金瑞<329920...@qq.com.INVALID> | | Date | 04/12/2023 19:14 | | To | | | Subject | Re: [discuss] consider revert the feature of multi-tenancy | Hi, Perhaps I need to emphasize again that regarding the multi-tenancy section discussed, the following is my conclusion: 1. The implementation of the PR cannot be called multi-tenancy 2. Multi-tenancy is not equal to resource control. The reasons are: 1. While resource isolation or control can be used to enable multi-tenancy, it's not the only requirement. 2. A true multi-tenant system should also have features such as per-tenant configurations, customized tenant roles and permissions, and clear separation of tenant data. 3. In our case, the lack of configuration file isolation in the current implementation means that different tenants may inadvertently affect each other's configurations, which goes against the core principle of multi-tenancy. The example I raised in last mail is also used to illustrate it rather than the function scope definition. @Chao: You think this pr function is missing, and I can understand it, after all, you have found some bad cases. So next time if there are other PRs and I find out the bad case, is it better to consider launching a discussion and reverting it? (Jinrui): You have
Re: [discuss] consider revert the feature of multi-tenancy
Hi, I would like to end by stating my point of view. And I don't want to say it a second time. First of all, I agree to change the name of this feature to resource control. The reason why I think that both multi-tenancy and resource-control are suitable for us is that what we are currently doing is to limit the functions of users or databases resources. > @Jinrui 2. A true multi-tenant system should also have features such as > per-tenant configurations, customized tenant roles and permissions, and clear > separation of tenant data. > @Jinrui 3. In our case, the lack of configuration file isolation in the > current implementation means that different tenants may inadvertently affect > each other's configurations, which goes against the core principle of > multi-tenancy. The example I raised in last mail is also used to illustrate > it rather than the function scope definition. But I disagree with Jinrui said above: You can refer to this paper [1]. This document summarizes multi-tenancy very well. First of all, it does not mean that multi-tenancy must have the ability to configure isolation. For this feature, we do share configuration, but our data is logically isolated and can limit some resources such as cpu, memory, timeseries, etc. This is also a way to realize multi-tenancy. Of course, each tenant can even use a different operating system, which is also a way to realize multi-tenancy. Please don't limit the way to realize multi-tenancy. There I would like to paste a passage from the paper as a conclusion: > By sharing infrastructure, middleware, or application among tenants, > multi-tenancy can be achieved > In a multi-tenant architecture, a single instance of the software is deployed > on the server functioning on the service provider infrastructure. It provides > access to multiple tenants at the same time, as describe by Karataş et al. > [KCD+17]. Abdul et al. [ABG+18]describe several multi-tenant data layer > models, such as dedicated, isolated, shared, and hybrid. By sharing > infrastructure, middleware, or application among tenants, multi-tenancy can > be achieved. Figure 2.2 visualizes the multi-tenancy strategies described by > Walraven et al. [WLJ14]. However, application-level multi-tenancy with a > shared everything architecture can lead to cost reduction. Chong et al. > [CCW06] have outlined several viable database patterns, primarily for > Microsoft SQL Server, which supports application-level multi-tenancy. In a > multi-tenant situation, they present three major strategies which are: > distinct databases, shared databases with distinct schemes, and shared > databases with shared schemes. [1] Enabling multi-tenant scalable IoT platforms. https://elib.uni-stuttgart.de/bitstream/11682/11024/1/Enabling%20multi-tenant%20scalable%20IoT%20platforms.pdf Thanks, --- Houliang Qi BONC, Ltd Replied Message | From | Jinrui 张金瑞<329920...@qq.com.INVALID> | | Date | 04/12/2023 19:14 | | To | | | Subject | Re: [discuss] consider revert the feature of multi-tenancy | Hi, Perhaps I need to emphasize again that regarding the multi-tenancy section discussed, the following is my conclusion: 1. The implementation of the PR cannot be called multi-tenancy 2. Multi-tenancy is not equal to resource control. The reasons are: 1. While resource isolation or control can be used to enable multi-tenancy, it's not the only requirement. 2. A true multi-tenant system should also have features such as per-tenant configurations, customized tenant roles and permissions, and clear separation of tenant data. 3. In our case, the lack of configuration file isolation in the current implementation means that different tenants may inadvertently affect each other's configurations, which goes against the core principle of multi-tenancy. The example I raised in last mail is also used to illustrate it rather than the function scope definition. @Chao: You think this pr function is missing, and I can understand it, after all, you have found some bad cases. So next time if there are other PRs and I find out the bad case, is it better to consider launching a discussion and reverting it? (Jinrui): You have raised a good point. In some cases, reverting the code might be the best option to prevent further damage. So, yes, please feel free to start a discussion if you find any issues in the future. I appreciate your understanding and willingness to consider discussions and reverts if any bad cases are found in future PRs. It is definitely welcomed. It’s important for us to maintain a high standard of code quality and ensure that our users can rely on our product. Let's continue to work together to achieve this goal. Thanks, Zhang Jinrui 2023年4月12日 下午6:13，Chao Wang 写道： Hi， Especially when exposing these concepts to users, we don't want to crea
Re: [discuss] consider revert the feature of multi-tenancy
Hi, Jinrui > (Jinrui): Let's say there are two tenants in the system, Tenant A and Tenant > B. In the current implementation, if Tenant A disables the compaction > function, it will also be disabled for Tenant B. This means that Tenant B has > no control over this feature and must accept the configuration set by Tenant > A. This is not an ideal scenario for multi-tenancy, as each tenant should be > able to configure their own settings independently without affecting other > tenants. > Additionally, if one user modifies the configuration of their IoTDB instance, > it can potentially impact the experience of other users sharing the same > resources. This is not desirable for multi-tenancy, as each tenant should be > isolated from each other and their actions should not impact other tenants. > Therefore, it's important to have proper multi-tenancy implementation to > avoid these kinds of issues and ensure each tenant can operate independently > without interfering with each other. In our user manual, is it mentioned that different tenants can set different compaction parameters? Did it mention that different users can set different configurations? We mentioned that disk, timeseries, devices, cpu and meory, etc. can be limited in different tenants. Any function has a scope, does it make sense to discuss something beyond that scope? Why don't you say that different tenants can have different versions of iotdb? Can different tenants have different cpu chips? Can different users have different operating systems? I think the example you gave above is beyond the scope of the user manual we gave, and it doesn't make sense. Thanks, ------- Houliang Qi BONC, Ltd Replied Message | From | Jinrui 张金瑞<329920...@qq.com.INVALID> | | Date | 04/12/2023 17:28 | | To | | | Subject | Re: [discuss] consider revert the feature of multi-tenancy | Hi, @Chao: Well, there is a difference between multi-tenancy and resource control as you define. Sometimes in communication, a certain application form is often used to refer to the core technology or representative technology, or the core technology represents a certain application form. I think it is not accurate, but it is acceptable. Just like when I say cloud native, I will naturally think of k8s, and when it comes to containers, I will think of docker. I think the core technology of multi-tenancy is resource control or isolation, so it is also possible to mention multi-tenancy. Functions similar to multi-tenancy can be realized based on the resource control function. (Jinrui): I understand your point of view, but I still believe that there is a difference between multi-tenancy and resource control. While it may be acceptable to use a certain application form to refer to a core technology or representative technology, I think it's important to be accurate in our terminology, especially when it comes to technical discussions. In my opinion, the core technology of multi-tenancy is about providing separate environments for different tenants, while resource control or isolation is a means to achieve that. While functions similar to multi-tenancy can be realized based on the resource control function, I think it's important to distinguish between the two and use the appropriate term depending on the context. Especially when exposing these concepts to users, we don't want to create any misunderstandings about the core functionality of IoTDB, which is also mentioned in Xiangdong's mail. That's why we need to be precise in our definitions and avoid interchangeable usage of terms that could lead to confusion. @Chao: I think there is a problem with this code and it needs to be fixed. You can continue to submit the code to fix it. We don't need to revert and wait for the repair to be completed before submit it. What you emphasize is that it affects the stability of the code, so it's better to revert. I Agree with that. But I don’t see how a function that is turned off by default affects the stability and affects the user? Has it affected the development of other functions? Does it affect pipeline testing? Only bug-free code does not affect stability? As Houliang said, can someone be sure that their code has no bugs? (Jinrui): While it's true that no one can guarantee bug-free code, it's also important to ensure that the code meets the minimum requirements for functionality and stability. We don't want to risk introducing more issues to the system by merging incomplete or flawed code. I would like to clarify that the current issue with the code is not just about the presence of bugs, but also about the basic functionality that is missing. Furthermore, even if the function is turned off by default, it could still have an impact on the development of other functions. I understand that we may have different opinions on whether this code should
Re: [discuss] consider revert the feature of multi-tenancy
Hi, Jinrui > (Jinrui): Just like how I believe that multi-tenancy and multi-user are not > the same thing, regarding the suggestion to interchange "multi-tenancy" and > "resource control," while they may have some similarities in concept, they > are not interchangeable. First of all, you misunderstood me. I never said that multi-tenancy and multi-user are the same thing. What I want to emphasize is: one user per tenant is also one of the ways to realize multi-tenancy, and one tenant manages one database, which is also one of the ways to realize multi-tenancy, and dories, spanner also has such an implementation. > (Jinrui): multi-tenancy is a specific approach to architecture that enables > multiple tenants to share the same resources while keeping their data > isolated, whereas resource control is a broader term that can refer to any > mechanism used to manage and allocate system resources. Second: why can't we call it multi-tenant? We can achieve logical isolation of data through this feature and auth permission, and we can also manage and control users and database resources. Doesn't this just verify what you said above? Thanks, --- Houliang Qi BONC, Ltd Replied Message | From | Jinrui 张金瑞<329920...@qq.com.INVALID> | | Date | 04/12/2023 14:55 | | To | | | Subject | Re: [discuss] consider revert the feature of multi-tenancy | Hi, the following description assumes that multi-tenancy = resource control, the two words can be interchanged. Are there any objections? (Jinrui): Just like how I believe that multi-tenancy and multi-user are not the same thing, regarding the suggestion to interchange "multi-tenancy" and "resource control," while they may have some similarities in concept, they are not interchangeable. Multi-tenancy is a specific approach to architecture that enables multiple tenants to share the same resources while keeping their data isolated, whereas resource control is a broader term that can refer to any mechanism used to manage and allocate system resources. Based on this, continue to infer, if it is found that the released version has a bug a few days after it was released, should the release be cancelled, and it is better to wait for the bug to be fixed. (Jinrui): Actually, we were not discussing how to do code control. But since you brought up the issue of code quality, do you think the current code is of sufficient quality to be merged? Or did you notice the issue when reviewing the code? Of course, I understand that we don't have a quantifiable standard to judge whether code is mergeable or not, and everyone has their own criteria for making that judgment. Regarding your question about cancelling a release if a bug is found a few days after it was released, I think it depends on the severity of the bug and the impact it has on users. If the bug is minor and does not affect the core functionality of the software, it may be acceptable to wait for the bug to be fixed in the next release. However, if the bug is severe and causes major issues for users, it may be necessary to cancel the release and fix the bug immediately. As for the fact that it hasn’t been fixed for two days, this function is being discussed whether it should be discarded or not. Does it still take time to fix bugs? (Jinrui): I didn't fully understand what you meant, could you please explain it again? Thanks, Zhang Jinrui 2023年4月12日 下午1:35，Chao Wang 写道： Hi all, It seems that Houliang didn't make it very clear before. Let me add some more information. If you think this code is doing multi tenant things now, why do we need to change the name to some other words like resource control ? Is it > just to prevent users from misunderstandings from the definition? If that's the case, does it mean that users perceive multi tenancy as different from what we do? This is contradictory. Why change multi-tenancy to resource control is just because everyone’s perception of multi-tenancy is not very unified. It may be more unified to change it to resource control. This should not be contradictory. Suppose, under the advance statement in pr, the following description assumes that multi-tenancy = resource control, the two words can be interchanged. Are there any objections? It has been two days since this code was merged, and I don't know if anyone is fixing the issue I mentioned. Bug is definitely accepted, but I don't think this issue belongs to a 'bug' because it failed even the most basic functional testing. From your test, it is true that there is a problem with this pr, and there is no pr fix at present. For the stability of the code base, it should be better to revert, even if this function is turned off by default (reminds me of developing a new version of distributed Framework, it seems that many codes are merged when they can
Re: [discuss] consider revert the feature of multi-tenancy
Hi, all I think we have reached a consensus from the discussion, change the name of this feature to resource control, and continue to contribute to this feature. Thank you for your concern. Thanks, --- Houliang Qi BONC, Ltd Replied Message | From | Xiangdong Huang | | Date | 04/11/2023 15:39 | | To | | | Subject | Re: [discuss] consider revert the feature of multi-tenancy | Hi Houliang, It makes no sense to refer Doris. Doris is not a lightweight db, and edge side is never its goal. The topic of this discussion is whether to revert the feature of multi-tenancy. I wonder why you fall into these words I think I have mentioned at least twice (or maybe 3 times) that Jialin's suggestion is fine for me. Best, --- Xiangdong Huang School of Software, Tsinghua University Houliang Qi 于2023年4月11日周二 15:05写道： Hi Jinrui, (Jinrui) From my perspective, Multi-tenancy is different from resource-control and they are not the different term for same thing. According to our implementation, current feature focus on the resource control on users of one tenant rather than on different tenants. If we did not reflect the wording `multi-tenancy` in the code, why do we use it on user docs and PR's description ? Sorry, I am not agree with you, from my perspective, a user is a tenant, and each tenant has different resources. This is also multi-tenancy. Even each tenant can only have one db. In our current implementation, a user is a tenant. For doris, they also mention multi-tenancy, but it is limited user resources.[1], the same as our current implementation. For Spanner, a tenant can also have only one db. [2] The reason why I think that both multi-tenancy and resource-control are suitable for us is that what we are currently doing is to limit the functions of users or db resources. On this point, I agree with Wang Chao's point of view. As for whether the multi-tenant function you mentioned affects the positioning of IoTDB, I don't think it is accurate. I personally think that the multi-tenant function is a term for resource isolation technology and will not affect the positioning of IoTDB. I don't know how you define the multi-tenant function. If it refers to the connection with the billing system of the cloud service provider, it may be another form. This discussion will not continue to discuss multi-tenancy. (Jinrui) REVERT does not mean REJECT. It is only a quick way to keep the code more reliable before we reach the same page. And furthermore, I don't think it is harmful or discouraging and it is only a regular way we use to replace hot-fix. (Jinrui) The reviewers may be confused by the PR's description and then focus on whether `multi-tenant` should be integrated in current development stage of IoTDB. The topic of this discussion is whether to revert the feature of multi-tenancy. I STRONGLY think that this PR does not violate the positioning and future development of IOTDB, so I STRONGLY think that revert is not needed, as this function is not enabled by default, and we are continuing Iterate and refine this feature. Before the actual release, it is necessary to consider some scenarios and do some testing. [1] https://doris.apache.org/docs/dev/admin-manual/multi-tenant/ [2] https://cloud.google.com/solutions/implementing-multi-tenancy-cloud-spanner Thanks, --- Houliang Qi BONC, Ltd Replied Message | From | Chao Wang | | Date | 04/11/2023 13:42 | | To | dev@iotdb.apache.org | | Subject | Re: Re：Re: [discuss] consider revert the feature of multi-tenancy | Everyone's contribution counts. But what we are talking about is whether `multi-tenancy` is suitable for current IoTDB's development. From my perspective, Multi-tenancy is different from resource-control and they are not the different term for same thing. According to our implementation, current feature focus on the resource control on users of one tenant rather than on different tenants. If we did not reflect the wording `multi-tenancy` in the code, why do we use it on user docs and PR's description ? As I said before, the description is indeed not very clear, and the description can be modified as a resource control. So what's the point of wondering if this pr is a multi-tenant function? Even if it is a multi-tenant function, how will it affect the development of IoTDB? REVERT does not mean REJECT. It is only a quick way to keep the code more reliable before we reach the same page. And furthermore, I don't think it is harmful or discouraging and it is only a regular way we use to replace hot-fix. Yes, revert is a normal process, and PR also has some problems. Let's discuss the reason for reverting this PR. As Xiangdong said, this is a feature that will affect the positioning of IoTDB, so how does this feature affect the positioning of IoTDB? Agree. But if we don't make
Re: [discuss] consider revert the feature of multi-tenancy
Hi Jinrui, > (Jinrui) From my perspective, Multi-tenancy is different from > resource-control and they are not the different term for same thing. > According to our implementation, current feature focus on the resource > control on users of one tenant rather than on different tenants. If we did > not reflect the wording `multi-tenancy` in the code, why do we use it on user > docs and PR's description ? Sorry, I am not agree with you, from my perspective, a user is a tenant, and each tenant has different resources. This is also multi-tenancy. Even each tenant can only have one db. In our current implementation, a user is a tenant. For doris, they also mention multi-tenancy, but it is limited user resources.[1], the same as our current implementation. For Spanner, a tenant can also have only one db. [2] The reason why I think that both multi-tenancy and resource-control are suitable for us is that what we are currently doing is to limit the functions of users or db resources. On this point, I agree with Wang Chao's point of view. > As for whether the multi-tenant function you mentioned affects the > positioning of IoTDB, I don't think it is accurate. I personally think that > the multi-tenant function is a term for resource isolation technology and > will not affect the positioning of IoTDB. I don't know how you define the > multi-tenant function. If it refers to the connection with the billing system > of the cloud service provider, it may be another form. This discussion will > not continue to discuss multi-tenancy. > (Jinrui) REVERT does not mean REJECT. It is only a quick way to keep the code > more reliable before we reach the same page. And furthermore, I don't think > it is harmful or discouraging and it is only a regular way we use to replace > hot-fix. > (Jinrui) The reviewers may be confused by the PR's description and then focus > on whether `multi-tenant` should be integrated in current development stage > of IoTDB. The topic of this discussion is whether to revert the feature of multi-tenancy. I STRONGLY think that this PR does not violate the positioning and future development of IOTDB, so I STRONGLY think that revert is not needed, as this function is not enabled by default, and we are continuing Iterate and refine this feature. Before the actual release, it is necessary to consider some scenarios and do some testing. [1] https://doris.apache.org/docs/dev/admin-manual/multi-tenant/ [2] https://cloud.google.com/solutions/implementing-multi-tenancy-cloud-spanner Thanks, --- Houliang Qi BONC, Ltd Replied Message | From | Chao Wang | | Date | 04/11/2023 13:42 | | To | dev@iotdb.apache.org | | Subject | Re: Re：Re: [discuss] consider revert the feature of multi-tenancy | Everyone's contribution counts. But what we are talking about is whether `multi-tenancy` is suitable for current IoTDB's development. From my perspective, Multi-tenancy is different from resource-control and they are not the different term for same thing. According to our implementation, current feature focus on the resource control on users of one tenant rather than on different tenants. If we did not reflect the wording `multi-tenancy` in the code, why do we use it on user docs and PR's description ? As I said before, the description is indeed not very clear, and the description can be modified as a resource control. So what's the point of wondering if this pr is a multi-tenant function? Even if it is a multi-tenant function, how will it affect the development of IoTDB? REVERT does not mean REJECT. It is only a quick way to keep the code more reliable before we reach the same page. And furthermore, I don't think it is harmful or discouraging and it is only a regular way we use to replace hot-fix. Yes, revert is a normal process, and PR also has some problems. Let's discuss the reason for reverting this PR. As Xiangdong said, this is a feature that will affect the positioning of IoTDB, so how does this feature affect the positioning of IoTDB? Agree. But if we don't make it clear before PR merged, pushing forward the discussion is better than directly merging, from my side. Agree. I remember sending an email to discuss it. Does that mean that every PR needs to send an email to discuss clearly? After all, pushing forward the discussion is better than directly merging. Thanks! Chao Wang BONC ltd On 4/11/2023 13:10，张金瑞<329920...@qq.com.INVALID> wrote： (Sorry for the format issue in previous mail) == Hi, all I tried this feature locally according to the User Manual, and I am blocked at the beginning. Firstly, I didn't found the parameters `quota_enable` and `rate_limiter_type` in iotdb-common.properties to enable this functionality. I am not sure whether it is by design but it is not aligned with the user manual. And I have to add thes
Re: [discuss] consider revert the feature of multi-tenancy
Hi, all Leaving aside this feature, has the PR of the big feature we merged in been discussed in detail? How to define detailed discussion? I think that we should discuss some of our discussion points clearly at the beginning of the design, instead of how to revert the PR after the PR is merged. I think there is a problem with this process. Who can guarantee that there are no bugs and no problems in the developed functions, and we are all improving through continuous iteration. And this feature also refers to the design of some other excellent projects, such as doris and hbase. As for the name of this feature, in doris, it is called multi-tenancy[1], in hbase it is called quota[2], we can call it resource-control, I think it is ok. After all, we did not reflect the wording of multi-tenancy in the code implementation. [1] https://doris.apache.org/docs/dev/admin-manual/multi-tenant [2] https://hbase.apache.org/book.html#quota Thanks, --- Houliang Qi BONC, Ltd Replied Message | From | Chao Wang | | Date | 04/11/2023 09:15 | | To | dev@iotdb.apache.org | | Cc | dev@iotdb.apache.org | | Subject | Re: [discuss] consider revert the feature of multi-tenancy | Hi, Xiangdong, what is the side effect when we manually create a time series? How about the pr https://github.com/apache/iotdb/pull/9430, limit the timeseries number of cluster, anyone analyze the side effect about creating a time series? This discuss is not for getting "+1" or "-1" (though anyone can reply the vote..). I just want to discuss that do we REALLY consider and analyze the feature and the implementation carefully? Why not discuss before the PR submission, but wait until the PR submission before discussing, wouldn't it waste the energy of community participants? I have also seen emails sent before, not without notifying everyone. In addition, I think Jialin's suggestion is more reasonable. The description of this function may not be particularly clear. It can be said in another way, such as resource control. However, reverting will undoubtedly be harmful to the community, will discourage the enthusiasm of community participants, and is very unfriendly to community participants. If in doubt, I think it would be better to raise it as soon as possible, instead of waiting for others to finish their hard work before questioning. Another point is that the multi-tenancy function may be a function required by other companies' IOTDB releases, but will other people's contributions to the community affect the development of the community? I think it will be more conducive to the development of community diversity. Thanks! Chao Wang BONC ltd ccgow...@163.com On 4/10/2023 23:45，Xiangdong Huang wrote： Besides the above, when we merge this pr, we posted the design in the feishu[4] and discussed it online as least two times, and emailed and discussed it with everyone[5], it has been passed 10 days. I think I know this and I have shown my concern about the possible harm of this featuer to IoTDB's edge mode... 1) how many side-effects the feature will bring; We have done some tests under[1], which says with 20 databases and 1 user when we set `quota_enable` to true to enable the multi-tenancy feature, the write performance is only slowed down 1.75%, the read latency has not much difference, we will do more tests to show the side-effects in the feature. The experiment is rather simple... When we really want to show the added codes having no side-effects, all the exepriemnt settings should follow a rule that how to fully expose the possible problems. For example, as mult-tenancy limits the available # of devices, timeseries, and the spaces of disk, it should have side-effect on create new device/timeseries, and writing new data. So, - what is the side effect when we manually create a time series? - what is the side effect when we use automatical creating a time series? - what is the side effect when we write new data? (as the data can be compressed when it is flushed on disk in async mode, how to check the disk space?). Besides, as it impaces each write operation, we need to focus on write operstions which's batchsize=1. This discuss is not for getting "+1" or "-1" (though anyone can reply the vote..). I just want to discuss that do we REALLY consider and analyze the feature and the implementation carefully? If not, then this big feature is not the time to be merged (and I will call a vote then), and then let's rethink it and make it really available together. If yes, we also need to rethink it and improve it for better performance. Best, --- Xiangdong Huang School of Software, Tsinghua University Chao Wang 于2023年4月10日周一 19:14写道： Agree with Houliang's opinion. Thanks! Chao Wang BONC ltd On 4/10/2023 19:01，Houliang Qi wrote： -1 First of all, thanks Xiangdong for
Re:[discuss] consider revert the feature of multi-tenancy
-1 First of all, thanks Xiangdong for pointing out IoTDB's Charter. "RESOLVED, that the Apache IoTDB Project be and hereby is responsible for the creation and maintenance of software related to an IoT native database with high performance for data management and analysis, on the edge and the cloud." As the charter post, IoTDB can be deployed in the cloud, this is why we deploy the multi-tenancy feature. The cloud can be a public or private cloud if we can deploy only one IoTDB cluster, and manage multi databases and users with different resources, which will simplify the maintenance. -> 1) how many side-effects the feature will bring; We have done some tests under[1], which says with 20 databases and 1 user when we set `quota_enable` to true to enable the multi-tenancy feature, the write performance is only slowed down 1.75%, the read latency has not much difference, we will do more tests to show the side-effects in the feature. -> 2) how to reduce the effect when IoTDB is deployed on the edge. We supply one switch about this feature, called `quota_enable`, by default this value is false, so it has no effect when IoTDB is deployed on the edge. This also answers Jinrui's doubt. -> 3) some checks failed on WinOS, are they irrelevant? No, I think they are not irrelevant, the false check message is about the Compaction module, and I see the former pr[2][3] which have been merged 4 days ago has the same issue, so I suspect that the compaction module has occasional bugs -> 4) The feature SHOULD be discussed carefully in the community, rather that submit PRs and merged after some reviews. Besides the above, when we merge this pr, we posted the design in the feishu[4] and discussed it online as least two times, and emailed and discussed it with everyone[5], it has been passed 10 days. The IoTDB community is open and different opinions are welcome. After all, we all have the same original intention of wanting IoTDB's features to be more diverse. [1] https://apache-iotdb.feishu.cn/docx/DbqCd8t3EoxlCFx1yYicd9N4n4s [2] https://github.com/apache/iotdb/actions/runs/4625220921/jobs/8181102446 [3] https://github.com/apache/iotdb/actions/runs/4531046594/jobs/7980725316 [4] https://apache-iotdb.feishu.cn/docx/doxcnKOYKDmJ40FpVnVsPMd3nTg [5] https://lists.apache.org/thread/y6dqcm2o7qk0nbkllb61bp8cv6d3m1h7 Thanks, ------- Houliang Qi BONC, Ltd Replied Message | From | 张金瑞<329920...@qq.com.INVALID> | | Date | 04/10/2023 15:03 | | To | dev | | Subject | Re：[discuss] consider revert the feature of multi-tenancy | +1, Agree with Xiangdong's opinion. And on the other hand, checking this PR's side effects may take lot of time and during this period, there may be lots of users using latest code to deploy/upgrade their systems. So the best practice is reverting this PR until the side-effect is eliminated Thanks, Zhang Jinrui，Apache IoTDB PMC Original From:"Xiangdong Huang"< saint...@gmail.com ; Date:2023/4/10 10:05 To:"dev"< dev@iotdb.apache.org ; Subject:[discuss] consider revert the feature of multi-tenancy Hi all, I see the multi-tenancy feature is merged, and several committers made a lot of contributions on that. As multi-tenancy is quite a big feature, which may change IoTDB's position. The feature SHOULD be discussed carefully in the community, rather that submit PRs and merged after some reviews. Therefore, I call to revert the PR and discuss ASAP about the feature after that. At least, the proposer need to answer the following questions, 1) how many side-effect the feature will bring; 2) how to reduce the effect when IoTDB is deployed on the edge. 3) some checks failed on WinOS, are they irrelevant? I don't mean of rejecting any big contribution to IoTDB or harming the community's diversity, but accepting this feature is really big decision and it deserves us to take time to deliberate. Attached IoTDB's Charter: "RESOLVED, that the Apache IoTDB Project be and hereby is responsible for the creation and maintenance of software related to an IoT native database with high performance for data management and analysis, on the edge and the cloud." [1] https://github.com/apache/iotdb/pull/9534/checks Best, --- Xiangdong Huang School of Software, Tsinghua University
The multi-tenancy design about iotdb
Hi everyone, we have designed a multi-tenancy function of iotdb. which can limit the resource usage of databases or users; the following link is the user's manual[1] and the design doc[2]. And we have developed one of the space quota[3], due to no reviewers for a long time, which have some conflicts now, We will resolve the conflicts later if no one has objections to the design. And submit the remaining code to the community. Any comments are welcomed. [1] https://apache-iotdb.feishu.cn/docx/BHu0dnhN6oi7ndxcAxMccuDVnEf (In Chinese) [2] https://apache-iotdb.feishu.cn/docx/doxcnqR74WVa32KWZICqVm495nh https://apache-iotdb.feishu.cn/docx/YaJldCLDHopdgfxUlCkcBxQ2nSR [3] https://github.com/apache/iotdb/pull/8833/files Thanks, --- Houliang Qi BONC, Ltd
Re: new committer: Haiming Zhu
Congratulations! Haiming made a great work for WAL. Thanks, --- Houliang Qi BONC, Ltd Replied Message | From | 谭新宇<1025599...@qq.com.INVALID> | | Date | 01/5/2023 10:09 | | To | | | Subject | Re: new committer: Haiming Zhu | Congratulations Haiming! Xinyu Tan 2022年12月28日 23:31，Jialin Qiao 写道： Hi, The Project Management Committee (PMC) for Apache IoTDB has invited Haiming Zhu to become a committer and we are pleased to announce that he has accepted. Being a committer enables easier contribution to the project since there is no need to go via the patch submission process. This should enable better productivity. You could get the write-access of our repository by: (1) Update your github id in: id.apache.org , also an email (anything to your apache email will forward to this email). (2) Accept the invitation in github to be a member of the apache group. (3) Link your accounts via: git.apache.org (To lighten the third option MFA Status, you need to download Google Authenticator (Google 身份验证器) in your mobile) (4）Sign in github by Google Authenticator (5) Clone repository with ssh. Welcome, Haiming! Yours, The Apache IoTDB PMC
Re: new committer: Rongzhao Chen
Congratulations！ ConfigNode is the brain of the cluster, Rongzhao make a great contribution to ConfigNode. Hope to see more useful functions contributed by Rongzhao! Thanks, --- Houliang Qi BONC, Ltd Replied Message | From | Eric Pai | | Date | 01/4/2023 17:15 | | To | dev@iotdb.apache.org | | Subject | 答复: new committer: Rongzhao Chen | Congratulations, Rongzhao! 发件人: Jialin Qiao 日期: 星期三, 2023年1月4日 17:10 收件人: dev@iotdb.apache.org 主题: new committer: Rongzhao Chen Hi, The Project Management Committee (PMC) for Apache IoTDB has invited Rongzhao Chen to become a committer and we are pleased to announce that he has accepted. Being a committer enables easier contribution to the project since there is no need to go via the patch submission process. This should enable better productivity. You could get the write-access of our repository by: (1) Update your github id in: id.apache.org , also an email (anything to your apache email will forward to this email). (2) Accept the invitation in github to be a member of the apache group. (3) Link your accounts via: git.apache.org (To lighten the third option MFA Status, you need to download Google Authenticator (Google 身份验证器) in your mobile) (4）Sign in github by Google Authenticator (5) Clone repository with ssh. Welcome, Yongzao! Yours, The Apache IoTDB PMC
Re: new PMC: Chao Wang
Congratulation! Thanks, --- Houliang Qi BONC, Ltd Replied Message | From | Chao Wang | | Date | 12/16/2022 17:18 | | To | dev@iotdb.apache.org | | Subject | Re: new PMC: Chao Wang | congrats! Thanks! Chao Wang BONC ltd ccgow...@163.com On 12/16/2022 16:21，lu wrote： congrats, well-deserved | | cmlmaka...@126.com | | 邮箱：cmlmaka...@126.com | 回复的原邮件 | 发件人 | Zhou Yifu | | 日期 | 2022年12月15日 17:19 | | 收件人 | dev@iotdb.apache.org | | 抄送至 | | | 主题 | Re: new PMC: Chao Wang | Congrats! Well-deserved! Thanks, Yifu Zhou 获取 Outlook for iOS<https://aka.ms/o0ukef; 发件人: Jialin Qiao 发送时间: Thursday, December 15, 2022 2:56:44 PM 收件人: dev@iotdb.apache.org 主题: new PMC: Chao Wang Hi, The Project Management Committee (PMC) for Apache IoTDB has invited Chao Wang to become a PMC and we are pleased to announce that he has accepted. Welcome, Chao Wang! Yours, The Apache IoTDB PMC
Re:[RESULT][VOTE] Release Apache IoTDB 1.0.0
Congratulations! It’s a big step for IoTDB! Thanks, --- Houliang Qi BONC, Ltd Replied Message | From | Haonan Hou | | Date | 12/5/2022 20:39 | | To | dev-iotdb | | Subject | [RESULT][VOTE] Release Apache IoTDB 1.0.0 | Hi all, The release is approved by accepting 8 +1 votes: 4 from PMCs, Jialin Qiao, Yuan Tian, Gaofei Cao, Chao Wang 2 from committers, HW-Chao Wang, CloudWise-luke.miao 2 from contributors, Guanfei Guo, William Song Vote threads: https://lists.apache.org/thread/q693fy4cgcbor1l30521px41qx6pt7td https://lists.apache.org/thread/byo1fpkczszvw5g7qf9pjk5szrw8q1s7 Thanks, Haonan Hou
Re: [VOTE] Apache IoTDB 1.0.0 RC3 release
-1 Can not registering a new confignode to the cluster, for more details, please see https://issues.apache.org/jira/browse/IOTDB-5105 Thanks, --- Houliang Qi BONC, Ltd Replied Message | From | Gaofei Cao | | Date | 12/1/2022 18:01 | | To | | | Subject | Re: [VOTE] Apache IoTDB 1.0.0 RC3 release | -1 LastFlushTime may be set incorrectly when DataRegion is recovering, please see more details in IOTDB-5103(https://issues.apache.org/jira/browse/IOTDB-5103) in RC3. Thanks, Gaofei Cao Haonan Hou 于2022年12月1日周四 13:14写道： Hi all, Apache IoTDB 1.0.0 version has a new architecture that supports standalone and cluster mode. Apache IoTDB 1.0.0 has been staged under [2] and it’s time to vote on accepting it for release. All Maven artifacts are available under [1]. Voting will be open for 72hr. A minimum of 3 binding +1 votes and more binding +1 than binding -1 are required to pass. Release tag: v1.0.0 Hash for the release tag: 3ed60f37a6d60dba93dbe46d1fdfc630bf3561b2 Before voting +1, PMC members are required to download the signed source code package, compile it as provided, and test the resulting executable on their own platform, along with also verifying that the package meets the requirements of the ASF policy on releases. [3] You can achieve the above by following [4]. [ ] +1 accept (indicate what you validated - e.g. performed the non-RM items in [4]) [ ] -1 reject (explanation required) [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1091 [2] https://dist.apache.org/repos/dist/dev/iotdb/1.0.0/rc3 [3] https://www.apache.org/dev/release.html#approving-a-release [4] https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release [5] https://dist.apache.org/repos/dist/dev/iotdb/1.0.0/rc3/RELEASE_NOTES.md [6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS Best, Haonan Hou
Re: About the standalone version
1C1D is really friendly for scalable distributed versions. However, in view of our claim of end-to-end cloud integrated time series database, I am more concerned about the performance of resource usage and performance when deployed on the edge compared with the current stand-alone version(v0.13). Thanks, --- Houliang Qi BONC, Ltd Replied Message | From | Eric Pai | | Date | 10/30/2022 13:23 | | To | dev@iotdb.apache.org | | Subject | Re: About the standalone version | Using a unified architecture is a convenient way to learn and expand IoTDB indeed. If so, we should consider the extra resource cost in edge environment, which may need more test in later versions. For example, a 4G4C docker container can support X timeseries and Y QPS in a single process architecture. If it migrates to a 2 processes one, could it support the performance requirements above as well? And what's the best practice of the usage ratio between conifg node of the total limited available resource, 2:2 or 1:3, or any others? 在 2022年10月30日，13:07，HW-Chao Wang <576749...@qq.com.invalid> 写道： +1，i support 1c1d for standalone verson，this is easy to learning for users. and then cluster version will be a trend. ---Original--- From: "Jialin Qiao"
Re: The structure of distribution
+1 Thanks, --- Houliang Qi BONC, Ltd Replied Message | From | Jialin Qiao | | Date | 10/20/2022 23:56 | | To | | | Subject | Re: The structure of distribution | +1 — Jialin Qiao Apache IoTDB PMC Haonan Hou 于2022年10月20日周四 18:32写道： Hi all, We will release v1.0 in the nearly future, it's time to decide the final version of distribution structure. After investigate other projects, like Hadoop and Spark. I think it will be better that remove the confignode and datanode folder, and combine their script and configuration files into the conf and sbin under IOTDB_HOME. The design document is in [1]. I also submitted a PR[2] to make such changes, in where you can find the screenshot of new structure. [1] https://apache-iotdb.feishu.cn/docx/doxcnFpCzjZadvMPQHUROW4mrTe [2] https://github.com/apache/iotdb/pull/7672 Best, — Haonan Hou
Re: [VOTE] Apache IoTDB 0.13.0 RC1 release
Hi, +1 (binding) The source release: LICENSE and NOTICE [ok] apache headers [ok] no jar files [ok] signatures and hashes [ok] The binary distribution: version number in CLI [ok] start in macOS, jdk8 [ok] statements executed successfully: [ok] SET STORAGE GROUP TO root.ln; insert into root.turbine.d1(timestamp,s0) values(1,1); select ** from root Thanks, --- Houliang Qi BONC, Ltd On 03/18/2022 17:52，JT wrote： Hi, +1 (binding) The source release: apache headers [ok] signatures and hashes [ok] LICENSE and NOTICE [ok] no jar files [ok] could compile from source: ./mvnw.sh clean install [ok] The binary distribution: version number in CLI [ok] signatures and hashes [ok] start in win11, jdk8 [ok] statements executed successfully: [ok] SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); select ** from root; Best Tian Jiang 发件人: Jialin Qiao 发送时间: 2022年3月18日 12:49 收件人: dev@iotdb.apache.org 主题: Re: [VOTE] Apache IoTDB 0.13.0 RC1 release Hi, +1 (binding) The source release: apache headers [ok] signatures and hashes [ok] LICENSE and NOTICE [ok] no jar files [ok] could compile from source: ./mvnw.sh clean install [ok] The binary distribution: version number in CLI [ok] signatures and hashes [ok] start in mac, jdk8 [ok] statements executed successfully: [ok] SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); select ** from root; Thanks, — Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院 清华大学软件学院薛恺丰 <827011...@qq.com.invalid> 于2022年3月18日周五 09:15写道： Hi +1 I have checked: 1. download source code 2. use benchmark to ingest data, raw data query, aggregation query and last point query. Best, Kaifeng Xue --原始邮件-- 发件人: "dev" https://repository.apache.org/content/repositories/orgapacheiotdb-1076 <https://repository.apache.org/content/repositories/orgapacheiotdb-1076 ; [2] https://dist.apache.org/repos/dist/dev/iotdb/0.13.0/rc1 < https://dist.apache.org/repos/dist/dev/iotdb/0.13.0/rc1; [3] https://www.apache.org/dev/release.html#approving-a-release < https://www.apache.org/dev/release.html#approving-a-release; [4] https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release < https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release ; [5] https://dist.apache.org/repos/dist/dev/iotdb/0.13.0/rc1/RELEASE_NOTES.md < https://dist.apache.org/repos/dist/dev/iotdb/0.13.0/rc1/RELEASE_NOTES.md ; [6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS < https://dist.apache.org/repos/dist/dev/iotdb/KEYS; Best, Haonan Hou
Re: [DISCUSS] Release IoTDB v0.12.5
+1, Looking forward to the v0.12.5 Thanks, --- Houliang Qi BONC, Ltd On 02/18/2022 11:03，Haonan Hou wrote： +1 for v0.12.5 Best. Haonan Hou On Feb 18, 2022, at 10:44 AM, Steve Su wrote: Hi all, It has been 2 months after the v0.12.4 release, and we can see many bug fixes and improvements have been introduced in the rel/0.12 branch. Here is the change list: # New Features * [IOTDB-2078] Split large TsFile tool * [IOTDB-2192] Support extreme function # Improvements * [IOTDB-1297] Refactor the memory control when enabling time partitions * [IOTDB-2195] Control the concurrent query execution thread * [IOTDB-2475] Remove sg not ready log in batch process * [IOTDB-2502] Add query sql in error log if encountering exception * [IOTDB-2506] Refine the lock granularity of the aggregation query * [IOTDB-2534] add character support while using double quote * [IOTDB-2562] Change default value of sync mlog period parameter * Avoid too many warning logs being printed, when opening too many file handlers # Bug Fixes * [IOTDB-1960] Fix count timeseries bug in cluster mode * [IOTDB-2174] Fix Regexp filter serializing and deserializing error * [IoTDB-2185] fix get an exception bug when parsing the header of CSV * [IOTDB-2194] Fix SHOW TIMESERIES will only display 2000 timeseries in cluster mode * [IOTDB-2197] Fix datatype conversion exception in Spark Connector * [IOTDB-2209] Fix logback CVE-2021-42550 issue * [IOTDB-2219] Fix query in-memory data is incorrect in cluster mode * [IOTDB-] Fix OOM and data was written in incorrectly bugs of Spark Connector * [IOTDB-2251] [IOTDB-2252] Fix Query deadlock * [IOTDB-2282] fix tag recover bug after tag update * [IOTDB-2320] MemoryLeak cause by wal Scheduled trim task thread * [IOTDB-2381] Fix deadlock caused by incorrect buffer pool size counter * [IOTDB-2400] Fix series reader bug * [IOTDB-2426] WAL deadlock caused by too many open files * [IOTDB-2445] Fix overlapped data should be consumed first bug * [IOTDB-2499] Fix division by zero error when recovering merge * [IOTDB-2507] Fix NPE when merge recover * [IOTDB-2528] Fix MLog corruption and recovery bug after killing system * [IOTDB-2532] Fix query with align by device can't get value after clear cache * [IOTDB-2533] Fix change max_deduplicated_path_num does not take effect * [IOTDB-2544] Fix tag info sync error during metadata sync * [IOTDB-2550] Avoid show timeseries error after alter tag on sync sender * Fix a logical bug in processPlanLocally in cluster mode * Throw Exception while using last query with align by device * Add a judgement to determine raft log size can fit into buffer before log appending in cluster mode * Fix grafana can't be used bug I think it's time to release v0.12.5. How do you think? Best, Steve Yurong Su Tsinghua University
Re: IoTDB-quality applying for subproject
Great! It’s really good news, and the IoTDB-quality is really useful! Hope to contribute to it! Thanks, --- Houliang Qi BONC, Ltd On 11/25/2021 10:48，Chao Wang wrote： Great! Thank you for your contribution! Thanks! Chao Wang BONC ltd ccgow...@163.com On 11/25/2021 10:16，Trevor Hart wrote： Great news as I would love to contribute to it! Thanks Trevor On Thu, 25 Nov 2021 15:15:03 +1300 陈 鹏宇 wrote Hi everyone, I'm on behalf of IoTDB-quality developers. We have recently decided to make our project open source, and formally apply it for a subproject of Apache IoTDB. IoTDB-quality is a collection of IoTDB UDFs designed for data quality diagnosis, including common statistics, anomaly detection, frequency analysis, regular expression processing, data quality analysis and data repairing. For more documentation, please visit the homepage of IoTDB-quality. https://thulab.github.io/iotdb-quality/ We are also hoping for contributions from community. Pengyu Chen Nov 25 2021
Re: About making an iterable cluster version in enterprise
Hi， We also have some similar situations in the test environment. We very much support creating a stable and available version based on the current version. Thanks, --- Houliang Qi BONC, Ltd On 11/24/2021 21:29，Xiangdong Huang wrote： Hi, I think it is fine to keep maintaining the current cluster module, either on rel/0.12 and master branch. Look forward to progress. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 程建云 于2021年11月24日周三 下午5:38写道： Hi, all After running cluster version with online stream about two weeks, we experienced two times of failures that cluster is no response and can't recover by restarting. And we didn't find an effective way to recover data from cluster. So we'd like to make testable cluster version in enterprise which should have the properties: 1. Write operation won’t be blocked frequently. 2. Query bugs are tolerant as it could be fixed and iterate quickly. 3. Most of issues could be resolve by restart nodes or cluster. 4. Exist a solution to solve the unrecoverable issue after lose small part of data. 5. Cluster restart could complete in a proper time. 6. System has monitor mechanism. We’re planning improve from below aspects: 1. Meta data use too much memory In our scenario, the measurement scale is large which would be around 1 billion but we have small data point ingestion (100K per second). We found the cluster node can’t afford the metadata storage as memory limitation(each nodes has 256G memory). As the small data point request rate, the CPU load is only about 1% ~ 2%. For the scenario, we intended to import some 3rd party storage component like RocksDB to help manage schema meta data. Of course, this would be optional and can be configured. 1. Raft implementation For this one, we planned to make it two steps. First, we’d like to abstract the interfaces of Raft, try to make Raft as a independent component. This should also be one work item when implement new architecture. Second, we’d like to import some 3rd party Raft library like Ratis and make it configurable ideally. 1. Engineering components Cluster missed some components like monitor system(this one should be working in progress by community, we’d like to help if needed), migration single node data into cluster which would help migrate single node to cluster and tools to help do failure recovery. We need to make these tools to make the system observable and recoverable. 1. Test As new test architecture is importing into community, we would try to complement test cases under new architecture. Most of the solutions above are not investigate deeply, any idea is welcomed. What’s the benefit of the work? We intend to make the version run on production so that we can collect feedback/bugs from real user and iterate by that. And finally become a baseline of stable cluster version. Why won’t make it in new architecture? We don’t do this under new architecture because the new architecture just started planning and we can’t wait anymore. And nearly all of the work doesn’t conflict with new architecture and could be usable in new architecture. Please feel free to reply the email to discussion if you have any concern or idea. Welcome to discuss if you have any concern. -- Thanks! Jianyun Cheng
Re:share a rust based IoTDB-client
Good！ Can we put it on our official website in the API page? In this way, users in need can find it directly. Thanks, --- Houliang Qi BONC, Ltd On 11/17/2021 10:42，Xiangdong Huang wrote： Hi, to whom may be interested in: I find a rust based iotdb-client when I explore Twitter :D [1] https://twitter.com/francis_run/status/1457699833981046790?s=20 [2] https://github.com/francis-du/iotdb-rs Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院
Re: What is the next architecture of IoTDB?
Hi, all This is really a good suggestion. I also think it is necessary to rethink the design of distributed architecture. The other important thing is if we start from scratch, the project may be relatively large, due to the high voice for the distributed version, and the community has also releases a 0.12 tested distributed version(only suggest tested, not suggest run it in product environment), I think we can first improve the distributed version that can be used in product environment based on the existing version. At the same time, when we are doing integration-test , we should consider the universality of the integration-test design, when we start to develop the next generation distributed architecture, the previously designed integration-test cases can be reused, So what we are doing now is meaningful. Thanks, --- Houliang Qi BONC, Ltd On 11/10/2021 12:35，Junqing Wang wrote： Agree with Eric Pai. The quality of the cluster version is very important. For now, we should focus more on quality rather than new architecture. Without the quality, the cluster version is hard to use in the production environment. Later I will contribute a PR with an improved IT testing framework that reuses existing IT cases to replay in a cluster environment. And I created a channel #integration-test in slack, everyone is welcome to join the discussion. Eric Pai 于2021年11月10日周三 上午11:14写道： Hi, It's nice to plan next generation architecture of IoTDB. However, before we start to design and discuss the refinement and optimization, we should ensure the cluster quality as our expected. In our inner test, by replaying the existed IT cases in a cluster environment, we have found many basic function work abnormally, e.g. ORDER BY DESC query, aggregation query, some filters with wrong serialize/deserialize logic, NPE, and so on. These bugs are easy to reappear but no one has fully tested them, and there's not a framework to ensure that every new feature, or existing feature work well both in cluster and standalone environment. Manual test by contributors is a heavy work with less scenario coverage. Designing new architecture is necessary, but I think the quality assurance may have the high priority than any other things. Good quality assurance make IoTDB to 60/100, and then we can make it to 100/100 by applying any optimizations. 在 2021/11/10 上午10:48，“songbing...@iie.ac.cn” 写入: What is the next architecture of IoTDB? As we know，IoTDB cluster version is based on the standalone architecture and almost reused all the basic components of standalone version and just add a multi-raft protocol based on the thrift. With the application spreading, the performance of the cluster version of IoTDB is changllaged for many aspects： the num of nodes in cluster can support, the write and read performance in multi-replicas, the linear ratio of read and write in cluster, the efficiency and smartness of scheduler in cluster based on the dynamic workloads, the joint query between time series data and relational data, the federal cluster query across data centers, AZ and regions and the spatio-temporal data analysis in one DB etc… So, I suggest we should make a blueprint for the next architecture of IoTDB and we can learn the advantage of the InfluxDB. The InfluxDB has designed the next architecture named IOx and use Fusion as the next query engine and adopt parquet as the file format etc. It's time to action, WDYT？ Bruce Song 宋秉华 icloudsong
Re:You are welcome to join the code review of Cluster module refactor
Hi, Jianyun It’s a wonderful job, thank you very much for your contribution. Thanks, --- Houliang Qi BONC, Ltd On 10/25/2021 10:32，jianyun cheng wrote： Hi, all Thanks for the contribution from Dr. Huang, Xinyu Tan and Sijia Li, the work of refactor cluster module is ready to review after months work. Especially thanks to Dr. Huang, who have lead the refactor and made a lot of work before the others join. Here is the summary of why we do that and what we have done. The Goal of Refactor The main goal of the refactor is make the code structure of cluster module more clear. Before the refactor, the structure of cluster module is a bit mess. It is not easy to understand even the one has been familiar with code of server module. As the server module is clear enough, and most of developer look into server when starts joining the project. So we decided to refer the structure of server module to refactor the cluster module. What we have done 1. Split Thrift RPC service and RPC implementation to make the logic here clear. For detail, refer the discuss: [Cluster-refactor] About refine classes name · Issue #3881 · apache/iotdb (github.com)<https://github.com/apache/iotdb/issues/3881> 2. Weaken the role of MetaGroupMember. metaGroupMember is just an engine for serving meta raft group, which should not control the whole server too deep. Many fields (like coordinator, etc.) are extracted to ClusterIoTDB (renamed from ClusterMain), and ClusterIoTDB is responsible for creating them. 3. Similar with the relationship between StorageEngine and StorageProcessor in the server module, DataGroupMember can be considered as StorageProcessor, we created a DataGroupEngine to control them. 4. Refactored thrift client class hierarchy to reduce the duplication and imported Apache commons-pool the help manager thrift client object. 5. Write related unit tests for new adding code. For existing code, fixed all failed unit test cases 6. Performance verify. Performance verify is on going, the result will be updated in the PR description. Although we have done the refactor very carefully, but as we don’t know everything very well, the unintentional mistakes especially when do code merge is hard to avoid. Your review and comments is very import for us. PR link: [IOTDB-1639] Refactoring the cluster class structure to make it consistent with the server module by LebronAl · Pull Request #4079 · apache/iotdb (github.com)<https://github.com/apache/iotdb/pull/4079> -- Jianyun Cheng Thanks!
Re: Some suggestions about our release schedule - [DISCUSS] ready for release v0.12.3?
Hi， It seems there have one bug fix pr not merged[1], I think after the pr was merged, then we can release v0.12.3. [1] https://github.com/apache/iotdb/pull/4015 Thanks, --- Houliang Qi BONC, Ltd On 09/23/2021 09:15，Haonan Hou wrote： Hi, It seems that it's over 3 days after this discussion. And more bug-fix PRs merged into rel/0.12 branch. Shall we release v0.12.3 now? If so, I would like to do the releasing works. Best, Haonan Hou On Sep 16, 2021, at 5:47 PM, Xiangdong Huang wrote: Hi, +1 to release a bugfix version that fixes critical bugs asap. But as ASF requires any consensus discussion should last 3 days, a tradeoff solution is, step 1, provide a bugfix solution first, and step 2, release the bugfix version 3 days later. In these 3 days, if some Bugfix PR wants to be released in this upcoming version, call reviewers in the mailing list and then we can promote the PR's priority to merge it in time. But, if the bugfix version is for fixing a critical bug, we will do not wait more after 3 days' discussion (which means the release train will launch on time). So, IMO, considering there is a critical bug, we can discuss which PRs should be released in v0.12.3, and begin to release v0.12.3 on Saturday. Welcome to show more suggestions. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Eric Pai 于2021年9月16日周四 下午5:05写道： Hi, As it's a critical bug fix, a patch version is necessary. +1 for releasing v0.12.3. Maybe we can publish a release schedule calendar in github or some other pages __, thus everyone will know the whole roadmap and milestones. 在 2021/9/16 下午4:42，“Jialin Qiao” 写入: Hi, Release big version at a fixed schedule is controllable. Bugfix version release could be separated into two: Suppose we just released a version. (1) If someone fixes a non-critical bug in this version, we could start a discussion: "how about release the next minor version 2 weeks later." This will make everyone comfortable to catch up with the release. (2) If someone fixes a critical bug in this version, we could discuss: "We just fix a critical bug, how about release now?" On this occasion, this could be a critical bug * IOTDB-1693 <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FIOTDB-1693data=04%7C01%7C%7Cf0ab608957a84e09508508d978edf348%7C84df9e7fe9f640afb435%7C1%7C0%7C637673785668847486%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=VNoXQ0yDE8QgFpPz1uq%2FJcLcUQIL%2FjykU4kBmyjxm%2FY%3Dreserved=0> fix IoTDB restart does not truncate broken ChunkGroup bug The restart will leave some dirty data in TsFile, which causes the TsFileSequenceRead to throw an exception. Therefore, release 0.12.3 is ok from me :) Thanks, — Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院
Re: Enable wiki in Github
+1 Thanks, --- Houliang Qi BONC, Ltd On 09/23/2021 09:47，Chao Wang wrote： +1 Thanks! Chao Wang BONC ltd ccgow...@163.com On 9/23/2021 09:46，Haiming Zhu wrote： Hi, +1 :) Best, Haiming Zhu School of Software, Tsinghua University 朱海铭 清华大学 软件学院 钟文豪 于2021年9月23日周四 上午9:45写道： Hi, +1 Thanks.zhongwenhao At 2021-09-21 21:26:21, "Xiangdong Huang" wrote: Hi, Using Github Wiki can indeed make it easy to update documents. If we move the wiki from confluence to Github wiki, we still need to sync the wiki documents in some place of ASF. As Github wiki is also a git codebase (https://github.com/YOUR_USERNAME/YOUR_REPOSITORY.wiki.git), if we enable it for iotdb (i.e., github.com/apache/iotdb.wiki.git), will ASF have a corresponding git codebase automatically? Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Willem Jiang 于2021年9月20日周一 上午10:12写道： Here is one concern we need to address. Although we are using Github as our daily development platform, ASF is still syncing the data to ASF infrastructure through email forwarding or git sync. In this way we still have a chance to commit the code through ASF infra. If we move the wiki from confluence to Github wiki, we still need to sync the wiki documents in some place of ASF. Willem Jiang Twitter: willemjiang Weibo: 姜宁willem On Sun, Sep 19, 2021 at 4:55 PM Jialin Qiao wrote: Hi, We have tried a lot about how to manage our documents (UserGuide, SystemDesign, ContributeGuide...). First, we manage these docs as many .md files in our repo, however, update it is troublesome, need to submit a pr and the website update is not in time. Next, we try the confluence, However, the editor of confluence is not friendly. md is good. Therefore, we try to use Github issues to manage these things. E.g., join the community https://github.com/apache/iotdb/issues/1995. Finally, we realize that GitHub wiki may be the best place to manage such documents, it is the nearest to our code, users do not need to go to another place (all in Github). As for the website, we could read the .md files from GitHub wiki. What do you think? If everyone recognized it, I will submit an issue to the INFRA. Thanks, — Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院
Re: [DISCUSS] ready for release v0.12.3?
+1 Thanks, --- Houliang Qi BONC, Ltd On 09/16/2021 11:54，HW-Chao Wang<576749...@qq.com.INVALID> wrote： +1 ---Original--- From: "Jialin Qiao"
Re: [DISCUSS] anniversary celebration of Apache IoTDB to TLP
+1 for T-shirt or Moon cake, And I think we should reserve some money for our future activities, such as meetup, some competitions, etc Thanks, --- Houliang Qi BONC, Ltd On 08/28/2021 03:57，Simon Mayer wrote： +1 Holen Sie sich Outlook für Android<https://aka.ms/AAb9ysg> From: Julian Feinauer Sent: Friday, August 27, 2021 9:42:40 PM To: dev@iotdb.apache.org Subject: Re: [DISCUSS] anniversary celebration of Apache IoTDB to TLP Hey, First of big thanks to Prof Wang for this gift to the project! I would really love to have an IoTDB shirt (i have some db shirts already but none from the coolest tsdb...). And we should send one to Andy Pavlo then as well as he collects shirts from Databases. I think it would be cool to install some kind of scholarship with the money like googles summer of code to motivate students to work on the project. Just my ideas :) But i love swag as well (cups, stickers, hoodies, shirts,...) Julian Holen Sie sich Outlook für Android<https://aka.ms/ghei36> From: Xiangdong Huang Sent: Friday, August 27, 2021 7:38:49 AM To: dev Subject: Re: [DISCUSS] anniversary celebration of Apache IoTDB to TLP Hi, I think we can create and share a document. Anyone can add contributors (if they can provide at least one URL to show the contribution). Once getting a name list, we can start voting to know the quota. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Jialin Qiao 于2021年8月26日周四 上午11:41写道： Hi, Excellent! Apart from gifts such as T-shirt and moon cake, we could discuss about how to assign the quota. As this is the first year we do this, we could consider all contributors (without time limit) to IoTDB this time. Next time we could limit the contribution from now on. If we accept this precondition, then; - how to define the contributors. We could first collect the candidates. (1) All contributors who submitted code to IoTDB statisticsed by github are added by default. (2) Contributors that attend lectures, meetups to introduce IoTDB. (3) Contributors that help organize meetups of IoTDB. Then, we could get a list of contributors. The next question is: should we list their contribution? When nominating, we could list contributions in brief. When voting, I prefer only to give a list of names, which may encourage contributors to take part in the mail list to share their activities. - how to assign the quota. Using voting software is ok, whether anonymous or not is fine from my side. Thanks, — Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院 Chao Wang 于2021年8月25日周三 上午9:10写道： Wow! Thanks for the donation of Prof. Jianmin Wang. The products around the community are very good, like Mid-Autumn moon cake. Thanks! Chao Wang BONC ltd ccgow...@163.com On 8/24/2021 21:06，Xiangdong Huang wrote： Hi Willem, Thanks. Yes, it is a good suggestion, I think we can do it in parallel. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Willem Jiang 于2021年8月24日周二 下午9:04写道： Wo, thanks for the donation of Prof. Jianmin Wang. I think it could be meaningful if we can send a T-shirt of IoTDB with the logo of the anniversary celebration to the contributor instead of just send out the money. In this way we can get touch with all kinds of contributor. Just my 2 cents. Willem Jiang Twitter: willemjiang Weibo: 姜宁willem On Tue, Aug 24, 2021 at 8:32 PM Xiangdong Huang wrote: Hi all, IoTDB graduated from the incubator last year. And next month, it is the first anniversary of Apache IoTDB (Apache announced IoTDB's graduation on 23th, Sep, 2020). This year, we have: - 1277 discussions on the mailing list. - 740 Jira issues - 272 Github issues - 1767 pull requests - we enabled Github Discussion, created Apache IoTDB Slack, etc.. - 3 major versions are released In total, we have: - 138 contributors that contributing codes (statistics by Github) - 24 PMCs and 41 Apache committers - GitHub starts: 1.5k We can see the community grows larger and larger. The achievement belongs to everyone in the community, including the committers, but also all users, evangelists, and mentors, etc.. Thank everyone for your contributions. As you know, Apache IoTDB is initially donated from Tsinghua University in 2018. After a short discussion with Prof. Jianmin Wang, the dean of the School of Software, Tsinghua University, he decides to donate 100K Chinese Yuans to the contributors in the community, as the gift of Apache IoTDB's anniversary celebration. Everyone who is in the community can get a part of the money (With regard to how to pay people around the world, let's put it aside now). Maybe it is not too much, but I think it will be very meaningful. So, I'd like to start a discussion, to decide, - how to define the contrib
Re: discuss why we have to replace hostname to ip in the cluster module
Hi, Xiangdong > I do not think we will abandon using IP. Either using IP or hostname depends > on users. If they are sure they have a good DNS, they can use DNS; If they > like IP or the nodes do not set the DNS, they can use IP. If they like, they > can also mix-use the hostname and IP addresses. We do not need to care….. As mentioned above, since users can use either IP or hostname, I don't realize what the focus of this discussion is. I think the focus is on what the user manual is exposed to the user? What are the functions provided? As for the implementation of the code, I think it is OK to use hostname or IP. In the k8s cluster, the hostname of some pods is also changed. If we only replace IP with hostname, it will not make our iotdb run perfectly on k8s. Therefore, I think it is not important to use hostname or IP. The important thing is to realize the function of automatic service maintenance that can be deployed on k8s. Thanks, --- Houliang Qi BONC, Ltd On 08/26/2021 18:29，Xiangdong Huang wrote： Hi, DNS request is an additional cost in nodes communication. I do not think we will abandon using IP. Either using IP or hostname depends on users. If they are sure they have a good DNS, they can use DNS; If they like IP or the nodes do not set the DNS, they can use IP. If they like, they can also mix-use the hostname and IP addresses. We do not need to care. We should make the relationship between node IP, node hostname and node identifier be clear. IMO, node identifier is the only option. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Eric Pai 于2021年8月26日周四 下午4:46写道： Hi, Xiangdong, Supporting hostname is really a good feature! However, we should consider these cons carefully: 1. DNS request is an additional cost in nodes communication. In cloud environment, the latency may be as large as hundreds of milliseconds(e.g. across available regions), and local DNS cache is also an issue in some extreme scenarios. 2. We should make the relationship between node IP, node hostname and node identifier be clear. A node should not be treated as a new one if it changes its IP or hostname. Or allowing to change IP only. 在 2021/8/25 下午11:15，“Xiangdong Huang” 写入: Hi, Today Julian and I discussed some logic of the cluster module. Both of us feel it strange that we replace the hostname to IP before a cluster node starts up. I think it is not a very common way. hostname sometimes is very very helpful. For example, in K8S environment, a node's hostname never changes, but its IP may change. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院
Re:All thread pools are monitored and DO NOT USE Executors.newXXX() ANYMORE
Decent work! The use of threads is also messy in the iotdb-cluster module. We can continue to optimize based on this pr. However, this requires code submitters and reviewers to ensure do not use `Executors.newXXX()` FUNCTIONS ANY MORE IN THE FUTURE. It would be better if a code detection tool could do this, and I think this rule can be added to the PR template, so that the PR submitter can see this rule through self inspection. Thanks, --- Houliang Qi BONC, Ltd On 08/11/2021 00:37，Xiangdong Huang wrote： Hi, We use many thread pools in IoTDB and it is hard to know how many thread pool we have totally and whether a thread pool is busy. Therefore, I unify the creation of ThreadPool, register it to JMX and allow JMX to check its active size, pending size, etc.. When the threadpool is `shutdown()` or `shutdownNow()`, the Pool will be deregistered from JMX. Only The server module has been upgrade to such kind of ThreadPool in rel/0.12 branch. NOTICE, Once this PR is merged, in the server module, DO NOT USE `Executors.newXXX()` FUNCTIONS ANY MORE IN THE FUTURE. For all committers, PLEASE avoid codes like `Executors.newXXX()` when you review PRs in the future. I will do the same thing in the Master branch and replace all thread pools in the cluster module using this kind of thread pool. see PR [1]. [1] https://github.com/apache/iotdb/pull/3717 Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院
Re: About enabling Github Discussion
+1 Thanks, --- Houliang Qi BONC, Ltd On 08/9/2021 09:19，李萌 wrote： Good idea, but we need to see if it meets the apache regulations first I think. On 2021/08/07 02:43:35, Jialin Qiao wrote: Hi, Github Discussion is a forum for the community to collaborate around open source projects [1]. You might consider it as a replacement of us...@iotdb.apache.org. I noticed Apache Couchdb has been already using it [2] I'd like to apply a Github discussion for the IoTDB community, it could be a replacement for slack, WeChat group, QQ group... what do you think? [1] https://github.blog/2020-05-06-new-from-satellite-2020-github-codespaces-github-discussions-securing-code-in-private-repositories-and-more/ [2] https://github.com/apache/couchdb/discussions Thanks, — Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院
Re: [NOTICE] Apache IoTDB 0.12.1 and 0.11.4 may loss data when enable cross space compaction
Thanks Lingzhe for propose this problem. Data correctness is indeed a very important issue. If the accuracy of data can not be guaranteed, the impact on the business is unknown. In the previous versions, we lost data several times before, which also exposed the defects of our test. Bugs are inevitable, but if there is data loss? Can we have other plans to get the data back? Thanks, --- Houliang Qi BONC, Ltd On 07/9/2021 09:53，HW-Chao Wang<576749...@qq.com.INVALID> wrote： +1，loss data is a blocker bug,we should release new version for it. we should adequate testing before release…… ---Original--- From: "Lingzhe ZHANG"https://github.com/apache/iotdb/pull/3515 [2] https://github.com/apache/iotdb/pull/3516 Best, --- Lingzhe Zhang School of Software, Tsinghua University 张凌哲 清华大学 软件学院
Re: Move spotless check into a separate task
+1 Thanks, --- Houliang Qi BONC, Ltd On 06/17/2021 17:22，Haonan Hou wrote： Hi, +1 Thanks, Haonan On Jun 17, 2021, at 5:15 PM, JT wrote: Greetings everyone: I found it very annoying if I forgot to use spotless to unify the format, then all tests would fail directly and that CI run would be totally useless and a waste of time, as I could not see whether my changes would fix some tests or not. So I suggest to move the spotless check into as a separate CI task, so even if I have to correct the format, I can still use this run to check if the tests are fixed. Best Tian Jiang
Re: [DISCUSS] Share a strange case: 6667 port may be considered as IRC port
Hi， +1 for change the default port. And thanks for Chao post the port list which may have potential problems. Thanks, --- Houliang Qi BONC, Ltd On 06/10/2021 20:20，Chao Wang wrote： Hi， change a port? Maybe we could avoid choosing the port in https://web.mit.edu/rhel-doc/4/RH-DOCS/rhel-sg-en-4/ch-ports.html. Thanks! Chao Wang BONC ltd ccgow...@163.com On 6/10/2021 19:32，Haonan Hou wrote： Hi all, Recently we occur a strange case, a user can not connect to an IoTDB server, but when he changes his network to a clean environment, everything is fine. It may be caused that 6667 port is considered as IRC port. The issue is here. [1] Please share any idea about it. [1] https://github.com/apache/iotdb/issues/3386 Best, Haonan Hou Sent from my iPhone
Re: ApacheCon & iotDB
Hi Trevor, Exciting news! Good luck! Thanks, --- Houliang Qi BONC, Ltd On 06/8/2021 07:58，Xiangdong Huang wrote： Hi Trevor, Look forward! Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Trevor Hart 于2021年6月8日周二 上午4:19写道： Good news everyone, I put in a submission for ApacheCon in the IOT stream which has been accepted. I will be presenting how I moved an application from using a traditional RDBMS to IotDB. Very excited! Thanks Trevor Hart
Re:new committer: Steve Yurong Su
Congratulations Yurong! Thanks, --- Houliang Qi BONC, Ltd On 05/12/2021 20:35，Jialin Qiao wrote： Hi all, The Project Management Committee (PMC) for Apache IoTDB has invited Steve Yurong Su to become a committer and we are pleased to announce that he has accepted. Steve brings UDF and Trigger to IoTDB, which is a fantastic function and well designed! Welcome Steve! Yours, The Apache IoTDB PMC
Re:[VOTE] Apache IoTDB 0.12.0 RC1 release
Hi, +1 from committer But only one issue not sure, the org.checkerframework:checker-qual:2.0.0 have two licenses: GPL2.0 and MIT, I not sure apache 2.0 is compatible with it or not. I checked the following operations on macOS 10.15.4 and jdk 1.8 The source release: Check third party dependencies [NOT SURE] LICENSE and NOTICE [ok] signatures and hashes [ok] All files have ASF header [ok] could compile from source: ./mvnw.sh clean install [ok] The binary distribution: LICENSE and NOTICE [ok] signatures and hashes [ok] Could run with the following statements [ok] SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); select * from root; Thanks, --- Houliang Qi BONC, Ltd On 04/6/2021 13:33，Xiangdong Huang wrote： Hi all, Big news! 0.12.0 RC1 is ready. Apache IoTDB 0.12.0 introduces many new features, like the new TsFile structure (version: 02), the User-Define-Function, and the cluster module. Welcome to validate the release and have a try. Form 0.12.0 on, we separate the binary release file into several: the server, the cluster, the cli, the cpp-client-lib, the python (through Pypi), the grafana-connector, and all-in-one. Other connectors, like hive, hadoop, spark and zeppelin connectors are not released as independent binary files. You can get the main changes from [5]. Apache IoTDB 0.12.0 has been staged under [2] and it’s time to vote on accepting it for release. All Maven artifacts are available under [1]. Voting will be open for 72hr. A minimum of 3 binding +1 votes and more binding +1 than binding -1 are required to pass. Release tag: v0.12.0 Hash for the release tag: 06adff7fad83c324aaeb90160766ecfd0bd2b84a Before voting +1, PMC members are required to download the signed source code package, compile it as provided, and test the resulting executable on their own platform, along with also verifying that the package meets the requirements of the ASF policy on releases.[3] You can achieve the above by following [4]. [ ] +1 accept (indicate what you validated) [ ] -1 reject (explanation required) [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1059 [2] https://dist.apache.org/repos/dist/dev/iotdb/0.12.0/rc1 [3] https://www.apache.org/dev/release.html#approving-a-release [4] https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release [5] https://dist.apache.org/repos/dist/dev/iotdb/0.12.0/rc1/RELEASE_NOTES.md [6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院
Re: start to release v0.11.3 RC2 and v0.12 RC1
Hi, all The following are the release notes I have compiled for 0.12.0. If there is any omission or errors, please feel free to continue to add or correct~ ## New Features [IOTDB-68] New shared-nothing cluster [IOTDB-507] Add zeppelin-interpreter module [IOTDB-825] Aggregation by natural month [IOTDB-890] SDT implementation [IOTDB-944] Support UDTF (User-defined Timeseries Generating Function) [IOTDB-965] Add timeout in query [IOTDB-1076] Create interface of TimeIndex [IOTDB-1077] Add insertOneDeviceRecords API in java session [IOTDB-1077] Add python test and insert one device interface [IOTDB-1055] Support data compression type GZIP [IOTDB-1024] Support multiple aggregated measurements for group by level statement [IOTDB-1276] Add explain sql support and remove debug_state parameter [IOTDB-1197] Add iotdb-client-go as a git submodule of IoTDB repo [IOTDB-1230] Support spans multi time partitions when loading one TsFile [IOTDB-1273] Feature/restrucutre python module as well as supporting pandas dataframe Add level merge to "merge" command ## Incompatible changes [IOTDB-1081] New TsFile Format [ISSUE-2730] Add the number of unseq merge times in TsFile name. ## Miscellaneous changes [IOTDB-868] Change mlog from txt to bin [IOTDB-1069] Restrict the flushing memtable number to avoid OOM when mem_control is disabled [IOTDB-1070] Add interface terminate for UDTF [IOTDB-1071] Built-in UDF registration service [IOTDB-1074] Add interface getDataType for UDFParameters [IOTDB-1098] Add interface validate for UDF [IOTDB-1104] Refactor the error handling process of query exceptions [IOTDB-1108] Add error log to print file name while error happened [IOTDB-1113] Optimize the execution efficiency of UDF [IOTDB-1152] Optimize regular data size in traversing [IOTDB-1180] Reset the system log file names and maximal disk-space size [ISSUE-2515] Set fetchsize through JDBC and Session [ISSUE-2598] Throw explicit exception when time series is unknown in where clause New distribution structure for v0.12 ## Bug Fixes [IOTDB-1049] Fix NullpointerException and a delete bug in Last query [IOTDB-1050] Fix Count timeserise column name is wrong [IOTDB-1068] Fix Time series metadata cache bug [IOTDB-1084] Fix temporary memory of flushing may cause OOM [IOTDB-1106] Fix delete timeseries bug [IOTDB-1126] Fix the unseq tsfile delete due to merge [IOTDB-1135] Fix the count timeseries prefix path bug [IOTDB-1137] Fix MNode.getLeafCount error when existing sub-device [ISSUE-2484] Fix creating timeseries error by using "create" or "insert" statement [ISSUE-2545, 2549] Fix unseq merge end time bug [ISSUE-2611] An unsequence file that covers too many sequence file causes OOM query [ISSUE-2688] LRULinkedHashMap does not work as an LRU Cache [ISSUE-2709,IOTDB-1178] Fix cache not cleared after unseq compaction bug, Fix windows 70,10 ci bug in unseq compaction ci [ISSUE-2741] getObject method in JDBC should return an Object [ISSUE-2746] Fix data overlapped bug after unseq compaction [ISSUE-2758] NullPointerException in QueryTimeManager.checkQueryAlive() [ISSUE-2905] Fix Files.deleteIfExists() doesn't work for HDFS file [ISSUE-2919] Fix C++ client memory leak bug Fix importCSVTool import directory bug & encode bug Fix import csv which can't import time format str Fix dead lock between deleting data and querying in parallel Fix sync bug for tsfiles's directory changed by vitural storage group Fix incompatible max_degree_of_index_node parameter Thanks, ------- Houliang Qi BONC, Ltd On 04/2/2021 11:37，Jialin Qiao wrote： Hi, Could someone collect the release notes of 0.12.0? :) Thanks, — Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院 小珈蓝 <282583...@qq.com> 于2021年4月1日周四 下午3:47写道： +1, Looking forward to the release of v0.12. CloudWise-Lukemiao -- 原始邮件 -- 发件人: "dev" < saint...@gmail.com; 发送时间:2021年4月1日(星期四) 下午2:05 收件人:"dev"
Re: start to release v0.11.3 RC2 and v0.12 RC1
Hi, all The following are the release notes I have compiled. If there is any omission or errors, please feel free to continue to add or correct~ ## New Features [IOTDB-68] New shared-nothing cluster [IOTDB-507] Add zeppelin-interpreter module [IOTDB-825] Aggregation by natural month [IOTDB-890] SDT implementation [IOTDB-944] Support UDTF (User-defined Timeseries Generating Function) [IOTDB-965] Add timeout in query [IOTDB-1076] Create interface of TimeIndex [IOTDB-1077] Add insertOneDeviceRecords API in java session [IOTDB-1077] Add python test and insert one device interface [IOTDB-1055] Support data compression type GZIP [IOTDB-1024] Support multiple aggregated measurements for group by level statement [IOTDB-1276] Add explain sql support and remove debug_state parameter [IOTDB-1197] Add iotdb-client-go as a git submodule of IoTDB repo [IOTDB-1230] Support spans multi time partitions when loading one TsFile [IOTDB-1273] Feature/restrucutre python module as well as supporting pandas dataframe Add level merge to "merge" command ## Incompatible changes [IOTDB-1081] New TsFile Format [ISSUE-2730] Add the number of unseq merge times in TsFile name. ## Miscellaneous changes [IOTDB-868] Change mlog from txt to bin [IOTDB-1069] Restrict the flushing memtable number to avoid OOM when mem_control is disabled [IOTDB-1070] Add interface terminate for UDTF [IOTDB-1071] Built-in UDF registration service [IOTDB-1074] Add interface getDataType for UDFParameters [IOTDB-1098] Add interface validate for UDF [IOTDB-1104] Refactor the error handling process of query exceptions [IOTDB-1108] Add error log to print file name while error happened [IOTDB-1113] Optimize the execution efficiency of UDF [IOTDB-1152] Optimize regular data size in traversing [IOTDB-1180] Reset the system log file names and maximal disk-space size [ISSUE-2515] Set fetchsize through JDBC and Session [ISSUE-2598] Throw explicit exception when time series is unknown in where clause New distribution structure for v0.12 ## Bug Fixes [IOTDB-1049] Fix NullpointerException and a delete bug in Last query [IOTDB-1050] Fix Count timeserise column name is wrong [IOTDB-1068] Fix Time series metadata cache bug [IOTDB-1084] Fix temporary memory of flushing may cause OOM [IOTDB-1106] Fix delete timeseries bug [IOTDB-1126] Fix the unseq tsfile delete due to merge [IOTDB-1135] Fix the count timeseries prefix path bug [IOTDB-1137] Fix MNode.getLeafCount error when existing sub-device [ISSUE-2484] Fix creating timeseries error by using "create" or "insert" statement [ISSUE-2545, 2549] Fix unseq merge end time bug [ISSUE-2611] An unsequence file that covers too many sequence file causes OOM query [ISSUE-2688] LRULinkedHashMap does not work as an LRU Cache [ISSUE-2709,IOTDB-1178] Fix cache not cleared after unseq compaction bug, Fix windows 70,10 ci bug in unseq compaction ci [ISSUE-2741] getObject method in JDBC should return an Object [ISSUE-2746] Fix data overlapped bug after unseq compaction [ISSUE-2758] NullPointerException in QueryTimeManager.checkQueryAlive() [ISSUE-2905] Fix Files.deleteIfExists() doesn't work for HDFS file [ISSUE-2919] Fix C++ client memory leak bug Fix importCSVTool import directory bug & encode bug Fix import csv which can't import time format str Fix dead lock between deleting data and querying in parallel Fix sync bug for tsfiles's directory changed by vitural storage group Fix incompatible max_degree_of_index_node parameter Thanks, ------- Houliang Qi BONC, Ltd On 04/2/2021 11:37，Jialin Qiao wrote： Hi, Could someone collect the release notes of 0.12.0? :) Thanks, — Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院 小珈蓝 <282583...@qq.com> 于2021年4月1日周四 下午3:47写道： +1, Looking forward to the release of v0.12. CloudWise-Lukemiao -- 原始邮件 -- 发件人: "dev" < saint...@gmail.com; 发送时间:2021年4月1日(星期四) 下午2:05 收件人:"dev"
Re: start to release v0.11.3 RC2 and v0.12 RC1
+1, Looking forward to the release of v0.12. Thanks, --- Houliang Qi BONC, Ltd On 04/1/2021 15:03??Yuxiang Song<1063877...@qq.com> wrote?? +1?? look forward to the release of v0.12. ---- ??: ""
Re: comment out the unused configuration items for users
+1 Thanks, --- Houliang Qi BONC, Ltd On 03/23/2021 15:01，Mark Liu wrote： +1 Best, --- Mark Liu 王超 于2021年3月23日周二 下午2:34写道： Hi, all, How do you think? Please leave your comments. Chao Wang BONC, Ltd ccgow...@163.com On 3/20/2021 10:18，王超 wrote： Currently, iotdb has many configuration items, but only a few items need to be changed by basic users, including IP, port and heap space. In addition, other basic need not be adjusted. Therefore, I suggest that we can also comment out irrelevant configuration items and give users a relatively simple and clean configuration file. At present, in the configuration files of influxdb, nginx and other systems, irrelevant configuration items are basically commented. Only when you want to change it, you need to cancel the comment and configure it separately. Chao Wang BONC, Ltd ccgow...@163.com
Re: [Announce] February’s “Contributors of The Month” and new contributors to the community
I'm very surprised and honored to receive the award. :) Let's make IoTDB better! Thanks, --- Houliang Qi BONC, Ltd On 03/12/2021 16:07，王超 wrote： Thanks for their work. It's glad to see so many new faces. Chao Wang BONC, Ltd ccgow...@163.com On 3/12/2021 15:47，Jesse Zhou wrote： Hi guys! The result of February‘s “Contributors of The Month” has been tallied(view last month’s Contributor of The Month in [1]), they are sunjincheng121(Jincheng Sun), with 132571 lines changed and 8 mails sent; HTHou(Haonan Hou), with 2116 lines changed and 4 mails sent; neuyilan(Houliang Qi), with 2692 lines changed and 3 mails sent. Thanks to them for their hard work. There are also 22 other contributors who contributed to IoTDB this month, Thanks for the commitment . In the meantime, we have seven new contributors submitted their first PR in IoTDB, They are wuzhaojie(Zhaojie Wu), WilliamSong11(Yuxiang Song), GLBB, THUMarkLau(Xuxin Liu), chenjun40, 543202718(Haoyu Wang) and jxlgzwh(Wenhao Zhong). Welcome aboard! Best, Jesse Zhou [1] https://lists.apache.org/thread.html/r100b31c73a9c83adacc54ba2e1164a0b88fff92bfa8a8b7bdb8fa7de%40%3Cdev.iotdb.apache.org%3E
Re: [DISCUSS] Move all system design documents to confluence space
Hi, I agree with that put all design doc together to one place. There is only one question to prompt: For Chinese users, visit iotdb.apache.org or cwiki.apache.org these two websites may encounter some problems. At present, users may also visit https://gitee.com/apache/iotdb (it can be seen as a mirror image of GitHub in China) this website looks at some design documents. If you put all the design documents on the conference, it may be even more inconvenient for Chinese users. Thanks, --- Houliang Qi BONC, Ltd On 02/26/2021 13:47，Jialin Qiao wrote： +1 -- Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院 -原始邮件- 发件人: "Haonan Hou" 发送时间: 2021-02-25 14:27:17 (星期四) 收件人: dev-iotdb 抄送: 主题: [DISCUSS] Move all system design documents to confluence space Hi, Currently we have two places to share the design documents. One is our site[1], another is the confluence space[2]. After discussing with Jialin, we think we’d better put all design doc together to the confluence and reorganize the structure according different modules, like TsFile, Storage Engine, etc. Also, We will link the System Design url on our website to [2]. Welcome to share any idea. Thanks! [1] https://iotdb.apache.org/SystemDesign/Architecture/Architecture.html [2] https://cwiki.apache.org/confluence/display/IOTDB/System+Design Best, Haonan Hou
Re:New committers: Xiangwei Wei and Jesse Zhou
Congratulations, Xiangwei and Jesse！ Thanks, --- Houliang Qi BONC, Ltd On 02/20/2021 18:36，Jialin Qiao wrote： Hi, The Project Management Committee (PMC) for Apache IoTDB has invited Xiangwei Wei and Jesse Zhou to become committers and we are pleased to announce that they have accepted. Being a committer enables easier contribution to the project since there is no need to go via the patch submission process. This should enable better productivity. Being a PMC member enables assistance with the management and to guide the direction of the project. You could get the write-access of our repository by: (1) Update your github id in: id.apache.org , also an email (anything to your apache email will forwart to this email). (2) Accept the invitation in github to being a member of the apache group. (3) Link your accounts via: git.apache.org (To lighten the third option MFA Status, you need to download Google Authenticator (Google 身份验证器) in your mobile) (4) Sign in github by Google Authenticator (5) Clone repository with ssh. Welcome Xiangwei and Jesse! Yours, The Apache IoTDB PMC
Re:AW: [VOTE] change release tags from release/{full-version} to v{full-version}
+1
Thanks,
---
Houliang Qi
BONC, Ltd
On 01/28/2021 18:12，Jialin Qiao wrote：
+1
--
Jialin Qiao
School of Software, Tsinghua University
乔嘉林
清华大学 软件学院
-原始邮件-
发件人: "Dawei Liu"
发送时间: 2021-01-28 18:11:24 (星期四)
收件人: "dev@iotdb.apache.org"
抄送:
主题: Re:AW: [VOTE] change release tags from release/{full-version} to
v{full-version}
+1
Dawei
On 01/28/2021 17:58，Christofer Dutz wrote：
+1 (obviously) ;-)
Chris
-Ursprüngliche Nachricht-
Von: 孙泽嵩
Gesendet: Donnerstag, 28. Januar 2021 10:47
An: dev@iotdb.apache.org
Betreff: Re: [VOTE] change release tags from release/{full-version} to
v{full-version}
+1
Best,
---
Zesong Sun
School of Software, Tsinghua University
孙泽嵩
清华大学 软件学院
2021年1月28日 11:29，Mark Liu 写道：
+1
Best,
---
Mark Liu
Yonyou Network Technology Co.,Ltd.
Xiangdong Huang 于2021年1月28日周四 上午11:26写道：
Hi,
Currently, we use release/{major}.{minor}.{patch} as our tag stale,
e.g., release/0.11.2.
However, it may bring some troubles for some languages like GoLang.
As discussed in [1] and reference [2] in PLC4x, I'd like to call a
vote to change the style to v{major}.{minor}.{patch} e.g., v0.11.2.
[ ] +1
[ ] 0
[ ] -1, disagree and explain.
If passed, we will use the style from 0.12.0 on.
[1]
https://lists.apache.org/thread.html/r8e15776a85d30d247f513e419e3544c
b995e8267ca66f06612b3f08a%40%3Cdev.iotdb.apache.org%3E
[2]
https://lists.apache.org/thread.html/rc4415bb0a75dea7b7e7b9f1ef939ab8
98181815bae00fd9900d9593f%40%3Cdev.plc4x.apache.org%3E
Best,
---
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Re:[Discussion] Only allow squash-merge when merging pr
Hi, all Big +1 Thanks, --- Houliang Qi BONC, Ltd On 01/4/2021 12:41，Xinyu Tan wrote： Hi,all In current status, IoTDB repository is accepting `Allow merge commits` in merging pr, which causes a lot useless commits. In order to make the project code management process more formal, I would like to ask a vote to discuss whether we need to disable the other merge method and only allow squash-merge when merging pr. If everyone supports it, then we can refer to other projects[1][2][3] and notify infra. My vote is +1. [1] https://issues.apache.org/jira/browse/INFRA-16146?jql=text%20~%20%22squash%20merge%22 [2] https://issues.apache.org/jira/browse/INFRA-16669?jql=text%20~%20%22squash%20merge%22 [3] https://issues.apache.org/jira/browse/INFRA-17098?jql=text%20~%20%22squash%20merge%22 Best -- Xinyu Tan
Re: Happy new year!
Hi, Happy new year, hope the IoTDB ecology will be stronger in 2021 Thanks, --- Houliang Qi BONC, Ltd On 01/1/2021 06:59，Simon Mayer wrote： 2 Minutes bevor 12 o clock in Germany happy new year. Chinese have it already but the US guys I think in 7 or 8 hours so happy new years for you cheers Holen Sie sich Outlook für Android<https://aka.ms/ghei36> From: Jialin Qiao Sent: Thursday, December 31, 2020 12:35:40 PM To: dev@iotdb.apache.org Subject: Happy new year! Hi, This year, we made a lot of progress, both in the software and the community. - There are total 91 contributors that make contributions to IoTDB. - Apache IoTDB graduated successfully and became a TLP. - We released two big versions (0.10 and 0.11). - We accomplish the new query engine, the cluster version, compaction, UDF, C++ and Go client... - We attended a lot of conference and meetup to introduce IoTDB. Thank you all and happy new year! Best, -- Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院
Re: new committer: Xinyu Tan
Congratulations Xinyu! Thanks, --- Houliang Qi BONC, Ltd On 12/21/2020 10:51，Benedict Jin wrote： Congratulations On 2020/12/20 13:37:55, "Jialin Qiao" wrote: Hi, The Project Management Committee (PMC) for Apache IoTDB has invited Xinyu to become a committer and we are pleased to announce that he has accepted. Being a committer enables easier contribution to the project since there is no need to go via the patch submission process. This should enable better productivity. You could get the write-access of our repository by: (1) Update your github id in: id.apache.org , also an email (anything to your apache email will forwart to this email). (2) Accept the invitation in github to being a member of the apache group. (3) Link your accounts via: git.apache.org (To lighten the third option MFA Status, you need to download Google Authenticator (Google 身份验证器) in your mobile) (4) Sign in github by Google Authenticator (5) Clone repository with ssh. Welcome Xinyu! Yours, The Apache IoTDB PMC
Re: new committer: Houliang Qi
Thank you all for your warm welcome and hope IoTDB will get better and better. Thanks, --- Houliang Qi BONC, Ltd On 12/18/2020 20:12，Haimei Guo wrote： Hi Houliang, Congratulations! On Fri, Dec 18, 2020 at 4:15 PM Mark Liu wrote: Hi, Congratulations! Best, --- Mark Liu 孙泽嵩 于2020年12月18日周五 下午4:10写道： Hi Houliang, Congratulations! Best, --- Zesong Sun School of Software, Tsinghua University 孙泽嵩 清华大学 软件学院 2020年12月18日 15:08，王超 写道： Congratulation！ Daiwei Liu have send message of ongratulation! On 12/18/2020 15:02，Jialin Qiao wrote： Hi, The Project Management Committee (PMC) for Apache IoTDB has invited Houliang Qi to become a committer and we are pleased to announce that he has accepted. Being a committer enables easier contribution to the project since there is no need to go via the patch submission process. This should enable better productivity. You could get the write-access of our repository by: (1) Update your github id in: id.apache.org , also an email (anything to your apache email will forwart to this email). (2) Accept the invitation in github to being a member of the apache group. (3) Link your accounts via: git.apache.org (To lighten the third option MFA Status, you need to download Google Authenticator (Google 身份验证器) in your mobile) (4) Sign in github by Google Authenticator (5) Clone repository with ssh. Welcome Houliang! The Apache IoTDB PMC
回复：iotdb 集群部署
Hi, Welcome to try the cluster-version of IoTDB. 1. Now the cluster-version of IoTDB have not released, so the previous version do not support the distributed mode. 2. if you want to try the cluster-version of IoTDB, you can build from the master branch, and the cluster/target/ path contains the cluster-0.12.0-SNAPSHOT.zip file, then you can try with it. 3. For cluster setup, please see http://iotdb.apache.org/UserGuide/Master/Server/Cluster%20Setup.html for details. Thanks, --- Houliang Qi BONC, Ltd 在2020年12月14日 11:44，凌战 写道： hi，I have some following questions ： first，I want to try the cluster setup in pseudo-distributed mode， however，when I unzip apache-iotdb-0.11.1-bin.zip，I found that there is no start-node.sh under the directory sbin，which version I should use？ second，if I set up in pseudo-distributed mode，should I copy the whole package as an instance or only need to copy some configuration files？ | | 凌战 | | m18340872...@163.com | 签名由网易邮箱大师定制
Re:[NEWS] The largest PR (in 2020) is merged
Hi, Congratulations. It’s nice to see our cluster-version take a big step forward！ > By the way, it is Jialin Qiao who clicked the "squash and merge" button on > my Mac... Not me. I feared such a large PR :D Thanks Jialin’s finger and Xinangdong’s Mac :) Thanks, --- Houliang Qi BONC, Ltd On 12/8/2020 23:47，Xiangdong Huang wrote： Hi, I'd like to announce the news that the largest PR, IoTDB's cluster module, PR#460, JIRA ISSUE 68, is merged. This PR is opened on Oct 18th, 2019. Many developers contributed to it and many developers joined to review the codes. A Chinese document/paper for the detailed design of the module is published here[4]. Finally, 57K lines of codes are added and 2K lines of codes are deleted. The modifications involve the cluster module, the server module, the JDBC module, etc... Even reviewing the codes is huge work. Except for the cluster module, I have reviewed the modifications of all other modules. So, I have high confidence to say the PR won't impact the stability and performance if you just use IoTDB in the single-node mode. For the cluster module, I think some other developers have reviewed it. But, we need more reviewers and more tests for the modules. For prudential reasons, I suggest if we release the cluster module in the next 1 to 3 (or more) versions, let's pack two release files, one is apache-iotdb and the other is apache-iotdb-cluster-alpha [2], to provide a stable single-node version and an experimental cluster version to users. I believe after more and more users trying, testing and contributing to the module, the module will be stable quickly. Then, I list some TODO tasks to wind up the issue, see [3]. Finally, congratulations, all of you. Thanks for making such big contribution to the Apache IoTDB community. By the way, it is Jialin Qiao who clicked the "squash and merge" button on my Mac... Not me. I feared such a large PR :D Enjoy it. [1] https://github.com/apache/iotdb/pull/460 [2] https://issues.apache.org/jira/browse/IOTDB-1046 [3] https://issues.apache.org/jira/browse/IOTDB-1045 [4] http://scis.scichina.com/cn/2020/SSI-2019-0189.pdf Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院
Re: [VOTE] A new repository for Go client
+1 Thanks, --- Houliang Qi BONC, Ltd On 11/30/2020 20:51，Mark Liu wrote： +1 Best, --- Mark Liu Yonyou Network Technology Co.,Ltd. Jialin Qiao 于2020年11月30日周一 下午8:25写道： Hi, As discussed in [1], A new repository (iotdb-client-go) is better for maintaining the go client. So I'd like to call for a vote. Please vote accordingly: [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove with the reason The vote is open for the next 72 hours. [1] https://lists.apache.org/thread.html/r590437ed0edfb74512d389f2de8b5f73698304be1f97a09847a9693e%40%3Cdev.iotdb.apache.org%3E Thanks, -- Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院
Re: The design doc of [IOTDB-965] add timeout in query
Hi, +1 for Tian Jiang ideas, And I think we can extend the session api to add one parameter to indicate the operation timeouts(insert、query and so on), besides we can implement the `setQueryTimeout()` method in JDBC. Thanks, --- Houliang Qi BONC, Ltd On 11/27/2020 09:55，Tian Jiang wrote： Greetings, This is important indeed, but I have some suggestions: 1. This would best be based query cancellation, i.e., a query can be cancelled by a given interface, and the timed thread need only to call this interface to time out a query when needed. The merit of doing so is better abstraction and then we can readily implement `cancel()` in JDBC. 2. Query time out should be defined at query level, or at least session level. Users may often be running different types of queries at the same time, like short monitoring queries (milliseconds to seconds) and long analytic queries (hours to days), defining only one global timeout is seldom enough for complex applications. Best Tian Jiang On 11/26/2020 21:32，Xiangwei Wei wrote： If a query takes too much time, we'd better interrupt this query instead of hanging up the client. The design doc is in [1]. Any advice is appreciated :D A discussion is: How long can the query be considered as time out? maybe 5min, 10min? We need to set a default value, of course it can be modified by the user. [1] https://cwiki.apache.org/confluence/display/IOTDB/Query+over+time+design+doc -- Best, Xiangwei Wei
Re: Some thoughts on Golang client
Hi,
Good work!
But I have doubts about whether we can create a new project under apache.
Besides I have the following two ideas:
1. Upload the code directly to the https://github.com/apache/iotdb/go-client,
at the same time, in order to reduce some useless code generated by thrift , we
can put the go file generated by thrift into another warehouse. like
https://github.com/iotdb/iotdb-client/go-client;
2. Or we just upload the all go client codes to
https://github.com/iotdb/iotdb-client/go-client, and the other clients like
python、c++ can be put here to.
Thanks,
---
Houliang Qi
BONC, Ltd
On 11/20/2020 11:30，Xiangwei Wei wrote：
Hi,
Nice work!!
I think we can create an official iotdb golang client repository,
for example: https://github.com/apache/go-iotdb
+1 for that. It will be very helpful for golang users of IoTDB.
曼灵格 于2020年11月20日周五 上午11:22写道：
Hi,
I have developed a golang client for iotdb and placed it on the
https://github.com/manlge/go-iotdb.
Used some source code from
https://github.com/yanhongwangg/incubator-iotdb/tree/client-go
For details, see:
https://github.com/manlge/go-iotdb/blob/main/README.md
I think we can create an official iotdb golang client repository,
for example: https://github.com/apache/go-iotdb
get get github.com/apache/go-iotdb
package main
import (
"github.com/apache/go-iotdb/client"
)
var session client.Session
func main() {
session = client.NewSession("127.0.0.1", "6667")
session.Open(false, 0)
defer session.Close()
//do something
}
Best,
---
Mark Liu
Yonyou Network Technology Co.,Ltd.
--
Best,
Xiangwei Wei
Re:Iotdb integrated Prometheus
Nice work! This will enrich the ecology of IoTDB. Thanks, --- Houliang Qi On 11/6/2020 17:45<1101967...@qq.com> wrote?? Hi,all The following is the design of iotdb integrated Prometheus. https://cwiki.apache.org/confluence/display/IOTDB/Iotdb+integrated+Prometheus As mentioned inhttps://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-519?filter=allopenissues, IoTDB is a highly efficient time series database.Prometheus is a monitoring and alerting toolkit, which supports collecting data from other systems, servers, and IoT devices, saving data into a DB, visualizing data and provides some query APIs. Prometheus allows users to use their database rather than just Prometheus DB for storing time series databases.So we should integrate Prometheus, so that we can provide the function of collecting data. If you have any comments??looking forward to your reply. Thanks, Yanhong Wang
Re:I've submitted a PR for issue IOTDB-972
Hi Rongzhao, Good job, I think you can fix the bug that Yanhong created on the issue[1] together with this PR, it may need to change the code in Cli module. Besides the commit id before[2] can run correctly, so this may be newly introduced. [1] https://issues.apache.org/jira/browse/IOTDB-973 [2] df8f876a0172b48f38f96f94319ed9e2ec2bfd7b Thanks, --- Houliang Qi On 10/30/2020 17:20<1101967...@qq.com> wrote?? Hi all, Good job! But there is still some problems using command line executing start-cli.sh -e ie. start-cli.sh -e 'show storage group' I submitted this bug in jira, please check: https://issues.apache.org/jira/browse/IOTDB-973 Thank you, Yanhong Wang ---- ??: "dev" <532741...@qq.com; :2020??10??30??(??) 4:05 ??:"dev"https://github.com/apache/iotdb/pull/1906]. I used code that .sh couldn't recognize. Now I change the start-cli.sh file into sh version Thanks, --- Rongzhao Chen School of Software, Tsinghua University
Add raft log persist mechanism and use persist log to catch up
Hi all, The current version of CommittedLogManager only holds a small number of logs, as it only uses the persistent storage (namely the StableEntryManager) as a measure of recovery, not to extend its storage ability. The result is that when one follower is down, its difference from the leader will soon exceed the capability of the CommittedLogManager, which further results in a snapshot catch-up, and it is time-consuming. The idea is to merge StableEntryManager into CommittedLogManager, all committed logs should go to the persistent storage (unless persistency is disabled), and only the newest part of the logs will stay in memory. This pr[1] is intend to solve the problems, the design documents are listed here[2], any comments are welcome. [1] https://github.com/apache/iotdb/pull/1865 [2] https://cwiki.apache.org/confluence/display/IOTDB/Add+raft+log+persist+mechanism+and+use+persist+log+to+catch+up Thanks, --- Houliang Qi
[DISCUSS] Merge the cluster branch code to master branch
Hi all, As we discussed before[1], it is better to merge the cluster branch with the master branch. We have also done some tests, including the following two aspects: The performance of the server on the cluster branch and the performance of the master branch are tested; Distributed functional regression testing The test results[2] show that, the branch code on the cluster will not have a bad impact on the server performance. At the same time, all 364 test cases of distributed regression test passed. I think we can merge the cluster branch into the master branch. Any ideas? [1] https://lists.apache.org/thread.html/r37e0b606d5a5652493a6cd64b07b02dc16fae7fbae3e1a5827e75343%40%3Cdev.iotdb.apache.org%3E [2] https://cwiki.apache.org/confluence/display/IOTDB/2020-10-29 Thanks, --- Houliang Qi
Re:[Discuss] Add Latest File Compaction Strategy
Hi LingZhe, Good idea, A small question, how to define the latest files? 10 minutes, 1 hour, or the last day? I think this parameter is best left to the user to configure. Thanks, --- Houliang Qi On 10/21/2020 19:13，Dawei Liu wrote： Hi, Big +1. We have this usage scenario where we want to read and take out data within an hour in the quickest possible time, beyond which the data becomes worthless. Best, Dawei On 10/21/2020 19:08，Lingzhe ZHANG wrote： Hi all, Current compaction strategy just merge old file. When user write too fast, the compaction can just merge useless data. In this condition, the compaction can just cause bad effect. So we have to develop a compaction strategy that can merge latest file as soon as possible, and use it as default startegy. I will realize it[1] later. What do you think of it? [1] https://issues.apache.org/jira/browse/IOTDB-952 Best, - LingZhe ZHANG School of Software, Tsinghua University
Re: [DISCUSS]The cluster version management
Hi all， It is indeed easier to only maintain the master branch, but in the future, it may still encounter the current situation: the release of the cluster will be affected by the newly submitted code of the server. As far as the current situation, I support checkout out a cluster_release branch first and do not need to wait all the new server functions to ready, which is also convenient for quickly collecting user feedback. Finally, the master next release version would be 0.11.0, I hope we can release the cluster version with it. Thanks, --- Houliang Qi On 10/13/2020 10:12，Tian Jiang wrote： My suggestion would be: First, we will decide some tests which must be passed before cluster releases. Then, after passing such tests, we merge `cluster_new` into `master`. Next, any new PRs to `master`, related to distributed logic or not, must also pass these tests. Finally, hopefully, the cluster version will released along with 0.12.0. Tian Jiang On 10/13/2020 09:54，runhus...@foxmail.com wrote： Hi， I think that only maintaining master branch is also good, but how to release first cluster version? The cluster branch is always influenced by master branch now. May we need to checkout a new branch to release the first version? Thanks. Chao Wang BONC Ltd From: Jialin Qiao Date: 2020-10-12 20:38 To: dev Subject: Re: [DISCUSS]The cluster version management Hi, I suggest starting to merge the cluster_new branch to master and maintaining the cluster as a module. Maintaining two branches is hard... Thanks, -- Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院 -原始邮件- 发件人: "runhus...@foxmail.com" 发送时间: 2020-10-12 16:19:30 (星期一) 收件人: dev 抄送: 主题: Re: Re: [DISCUSS]The cluster version management Hi Houliang, Good idea! I also think we need to accelerate the development of cluster version. Now the cluster release branch is always update with master branch which is not stable. We need to force more on stability than feature, and we should add switch for new feature to be disabled util we have tested it. Like Tian Jiang says, maybe we will release cluster together with master later, but for the first release we need a cluster_develop and cluster_release branch. We should stop merge feature to the cluster_release branch when we decide to release. Thanks. Chao Wang BONC Ltd From: Tian Jiang Date: 2020-10-12 15:43 To: dev Subject: Re: [DISCUSS]The cluster version management This is an interesting suggestion. I thought that once `cluster_new` is merged into `master`, there would be no more `cluster_new`. Cluster will be a normal module just like jdbc or CLI, and developments should be based on `master` then, instead of a branch separately for the cluster version. And I doubt if we will release cluster version separately, since it may come naturally together with the releases from `master`. Best, Tian Jiang On 10/12/2020 15:28，Haimei Guo wrote： Looks good! On Mon, Oct 12, 2020 at 12:23 PM Houliang Qi wrote: Hi all, I’d like to start a discussion about the cluster version management: As someone mentioned that there should be a develop branch and a release branch[1]. The develop branch is used to submit the latest development, and the release branch is used for the functions that need to be released in the latest release. At the same time, I hope that the development of cluster can also have two branches: cluster_develop branch and cluster_release branch. The cluster_develop branch is used to merge the stand-alone version of the develop branch code and the latest development of the cluster. The cluster_release branch is used to release some of the latest features. Only the functions that need to be released are allowed to be merged into the cluster_release branch, or to fix some bugs. Other newly developed functions are not allowed to be merged into the cluster_release branch. After cluster_release has been fully tested, it can be released. Regarding the latest release, I would like to check out a cluster_release branch after the cluster_premerge branch merges into the master (develop), and then the master branch merges into the cluster_new (cluster_develop) branch. And I think the new functions do not have beed tested or need more than one month to tested should be switch off when release the cluster version. Does anyone have some ideas about this? [1] https://lists.apache.org/thread.html/rf7dce8d4cfcf4001feeba139cc897d6b40a1741e06ef87aabd56d8c9%40%3Cdev.iotdb.apache.org%3E Thanks, --- Houliang Qi
[DISCUSS]The cluster version management
Hi all, I’d like to start a discussion about the cluster version management: As someone mentioned that there should be a develop branch and a release branch[1]. The develop branch is used to submit the latest development, and the release branch is used for the functions that need to be released in the latest release. At the same time, I hope that the development of cluster can also have two branches: cluster_develop branch and cluster_release branch. The cluster_develop branch is used to merge the stand-alone version of the develop branch code and the latest development of the cluster. The cluster_release branch is used to release some of the latest features. Only the functions that need to be released are allowed to be merged into the cluster_release branch, or to fix some bugs. Other newly developed functions are not allowed to be merged into the cluster_release branch. After cluster_release has been fully tested, it can be released. Regarding the latest release, I would like to check out a cluster_release branch after the cluster_premerge branch merges into the master (develop), and then the master branch merges into the cluster_new (cluster_develop) branch. And I think the new functions do not have beed tested or need more than one month to tested should be switch off when release the cluster version. Does anyone have some ideas about this? [1] https://lists.apache.org/thread.html/rf7dce8d4cfcf4001feeba139cc897d6b40a1741e06ef87aabd56d8c9%40%3Cdev.iotdb.apache.org%3E Thanks, --- Houliang Qi
[RESULT][VOTE]Migrate the default branch from "master" to "main"/"develop"
Hi all, As the vote[1] have passed more than 72 hours, the vote results are as following [D] Develop +7 [M] Main +1 [K] Keep master 0 Thanks to everybody who votes! [1] https://lists.apache.org/thread.html/rf7dce8d4cfcf4001feeba139cc897d6b40a1741e06ef87aabd56d8c9%40%3Cdev.iotdb.apache.org%3E Thanks, --- Houliang Qi
Re: [VOTE] Migrate the default branch from "master" to "main"/"develop"
Hi Xiangdong, Thank you for your advice and I will start a new discussion with your opinion. Thanks, --- Houliang Qi On 10/12/2020 12:04，Xiangdong Huang wrote： Hi Houliang, Thanks for raising this up. 1. You'd better to start a new thread entitled "[VOTE][RESULT] ." for the vote. 2. start another thread to discuss about the cluster version management. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Houliang Qi 于2020年10月12日周一 上午11:57写道： Hi all, As the vote have passed more than 72 hours, the vote results are as following [D] Develop +7 [M] Main +1 [K] Keep master 0 And I’d like to start a new discussion about this: As someone mentioned that there should be a develop branch and a release branch. The develop branch is used to submit the latest development, and the release branch is used for the functions that need to be released in the latest release. At the same time, I hope that the development of cluster can also have two branches: cluster_develop branch and cluster_release branch. The cluster_develop branch is used to merge the stand-alone version of the develop branch code and the latest development of the cluster. The cluster_release branch is used to release some of the latest features. Only the functions that need to be released are allowed to be merged into the cluster_release branch, or to fix some bugs. Other newly developed functions are not allowed to be merged into the cluster_release branch. After cluster_release has been fully tested, it can be released. Regarding the latest release, I would like to check out a cluster_release branch after the cluster_premerge branch merges into the master (develop), and then the master branch merges into the cluster_new (cluster_develop) branch. And I think the new functions do not have beed tested or need more than one month to tested should be switch off when release the cluster version. Does anyone have some ideas about this? Thanks, --- Houliang Qi On 09/24/2020 02:38，Kevin A. McGrail wrote： I am +1 to rename it but don't have any good input on what would be a good name to use going forward. I'll defer to others on that. On 9/22/2020 4:23 AM, Xiangdong Huang wrote: Hi, There is a movement to move the default branch from "master" to "main" or "develop", because of two reasons: - Many people around the world thought the word "master" has some other meaning. - the word "master" can not clearly describe the branch's purpose. As we use the branch as our main working/developing branch, "main" or "develop" may be better. We had a discussion on private@ and I think it is time to start a vote in public. So, I'd like to call a formal vote for changing the default branch: - [M] main - [D] develop - [K] Keep "master" The vote will last at least 72 hours. The name who gets the most votes (and >= 3 votes) wins. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 -- Kevin A. McGrail kmcgr...@apache.org Member, Apache Software Foundation Chair Emeritus Apache SpamAssassin Project https://www.linkedin.com/in/kmcgrail - 703.798.0171
Re: [VOTE] Migrate the default branch from "master" to "main"/"develop"
Hi all, As the vote have passed more than 72 hours, the vote results are as following [D] Develop +7 [M] Main +1 [K] Keep master 0 And I’d like to start a new discussion about this: As someone mentioned that there should be a develop branch and a release branch. The develop branch is used to submit the latest development, and the release branch is used for the functions that need to be released in the latest release. At the same time, I hope that the development of cluster can also have two branches: cluster_develop branch and cluster_release branch. The cluster_develop branch is used to merge the stand-alone version of the develop branch code and the latest development of the cluster. The cluster_release branch is used to release some of the latest features. Only the functions that need to be released are allowed to be merged into the cluster_release branch, or to fix some bugs. Other newly developed functions are not allowed to be merged into the cluster_release branch. After cluster_release has been fully tested, it can be released. Regarding the latest release, I would like to check out a cluster_release branch after the cluster_premerge branch merges into the master (develop), and then the master branch merges into the cluster_new (cluster_develop) branch. And I think the new functions do not have beed tested or need more than one month to tested should be switch off when release the cluster version. Does anyone have some ideas about this? Thanks, --- Houliang Qi On 09/24/2020 02:38，Kevin A. McGrail wrote： I am +1 to rename it but don't have any good input on what would be a good name to use going forward. I'll defer to others on that. On 9/22/2020 4:23 AM, Xiangdong Huang wrote: Hi, There is a movement to move the default branch from "master" to "main" or "develop", because of two reasons: - Many people around the world thought the word "master" has some other meaning. - the word "master" can not clearly describe the branch's purpose. As we use the branch as our main working/developing branch, "main" or "develop" may be better. We had a discussion on private@ and I think it is time to start a vote in public. So, I'd like to call a formal vote for changing the default branch: - [M] main - [D] develop - [K] Keep "master" The vote will last at least 72 hours. The name who gets the most votes (and >= 3 votes) wins. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 -- Kevin A. McGrail kmcgr...@apache.org Member, Apache Software Foundation Chair Emeritus Apache SpamAssassin Project https://www.linkedin.com/in/kmcgrail - 703.798.0171
Re:Draft for board report
Hi Xiangdong, Thanks for proposing the main work of IoTDB for the last month, only minor issues: > The Apache CarbonData is an IoT native database with high performance ... Maybe it's: The Apache IoTDB is an IoT native database with high performance... Thanks, --- Houliang Qi On 10/11/2020 17:15，Xiangdong Huang wrote： Hi all, As IoTDB graduated last month, and the board requires writing reports every month for the first 3 months. Here is the draft. If no issues, I will submit it on Sep 14th. ## Description: - The Apache CarbonData is an IoT native database with high performance for data management and analysis on both the edge and the cloud. ## Issues: - There are no new issues requiring board attention at this time. ## Activity: As IoTDB has reported on Sep, we only summary the activities from Sep to Oct. - Apache IoTDB graduated last month (Sep 16th). Common tasks have been done for migrating IoTDB from the incubator to tlp, including the repository, the website description, the Travis-CI settings, etc.. - We had two public talks about IoTDB, one is "Use cases and optimizations of IoTDB" on ApacheCon 2020, and another is "IoTDB and Hadoop: Connecting the edge and the cloud open source ecosystem for IIoT" on Hadoop meetup in Shanghai, China. - Two proposals are submitted and accepted for Outreachy Intern - We are organizing the design documents on IoTDB's cwiki space - We are working for v0.11 - https://cwiki.apache.org/confluence/display/IOTDB/v0.11 ## Health Report: As IoTDB has reported on Sep, we only summary the health report from Sep to Oct. - Commit activity: - 434 commits in the last month - GitHub PR activity: - 136 PRs opened on GitHub, last month - 108 PRs closed on GitHub, last month ## Releases: - 0.10.1 was released on 2020-08-23 ## Project Composition: - There are currently 35 committers and 23 PMC members in this project. - The Committer-to-PMC ratio is roughly 3:2. ## Community changes: - Chao Wang was added as a committer on 2020-09-03 ## JIRA activity: - 81 issues opened in JIRA, last month Best, --- Xiangdong Huang School of Software, Tsinghua University
Re: Authority of IoTDB's confluence
HouliangQi neuyi...@163.com Thanks, --- Houliang Qi On 10/9/2020 14:54<1101967...@qq.com> wrote?? yanhongwang ---- ??: "dev" https://cwiki.apache.org/confluence/display/IOTDB/Design+Document Thanks, -- Jialin Qiao School of Software, Tsinghua University ??
Re: [jira] [Created] (IOTDB-923) [Distributed] Sync tool to support edge send data to cloud
Hi, Some supplements： For sender： The leader of the DataGroup is responsible for sending data to the remote; The sender just need to know the address of the remote cluster nodes, the routing information should be transparent to the sender; For follower: Provide the interface of sync TsFile in session api; Forward the TsFile to the node that should manage the TsFile(we can cache the routing information in the session of the client); > I think we could redesign the sync tool to support sync data to the formats > of other storage system Big +1, if we can provide tools to transfer TsFile format to other file format(CSV、Parquet、Avro), this will make it easier for other OLAP to analyze TsFile and enrich our ecosystem. Thanks, --- Houliang Qi On 09/30/2020 10:39，runhus...@foxmail.com wrote： Hi, here are my opinions: 1. the Leader of each DataGroup should sync data to remote, and ask the destination node of the data file before sending the data. 2. the remote IoTDB tells the destination of the data file to the edge IoTDB. 3. the remote Leader of the DataGroup receive the data file and sync to the follower. And, I think we could redesign the sync tool to support sync data to the formats of other storage system. This will enrich our ecosystem. Thanks. Chao Wang BONC Ltd From: Xiangdong Huang Date: 2020-09-29 11:50 To: dev CC: notifications Subject: Re: [jira] [Created] (IOTDB-923) [Distributed] Sync tool to support edge send data to cloud Hi, Any ideas? Send to some node and let the node partition the data file and then forward to other nodes? --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 WangChao (Jira) 于2020年9月28日周一 上午10:15写道： WangChao created IOTDB-923: -- Summary: [Distributed] Sync tool to support edge send data to cloud Key: IOTDB-923 URL: https://issues.apache.org/jira/browse/IOTDB-923 Project: Apache IoTDB Issue Type: New Feature Components: Core/Cluster Reporter: WangChao Now we have sync tool to support edge send data to cloud in single mode. We also need support sync tool in cluster version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Outhreachy Internship: Should we enrol as IOTDB? Anyone interested in mentorship
Hi Xiangdong, Yeah, for those devices can not run JVM and do not support TCP are not appropriate for The restful API approach. BTW, as you mentioned, if each device supports a different transport protocol, does IoTDB need to support this protocol accordingly?(like AMQP、CoAP、DDS、MQTT(already supported) and so on). Thanks, --- Houliang Qi On 10/9/2020 11:59，Xiangdong Huang wrote： Hi Houliang, I am trying to consider the following scenario: We have a device (e.g., a robot arm), an edge gateway, and a cloud center. Let's just consider the first two roles: the device and the edge gate. The device collects data from its PLC, and may store the data locally for a short time, and then sends to the edge gateway. The edgeway collects data from the device and may store the data locally for a short time and then sends to the cloud. On both the device and the edgeway, they may have no ability to connect to another service by TCP, and have to save data locally because of some reasons. Yes they can call rest API in their programs, but if they can not run Java locally, then we can do nothing. But TsFile is a data exchange way as it stores binary data. Writing with C++, and sending the file to another role (who can run Java) and then we can run IOTDB there. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Houliang Qi 于2020年10月9日周五 上午11:20写道： Hi, Xiangdong |I have a crazy idea (but not sure whether it works), make TsFile cross-language. I have an immature idea. Why not turn TsFile into a restful micro-service that can provide read-write and other operations. Since Java is cross platform, as long as there is a Java virtual machine, you don't need to care about the language implementation of these embedded devices. Thanks, --- Houliang Qi On 09/24/2020 15:05，Xiangdong Huang wrote： Hi, I have a crazy idea (but not sure whether it works), make TsFile cross-language. Now we can use Java to generate and query data from TsFile (Maybe we can also use C++ to do that by the effort of Giorgio). But how about Go, how about python? It will make the Tsfile really available on embedded devices if we can support those languages. And, how we maintain them if TsFile's structure is changed? (It is obvious that the file format will evolve time by time). So, a hard-work task is, implementing TsFile with different languages. A smart task is using some DSL to describe it and generating codes into different languages (at least generating part of codes). Is that a good task for the internship? Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Xiangdong Huang 于2020年9月24日周四 下午2:19写道： Hi Giorgio, Thanks for raising this up. I notice the DDL is entened to 29th. It is a good chance. Any ideas? Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Giorgio Zoppi 于2020年9月24日周四 上午2:18写道： Hi all, Apache Software Foundation has participated in the next round of Outreachy Internships (https://www.outreachy.org/). Can we send them some projects ? BR, Giorgio
Re: Outhreachy Internship: Should we enrol as IOTDB? Anyone interested in mentorship
Hi, Xiangdong |I have a crazy idea (but not sure whether it works), make TsFile cross-language. I have an immature idea. Why not turn TsFile into a restful micro-service that can provide read-write and other operations. Since Java is cross platform, as long as there is a Java virtual machine, you don't need to care about the language implementation of these embedded devices. Thanks, --- Houliang Qi On 09/24/2020 15:05，Xiangdong Huang wrote： Hi, I have a crazy idea (but not sure whether it works), make TsFile cross-language. Now we can use Java to generate and query data from TsFile (Maybe we can also use C++ to do that by the effort of Giorgio). But how about Go, how about python? It will make the Tsfile really available on embedded devices if we can support those languages. And, how we maintain them if TsFile's structure is changed? (It is obvious that the file format will evolve time by time). So, a hard-work task is, implementing TsFile with different languages. A smart task is using some DSL to describe it and generating codes into different languages (at least generating part of codes). Is that a good task for the internship? Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Xiangdong Huang 于2020年9月24日周四 下午2:19写道： Hi Giorgio, Thanks for raising this up. I notice the DDL is entened to 29th. It is a good chance. Any ideas? Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Giorgio Zoppi 于2020年9月24日周四 上午2:18写道： Hi all, Apache Software Foundation has participated in the next round of Outreachy Internships (https://www.outreachy.org/). Can we send them some projects ? BR, Giorgio
Re: [VOTE] Migrate the default branch from "master" to "main"/"develop"
Hi, all +1 for [D]Develop. Thanks, --- Houliang Qi On 09/22/2020 16:45，runhus...@foxmail.com wrote： Hi, I think develop is more suitable. Thanks. Chao Wang BONC Ltd From: Xiangdong Huang Date: 2020-09-22 16:23 To: dev Subject: [VOTE] Migrate the default branch from "master" to "main"/"develop" Hi, There is a movement to move the default branch from "master" to "main" or "develop", because of two reasons: - Many people around the world thought the word "master" has some other meaning. - the word "master" can not clearly describe the branch's purpose. As we use the branch as our main working/developing branch, "main" or "develop" may be better. We had a discussion on private@ and I think it is time to start a vote in public. So, I'd like to call a formal vote for changing the default branch: - [M] main - [D] develop - [K] Keep "master" The vote will last at least 72 hours. The name who gets the most votes (and >= 3 votes) wins. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院
Re: [VOTE] Start the graduation process
Hi， +1 from contributor. Thanks, --- Houliang Qi On 08/31/2020 10:03，Gaofei Cao wrote： Hi, big +1 from PPMC. Best, -- Gaofei Cao Dawei Liu 于2020年8月31日周一 上午9:58写道： Hi， big +1 Best, ——— Dawei Liu On 08/31/2020 09:55，Xiangdong Huang wrote： Hi, big +1 from PPMC. Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 袁骏 于2020年8月31日周一 上午9:51写道： Hi, +1 from PPMC It's been astounding progress for 2 years! -- 袁骏 首席架构师 清华大学大数据国家实验室 At 2020-08-30 23:54:00, "李天安" wrote: Hi, +1 from PPMC Thanks, Best Regards, — Tianan Li School of Software, Tsinghua University 李天安 清华大学 软件学院 On 08/30/2020 20:46，Julian Feinauer wrote： Hi folks, as we just had a very positive discussion about the maturity of our community [1] I want to formally start the vote if we want to start the graduation process. For Information about our maturity assessment please see [2]. Do you agree that the IoTDB project is ready for graduation and that we should start the graduation process now? As usual, this vote will be open for at least 72hrs. Best Julian [1] https://lists.apache.org/thread.html/r7607cbf6ff2020dc7e7cc27eb698ce265b768ed2a63d3e353c2dc572%40%3Cdev.iotdb.apache.org%3E [2] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=148645763
Re:IOTDB-845
Thanks for you contribution! Welcome to a simple introduction~ :) I think it would be better if we could briefly introduce what this PR does. Thanks, --- Houliang Qi On 08/21/2020 14:33<1101967...@qq.com> wrote?? I've submitted a PR for issue IOTDB-845 [https://issues.apache.org/jira/browse/IOTDB-845]
Re: [VOTE] Apache IoTDB 0.10.1 (incubating) RC2 release
Hi all, +1 from contributor I checked the following operations on macOS 10.15.4 and jdk 1.8 The source release: Incubating in name [ok] Has DISCLAIMER [ok] LICENSE and NOTICE [ok] Signatures and hashes [ok] Release candidates match with corresponding tags [ok] No jar files [ok] Check third party dependencies [ok] Could compile from source: ./mvnw.sh clean install [ok] The binary distribution: Incubating in name [ok] Has DISCLAIMER [ok] LICENSE and NOTICE [ok] Signatures and hashes [ok] Could run with the following statements [ok] SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); select * from root; Thanks, - Houliang Qi On 07/29/2020 22:23，孙泽嵩 wrote： Hi, +1 from committer I checked: - Incubating in name - Download links - Signatures and hashes - DISCLAIMER - Git tag - ASF headers in all source files - Compiling from source - LICENSE and NOTICE - Start in Mac - Run with following statements SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); select * from root; Best, --- Zesong Sun School of Software, Tsinghua University 孙泽嵩 清华大学 软件学院 2020年7月29日 19:57，Jialin Qiao 写道： Hi, +1 from PPMC The source release: incubating in name [ok] apache headers [ok] signatures and hashes [ok] DISCLAIMER [ok] LICENSE and NOTICE [ok] no jar files [ok] could compile from source: ./mvnw.sh clean install [ok] The binary distribution: version number in CLI [ok] signatures and hashes [ok] start in mac, jdk8 [ok] statements executed successfully: [ok] SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); select * from root; Thanks, — Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院 -原始邮件- 发件人: "Haonan Hou" 发送时间: 2020-07-29 17:46:45 (星期三) 收件人: "dev@iotdb.apache.org" 抄送: 主题: Re: [VOTE] Apache IoTDB 0.10.1 (incubating) RC2 release Hi, +1 from PPMC The source release: incubating in name [ok] apache headers [ok] signatures and hashes [ok] DISCLAIMER [ok] LICENSE and NOTICE [ok] no jar files [ok] could compile from source: ./mvnw.sh clean install [ok] The binary distribution: version number in CLI [ok] signatures and hashes [ok] start in mac, jdk8 [ok] statements executed successfully: [ok] SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); select * from root; Thanks, Haonan Hou On Jul 28, 2020, at 7:41 PM, Xiangdong Huang wrote: Hi all, RC2 is coming. This is a vote for releasing v0.10.1, which is a bug-fix version of v0.10.0. You can get its main changes from [5]. Comparing with RC1, RC2 fixes the following issue: - add README_ZH.md file into release. - upgrade orc-core dependency for license compatibility - remove org.json dependency for license compatibility Apache IoTDB (Incubating) 0.10.1 has been staged under [2] and it’s time to vote on accepting it for release. All Maven artifacts are available under [1]. If approved we will seek final release approval from the IPMC. Voting will be open for 72hr. A minimum of 3 binding +1 votes and more binding +1 than binding -1 are required to pass. tag: release/0.10.1 Hash for the release tag: e1bc304a966263217602e8e9bf90e0de1beaaa20 According to [3], "Before voting +1 [P]PMC members are required to download the signed source code package, compile it as provided, and test the resulting executable on their own platform, along with also verifying that the package meets the requirements of the ASF policy on releases." You can achieve the above by following [4]. [ ] +1 accept (indicate what you validated - e.g. performed the non-RM items in [4]) [ ] -1 reject (explanation required) [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1043 [2] https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.10.1/rc2 [3] https://www.apache.org/dev/release.html#approving-a-release [4] https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release [5] https://dist.apache.org/repos/dist/dev/incubator/iotdb/0.10.1/rc2/RELEASE_NOTES.md [6] https://dist.apache.org/repos/dist/
Re: [VOTE] Apache IoTDB 0.10.1 (incubating) RC1 release
Hi all, -1 from contributor I checked the following operations on macOS 10.15.4 and jdk 1.8 Check third party dependencies [Not OK] hive-connector’s dependencies(hive-serde -> orc-core ) use GPL 2.0 license, which incompatible with apache 2.0 license (Remove the orc-core dependency in hive-serde and use orc-core 1.6 is ok ) The source release: Check third party dependencies [not ok] Incubating in name [ok] Has DISCLAIMER [ok] LICENSE and NOTICE [ok] Signatures and hashes [ok] Release candidates match with corresponding tags [ok] no jar files [ok] Could compile from source: ./mvnw.sh clean install [ok] The binary distribution: Incubating in name [ok] Has DISCLAIMER [ok] LICENSE and NOTICE [ok] Signatures and hashes [ok] Could run with the following statements [ok] SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); select * from root; minor issue : The mvnw.sh does not have executable permissions —— Thanks, Houliang Qi On 07/27/2020 16:37，runhus...@foxmail.com wrote： Hi, +1 from contributor I checked: - Incubating in name - Download links - Signatures and hashes - Compiling from source - Start in Windows - Run with following statements SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); select * from root; select * from root.* limit 0 offset 23; minor issue : offset and limit only support Int32, maybe it's small. Thanks! runhus...@foxmail.com From: Dawei Liu Date: 2020-07-24 22:14 To: dev@iotdb.apache.org CC: dev@iotdb.apache.org Subject: Re: [VOTE] Apache IoTDB 0.10.1 (incubating) RC1 release Hi, +1 I checked: - Incubating in name - Download links - Start in Mac - Run with following statements set storage group to root.release; create timeseries root.release.d1.test(status) with DATATYPE=BOOLEAN, ENCODING=PLAIN, compression=SNAPPY show timeseries; insert into root.release.d1(time,status) values(1,false); select * from root.release.d1; minor issue : The package not include 'README_ZH.md' file Best —— Dawei Liu On 07/24/2020 21:31，孙泽嵩 wrote： Hi, +1 from committer : ) I checked: - Incubating in name - Download links - Signatures and hashes - DISCLAIMER - Git tag - ASF headers in all source files - Compiling from source - LICENSE and NOTICE - Start in Mac - Run with following statements SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); select * from root; Best, --- Zesong Sun School of Software, Tsinghua University 孙泽嵩 清华大学 软件学院 2020年7月24日 19:54，Jialin Qiao 写道： Hi, +1 from PPMC The source release: incubating in name [ok] apache headers [ok] signatures and hashes [ok] DISCLAIMER [ok] LICENSE and NOTICE [ok] no jar files [ok] could compile from source: ./mvnw.sh clean install [ok] The binary distribution: version number in CLI [ok] signatures and hashes [ok] start in mac, jdk8 [ok] statements executed successfully: [ok] SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); select * from root; Thanks, — Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院 -原始邮件- 发件人: "Haonan Hou" 发送时间: 2020-07-24 15:44:29 (星期五) 收件人: "dev@iotdb.apache.org" 抄送: 主题: Re: [VOTE] Apache IoTDB 0.10.1 (incubating) RC1 release Hi, +1 (PPMC binding) The source release: Incubating in name [ok] Has DISCLAIMER [ok] LICENSE and NOTICE [ok] signatures and hashes [ok] All files have ASF header [ok] could compile from source: sh mvnw.sh clean install [ok] The binary distribution: Incubating in name [ok] Has DISCLAIMER [ok] LICENSE and NOTICE [ok] signatures and hashes [ok] Could run with the following statements [ok] SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); select * from root; Thanks, Haonan Hou On Jul 24, 2020, at 1:05 AM, Xiangdong Huang mailto:saint...@gmail.com>> wrote: Hi all, This is a vote for releasin
Re:[ANNOUNCE] Add Haonan Hou as a new PPMC of IoTDB
Congratulations, Haonan! —— Thanks, Houliang Qi On 07/17/2020 10:38，Jialin Qiao wrote： Hi all, I am pleased to announce that after discussion and vote among PPMC groups, and publicity on IPMC mailing list, Haonan Hou is nominated as a new PPMC of IoTDB. Haonan has joined the IoTDB community for more than 10 months and is always active in the mailing list. He mainly focuses on the new TsFile structure, online upgrade tools, documentations. He also contributes a lot for the propagation of IoTDB and resolving users' problems in the github issues. Now, let's welcome Haonan! Best, -- Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院
Re: Online discussion for IoTDB design and features
Hi, I’m interested in participating. If someone can share the problems encountered in IoTDB in practice, and how to fix these problems? what lessons learned from these problems? I think this is very helpful for me to use and understand IoTDB. —— Thanks, Houliang Qi On 07/7/2020 09:37，Cesar Garcia wrote： Hi, Also interested in participating Best regards, El dom., 14 de junio de 2020 1:44 p. m., Xiangdong Huang escribió: Hi folks, These days I discussed with Julian and find that.. we have so many issues on JIRA and some design documents on the website, but guys in the community may still hard to follow them because they may do not know why there is such a feature, or why the design looks like this. Julian suggests holding a hangout to discuss such questions. I think it is a good idea, and I'd like to set it as a regular discussion (e.g., every 2~3 weeks). The form can be an online hangout (e.g., using Zoom), quite looks like an online meetup, but not as formal as a meetup (it is a little hard at the beginning because many guys are Chinese but we have to discuss in English). We just want to give some space to let interested persons discuss the new feature of IoTDB and the design of IoTDB. Before starting such an online hangout, we can collect the topics that wanted to be discussed, e.g., why issue IOTDB-XXX is proposed, how to implement IOTDB-XXX... After the hangout, we can record useful information and save them as our docs. (So to follow the Apache Way, everything is on the mailing list) How do you think? Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 -- --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院
I've submitted a PR for issue IOTDB-781
Hi, all Now, the distributed version do not support ALTER statements like ALTER tag or attributes[1], this PR[2] is used to add these functions. I have tested those ALTER statements[3] in the user guide and all executed successfully. please leave your opinions. [1] https://issues.apache.org/jira/browse/IOTDB-781 [2] https://github.com/apache/incubator-iotdb/pull/1405 [3] http://iotdb.apache.org/UserGuide/Master/Operation%20Manual/DDL%20Data%20Definition%20Language.html#update-tag-operation —— Thanks, Houliang Qi
I've submitted a PR for issue IOTDB-722
Hi, all I submitted a pr to support three consistency levels: strong, mid and weak. (https://github.com/apache/incubator-iotdb/pull/1371) issue: https://issues.apache.org/jira/browse/IOTDB-722 Strong consistency means the server will first try to synchronize with the leader to get the newest meta data, if failed(timeout), directly report an error to the user; While mid consistency means the server will first try to synchronize with the leader, but if failed(timeout), it will give up and just use current data it has cached before; Weak consistency do not synchronize with the leader and simply use the local data. Please leave your opinions, thanks. __ Thanks, Houliang Qi
I've submitted a PR for issue IOTDB-678
Hi, all If the LOCAL_IP and SEED_NODES in the configuration file have both a local ip(127.0.0.1) and a real ip, the iotdb service cannot be connected to other nodes, causing raft communication to fail, and the cluster is always in the election state. I think the LOCAL_IP and SEED_NODES should be consistent in the conf file; for the pseudo distributed mode, it can be all local ip, for the distributed mode, it must be the real ip, otherwise it will report an error at startup. I have submitted a PR for this: https://github.com/apache/incubator-iotdb/pull/1208 please leave your option, thanks. __ Thanks, Houliang Qi
