[jira] [Commented] (IOTDB-406) Refactor EnvironmentUtils for closing IoTDB timely in Integration Tests
[ https://issues.apache.org/jira/browse/IOTDB-406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010892#comment-17010892 ] Xiangdong Huang commented on IOTDB-406: --- Hm, I find some file writers are not closed... especially for ModificationFiles.. Now, the file open and file close are in different functions, and the close() function may be called in an async way. So, be careful for programming. TODO: For mergingModification in StorageGroupProcessor (SGP), it is closed only after the merging task is finished. # Can we close it once before the merging task starts? # Be careful if we call EnvironmentUtils.cleanEnv, MergeManager may be stopped in force, and thereby some mergingModifications are not closed. [~jt2594838], you'd better have a review. Besides, for StorageGroupProcessor (SGP), notice if you want to remove a SGP out of memory, (for example, in [~SilverNarcissus]'s PR for supporting SG time partition, a SGP with an stale time partition may be cleaned from memory), do not forget to close the corresponding LogFile (now, if a memtable is flushed, the corresponding logfile will be closed), and close all the mod files in this SGP with the time partition (now, the mod files in sequneceFileList and unsequenceFileList are only closed if some call deleteFiles() in SGP, which is just for test. That is, no body close these mod files..). I add a function called closeAllResources() in SGP, you can use it if you want to remove SGP from processorMap of StorageEngine. [~SilverNarcissus]. > Refactor EnvironmentUtils for closing IoTDB timely in Integration Tests > --- > > Key: IOTDB-406 > URL: https://issues.apache.org/jira/browse/IOTDB-406 > Project: Apache IoTDB > Issue Type: Task >Reporter: Xiangdong Huang >Assignee: Rui Liu >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently, we usually occurs that in some ITs, the IoTDB daemon is not closed > in previous tests. > Besides, IoTDB daemon is not needed to exposed to IT classes. > Therefore, I'd like to refactor the EnvironmentUtils class. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IOTDB-406) Refactor EnvironmentUtils for closing IoTDB timely in Integration Tests
[ https://issues.apache.org/jira/browse/IOTDB-406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010912#comment-17010912 ] Xiangdong Huang commented on IOTDB-406: --- Ok, after reading codes, I find we can not close mergingModification before running merge task... > Refactor EnvironmentUtils for closing IoTDB timely in Integration Tests > --- > > Key: IOTDB-406 > URL: https://issues.apache.org/jira/browse/IOTDB-406 > Project: Apache IoTDB > Issue Type: Task >Reporter: Xiangdong Huang >Assignee: Rui Liu >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently, we usually occurs that in some ITs, the IoTDB daemon is not closed > in previous tests. > Besides, IoTDB daemon is not needed to exposed to IT classes. > Therefore, I'd like to refactor the EnvironmentUtils class. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IOTDB-406) Refactor EnvironmentUtils for closing IoTDB timely in Integration Tests
[ https://issues.apache.org/jira/browse/IOTDB-406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010919#comment-17010919 ] Xiangdong Huang commented on IOTDB-406: --- [~jt2594838] and [~SilverNarcissus], how about the following case: 1. a SGP with time partition 1 need to be removed from memory; 2. But the SGP has a merge task and there are mergingModifications... 3. After the replacement, new data belong to that time partition comes... so the SPG is created once again, and then new modifications come, and a new merging task has to be triggered. (Does the mergeManager support that? and what will happen if now we have two mergingModification files) > Refactor EnvironmentUtils for closing IoTDB timely in Integration Tests > --- > > Key: IOTDB-406 > URL: https://issues.apache.org/jira/browse/IOTDB-406 > Project: Apache IoTDB > Issue Type: Task >Reporter: Xiangdong Huang >Assignee: Rui Liu >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently, we usually occurs that in some ITs, the IoTDB daemon is not closed > in previous tests. > Besides, IoTDB daemon is not needed to exposed to IT classes. > Therefore, I'd like to refactor the EnvironmentUtils class. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (IOTDB-401) Flushed data size inconsistent exception when writing measurement(same name but different device) with different data type
[
https://issues.apache.org/jira/browse/IOTDB-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jialin Qiao closed IOTDB-401.
-
Fix Version/s: 0.10.0-SNAPSHOT
Resolution: Fixed
> Flushed data size inconsistent exception when writing measurement(same name
> but different device) with different data type
> --
>
> Key: IOTDB-401
> URL: https://issues.apache.org/jira/browse/IOTDB-401
> Project: Apache IoTDB
> Issue Type: Bug
>Reporter: EJTTianyu
>Assignee: Xiangdong Huang
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.10.0-SNAPSHOT
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Hi,
> My current project requires exporting data from Kairosdb as tsfile. I use
> the following steps to complete this task.
> 1. Query and export kairosdb data to csv;
> 2. Export csv to tsfile.
> In the second step, I encountered the following exception:
> {code:java}
> ERROR org.apache.iotdb.tsfile.write.chunk.ChunkWriterImpl - meet error in
> chunkBuffer.writePageHeaderAndDataIntoBuff, ignore this page:
> org.apache.iotdb.tsfile.exception.write.PageException: No valid data point in
> this page
> at
> org.apache.iotdb.tsfile.write.chunk.ChunkBuffer.writePageHeaderAndDataIntoBuff(ChunkBuffer.java:91)
> at
> org.apache.iotdb.tsfile.write.chunk.ChunkWriterImpl.writePage(ChunkWriterImpl.java:308)
> at
> org.apache.iotdb.tsfile.write.chunk.ChunkWriterImpl.sealCurrentPage(ChunkWriterImpl.java:351)
> at
> org.apache.iotdb.tsfile.write.chunk.ChunkGroupWriterImpl.sealAllChunks(ChunkGroupWriterImpl.java:155)
> at
> org.apache.iotdb.tsfile.write.chunk.ChunkGroupWriterImpl.flushToFileWriter(ChunkGroupWriterImpl.java:124)
> at
> org.apache.iotdb.tsfile.write.TsFileWriter.flushAllChunkGroups(TsFileWriter.java:310)
> at
> org.apache.iotdb.tsfile.write.TsFileWriter.close(TsFileWriter.java:337)
> ERROR kairosdb.export.csv.TransToTsfile - error occurs when generate tsfile
> java.io.IOException: Flushed data size is inconsistent with computation!
> Estimated: 29, Actual: 0
> at
> org.apache.iotdb.tsfile.write.TsFileWriter.flushAllChunkGroups(TsFileWriter.java:312)
> at
> org.apache.iotdb.tsfile.write.TsFileWriter.close(TsFileWriter.java:337)
> {code}
> You can use the following strategy to reproduce this bug.
> First create a time series like d0.s0, whose data type is INT64. Write some
> data point to the TsfileWriter. Second create a time series like d1.s0, whose
> data type is FLOAT. When writing TSRecord to the same tsfile writer, the bug
> occurs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IOTDB-409) Chinese and English documents are inconsistent (Chinese documents are incorrect)
sunjincheng created IOTDB-409:
-
Summary: Chinese and English documents are inconsistent (Chinese
documents are incorrect)
Key: IOTDB-409
URL: https://issues.apache.org/jira/browse/IOTDB-409
Project: Apache IoTDB
Issue Type: Improvement
Reporter: sunjincheng
Attachments: image-2020-01-09-12-12-32-828.png,
image-2020-01-09-12-13-13-306.png
Chinese and English documents are inconsistent (Chinese documents are
incorrect)， as follows:
English:
!image-2020-01-09-12-12-32-828.png|width=206,height=130!
Chinese:
!image-2020-01-09-12-13-13-306.png|width=408,height=306!
In the current Chinese document, the user cannot know the directory
`IOTDB_HOME` is
`distribution/target/apache-iotdb-\{version}-SNAPSHOT-incubating-bin/apache-iotdb-\{version}-SNAPSHOT-incubating`.
What do you think?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IOTDB-410) It's better to support selecting Components when create JIRA.
sunjincheng created IOTDB-410: - Summary: It's better to support selecting Components when create JIRA. Key: IOTDB-410 URL: https://issues.apache.org/jira/browse/IOTDB-410 Project: Apache IoTDB Issue Type: Improvement Reporter: sunjincheng Attachments: image-2020-01-09-12-23-41-034.png Currently we can not select a component for a ISSUE. as follows: !image-2020-01-09-12-23-41-034.png|width=315,height=201! And we already have client/server/connector/website etc... components. So It's better to support selecting Components when create JIRA. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[CALL FOR VOTES] Release Apache IoTDB 0.9.1
Hi, After the vote letter titled "[VOTE] Release Apache IoTDB 0.9.1" was sent to gene...@incubator.apache.org on Jan 3, two +1 votes have been cast by Justin Mclean and Willem Jiang. We may need more votes, since a minimum of 3 binding +1 votes and more binding +1 than binding -1 are required to pass. @Christofer Dutz @Kevin A. McGrail @Julian Feinauer Regards, Lei Rui
[jira] [Commented] (IOTDB-410) It's better to support selecting Components when create JIRA.
[
https://issues.apache.org/jira/browse/IOTDB-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17011450#comment-17011450
]
sunjincheng commented on IOTDB-410:
---
Configured page address is:
[https://issues.apache.org/jira/plugins/servlet/project-config/\{project_name}/components
|https://issues.apache.org/jira/plugins/servlet/project-config/%7Bproject_name%7D/components]
> It's better to support selecting Components when create JIRA.
> -
>
> Key: IOTDB-410
> URL: https://issues.apache.org/jira/browse/IOTDB-410
> Project: Apache IoTDB
> Issue Type: Improvement
>Reporter: sunjincheng
>Priority: Major
> Attachments: image-2020-01-09-12-23-41-034.png
>
>
> Currently we can not select a component for a ISSUE. as follows:
> !image-2020-01-09-12-23-41-034.png|width=315,height=201!
> And we already have client/server/connector/website etc... components. So
> It's better to support selecting Components when create JIRA.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [Discuss] More reasonable issue permission on JIRA
I have checked the issue permission, It works now :) Thanks all of you made it. Best, Jincheng jincheng sun 于2020年1月8日周三 下午1:10写道： > Great thanks for driving this Chris！ > > I have left comment on the JIRA. Hope this can be resolved soon :) > > BTW: cc to @dev :) > > Best, > Jincheng > > > Christofer Dutz 于2020年1月7日周二 下午6:19写道： > >> Hi all, >> >> >> >> I just requested a custom permission scheme so we will be able to tweak >> the settings. >> >> >> >> https://issues.apache.org/jira/browse/INFRA-19676 >> >> >> >> >> >> Chris >> >> >> >> *Von: *jincheng sun >> *Datum: *Samstag, 21. Dezember 2019 um 06:11 >> *An: *"dev@iotdb.apache.org" >> *Cc: *Christofer Dutz , "jmcl...@apache.org" < >> jmcl...@apache.org>, "kmcgr...@apache.org" , " >> ningji...@apache.org" , Justin Mclean < >> jus...@classsoftware.com> >> *Betreff: *Re: [Discuss] More reasonable issue permission on JIRA >> >> >> >> Is there any progress here to update? :) >> >> >> >> Best, >> >> Jincheng >> >> >> >> >> >> jincheng sun 于2019年12月14日周六 下午10:32写道： >> >> Hi Xiangdong, >> >> >> >> Thanks for bring up this discussion! >> >> >> >> Big +1 for adding the role of "Contributors" to "Assignable User" list, >> as I think allowing JIRA. to be assigned to contributors is a prerequisite >> for growing our IoTDB, and this improvement will showing the welcome to new >> contributors. :) >> >> >> >> Best, >> >> Jincheng >> >> - >> >> Committer & PMC Member at @ApacheFlink >> >> Staff Engineer at @Alibaba >> >> Blog: https://enjoyment.cool >> >> Twitter: https://twitter.com/sunjincheng121 >> >> -- >> >> >> >> >> >> Xiangdong Huang 于2019年12月14日周六 下午1:46写道： >> >> Hi, >> >> Now IoTDB is using _Default Permission Scheme_ on jira [1]. >> >> The schema looks good, but one thing that I think we can have a change: >> >> Current the "Assignable User" needs to be Administrator/PMC/Committer. >> However, if someone (who is not a committer) reports a bug and he/she has >> the ability to fix that and can submit a PR. I think we can assign the >> issue to him/her. >> >> It will give more sense of honor to users who are not the committers but >> make real contribution to the project. And, it will attract more users to >> contribute to IoTDB, I think. Some users also point out that [2]. >> >> Therefore, I suggest to change "Assignable User" to >> Administrator/PMC/Committer/Reporter/Contributor.. >> >> Even though I have the administrator permission of IoTDB JIRA, but the >> above modification nees jira-administrator permission, which is higher >> than >> what I can do. >> >> Maybe @Christofer Dutz can help us. :-D. >> >> [1] >> >> https://issues.apache.org/jira/plugins/servlet/project-config/IOTDB/permissions >> [2] https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-365 >> >> Best, >> --- >> Xiangdong Huang >> School of Software, Tsinghua University >> >> 黄向东 >> 清华大学 软件学院 >> >>
Re: [CALL FOR VOTES] Release Apache IoTDB 0.9.1
I think Julian is checking for votes now... :) Lei Rui 于2020年1月9日周四 下午1:17写道： > Hi, > > > After the vote letter titled "[VOTE] Release Apache IoTDB 0.9.1" was sent > to gene...@incubator.apache.org on Jan 3, > two +1 votes have been cast by Justin Mclean and Willem Jiang. > > > We may need more votes, since a minimum of 3 binding +1 votes and more > binding +1 than binding -1 are required to pass. > @Christofer Dutz @Kevin A. McGrail @Julian Feinauer > > > > Regards, > Lei Rui > >
