[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[ https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16708796#comment-16708796 ] Sean Busbey commented on HBASE-17914: - please move this conversation to either dev@hbase or to a new JIRA. threads on resolved issues get very few eyes. > Create a new reader instead of cloning a new StoreFile when compaction > -- > > Key: HBASE-17914 > URL: https://issues.apache.org/jira/browse/HBASE-17914 > Project: HBase > Issue Type: Sub-task > Components: Compaction, regionserver >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0 > > Attachments: HBASE-17914-v1.patch, HBASE-17914-v1.patch, > HBASE-17914-v2.patch, HBASE-17914-v3.patch, HBASE-17914.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[
https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16708453#comment-16708453
]
Bhupendra Kumar Jain commented on HBASE-17914:
--
{quote}Which feature of Phoenix needs to hack StoreFileReader?{quote}
As I know Phoenix Local Index feature uses this. Please refer class
IndexHalfStoreFileReader
> Create a new reader instead of cloning a new StoreFile when compaction
> --
>
> Key: HBASE-17914
> URL: https://issues.apache.org/jira/browse/HBASE-17914
> Project: HBase
> Issue Type: Sub-task
> Components: Compaction, regionserver
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-17914-v1.patch, HBASE-17914-v1.patch,
> HBASE-17914-v2.patch, HBASE-17914-v3.patch, HBASE-17914.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[
https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16707123#comment-16707123
]
Duo Zhang commented on HBASE-17914:
---
{quote}
Couldn't think of any clean approach .. Please suggest if you have any better
way for this API
{quote}
Then is it possible to get rid of this feature? As said above, StoreFileReader
is for internal use only, projects other than HBase itself should not depend on
it any more. Which feature of Phoenix needs to hack StoreFileReader?
> Create a new reader instead of cloning a new StoreFile when compaction
> --
>
> Key: HBASE-17914
> URL: https://issues.apache.org/jira/browse/HBASE-17914
> Project: HBase
> Issue Type: Sub-task
> Components: Compaction, regionserver
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-17914-v1.patch, HBASE-17914-v1.patch,
> HBASE-17914-v2.patch, HBASE-17914-v3.patch, HBASE-17914.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[
https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16707104#comment-16707104
]
Bhupendra Kumar Jain commented on HBASE-17914:
--
{quote}but could you please find another way to implement the feature?{quote}
Couldn't think of any clean approach .. Please suggest if you have any better
way for this API
{quote}For now imaybe it is fine to add these parameters as this method is only
suppose to be used by Phoenix {quote}
Better to add these parameters as of now. Will open another JIRA for this ...
[~chrajeshbab...@gmail.com] [~anoop.hbase] Phoenix LocalIndex may also have
this issue ?
> Create a new reader instead of cloning a new StoreFile when compaction
> --
>
> Key: HBASE-17914
> URL: https://issues.apache.org/jira/browse/HBASE-17914
> Project: HBase
> Issue Type: Sub-task
> Components: Compaction, regionserver
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-17914-v1.patch, HBASE-17914-v1.patch,
> HBASE-17914-v2.patch, HBASE-17914-v3.patch, HBASE-17914.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[ https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704064#comment-16704064 ] Duo Zhang commented on HBASE-17914: --- The preStoreFileReaderOpen method has already been marked as Deprecated, which means this method has already exposed too many internal states and we want to stop doing this in the future. For now imaybe it is fine to add these parameters as this method is only suppose to be used by Phoenix , but could you please find another way to implement the feature? The StoreFileReader is IA.Private so the methods and behavior will not be stable... Thanks. > Create a new reader instead of cloning a new StoreFile when compaction > -- > > Key: HBASE-17914 > URL: https://issues.apache.org/jira/browse/HBASE-17914 > Project: HBase > Issue Type: Sub-task > Components: Compaction, regionserver >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0 > > Attachments: HBASE-17914-v1.patch, HBASE-17914-v1.patch, > HBASE-17914-v2.patch, HBASE-17914-v3.patch, HBASE-17914.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[ https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703024#comment-16703024 ] Bhupendra Kumar Jain commented on HBASE-17914: -- Ping [~chrajeshbab...@gmail.com], [~anoop.hbase] > Create a new reader instead of cloning a new StoreFile when compaction > -- > > Key: HBASE-17914 > URL: https://issues.apache.org/jira/browse/HBASE-17914 > Project: HBase > Issue Type: Sub-task > Components: Compaction, regionserver >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0 > > Attachments: HBASE-17914-v1.patch, HBASE-17914-v1.patch, > HBASE-17914-v2.patch, HBASE-17914-v3.patch, HBASE-17914.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[
https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701850#comment-16701850
]
Bhupendra Kumar Jain commented on HBASE-17914:
--
In {{StoreFileReader}} class , a new parameter {{shared}} is introduced to
control stream read / pread. But this parameter is not passed to the
co-processor in {{preStoreFileReaderOpen()}}
So co-processor when creating its own StoreFileReader, has not awareness about
type of read. Further based on the parameter value, the readCompleted() method
in StoreFileReader will close the actual HFile reader and underlying streams.
In one of our scenario, with co-processor returning its own StoreFileReader
[shared=false], the scan successfully happens only first time. Next scan
operation are resulting in stream closed exception. (I think, this case may
happens in phoenix IndexHalfStoreFileReader also if HBase compaction is
disabled)
So I think , the new parameter {{shared}} and {{refCount}} should be available
to the co-processor
> Create a new reader instead of cloning a new StoreFile when compaction
> --
>
> Key: HBASE-17914
> URL: https://issues.apache.org/jira/browse/HBASE-17914
> Project: HBase
> Issue Type: Sub-task
> Components: Compaction, regionserver
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-17914-v1.patch, HBASE-17914-v1.patch,
> HBASE-17914-v2.patch, HBASE-17914-v3.patch, HBASE-17914.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[ https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990447#comment-15990447 ] Hudson commented on HBASE-17914: ABORTED: Integrated in Jenkins build HBase-HBASE-14614 #190 (See [https://builds.apache.org/job/HBase-HBASE-14614/190/]) HBASE-17914 Create a new reader instead of cloning a new StoreFile when (zhangduo: rev 66b616d7a3d6f4ad6d20962e2dfc0c82a4092ddb) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestSeekBeforeWithInlineBlocks.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/DataBlockEncodingTool.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/MergeTableRegionsProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/FSDataInputStreamWrapper.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsckEncryption.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMobStoreCompaction.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * (edit) hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/BulkLoadSuite.scala * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionFileSystem.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileReader.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactionPolicy.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedStoreScanner.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mob/compactions/TestMobCompactor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileEncryption.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/FileLink.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/mob/CachedMobFile.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mob/compactions/TestPartitionedMobCompactor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobFile.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestMobFile.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestCompactor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreFileManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTSVWithVisibilityLabels.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileInlineToRootChunkConversion.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestStripeCompactionPolicy.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java * (
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[ https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15974268#comment-15974268 ] Hudson commented on HBASE-17914: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2885 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2885/]) HBASE-17914 Create a new reader instead of cloning a new StoreFile when (zhangduo: rev 66b616d7a3d6f4ad6d20962e2dfc0c82a4092ddb) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestMobFile.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileInlineToRootChunkConversion.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/mob/CachedMobFile.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionFileSystem.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mob/compactions/TestPartitionedMobCompactor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestEncryptionRandomKeying.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestPrefetch.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MockStoreFile.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestSeekBeforeWithInlineBlocks.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/HFileCorruptionChecker.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileReader.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedStoreScanner.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestSeekTo.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/FSDataInputStreamWrapper.java * (edit) hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/BulkLoadSuite.scala * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mob/compactions/TestMobCompactor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileEncryption.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMobStoreCompaction.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestCompactor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestEncryptionKeyRotation.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/DataBlockEncodingTool.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreFileManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactionPolicy.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTSVWithVisibilityLabels.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/EncodedSeekPerformanceTest.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[ https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973872#comment-15973872 ] Duo Zhang commented on HBASE-17914: --- Backporting the patch to branch-1 is not straight-forward as StoreFileReader is still called StoreFile.reader. Will open a backporting issue to do it. > Create a new reader instead of cloning a new StoreFile when compaction > -- > > Key: HBASE-17914 > URL: https://issues.apache.org/jira/browse/HBASE-17914 > Project: HBase > Issue Type: Sub-task > Components: Compaction, regionserver >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17914.patch, HBASE-17914-v1.patch, > HBASE-17914-v1.patch, HBASE-17914-v2.patch, HBASE-17914-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[ https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973667#comment-15973667 ] stack commented on HBASE-17914: --- +1 Lets try it. > Create a new reader instead of cloning a new StoreFile when compaction > -- > > Key: HBASE-17914 > URL: https://issues.apache.org/jira/browse/HBASE-17914 > Project: HBase > Issue Type: Sub-task > Components: Compaction, regionserver >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17914.patch, HBASE-17914-v1.patch, > HBASE-17914-v1.patch, HBASE-17914-v2.patch, HBASE-17914-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[ https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972605#comment-15972605 ] Duo Zhang commented on HBASE-17914: --- Thanks [~ram_krish]. Will commit tomorrow if no objections. > Create a new reader instead of cloning a new StoreFile when compaction > -- > > Key: HBASE-17914 > URL: https://issues.apache.org/jira/browse/HBASE-17914 > Project: HBase > Issue Type: Sub-task > Components: Compaction, regionserver >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17914.patch, HBASE-17914-v1.patch, > HBASE-17914-v1.patch, HBASE-17914-v2.patch, HBASE-17914-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[ https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972357#comment-15972357 ] ramkrishna.s.vasudevan commented on HBASE-17914: LGTM. > Create a new reader instead of cloning a new StoreFile when compaction > -- > > Key: HBASE-17914 > URL: https://issues.apache.org/jira/browse/HBASE-17914 > Project: HBase > Issue Type: Sub-task > Components: Compaction, regionserver >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17914.patch, HBASE-17914-v1.patch, > HBASE-17914-v1.patch, HBASE-17914-v2.patch, HBASE-17914-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[ https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972354#comment-15972354 ] Duo Zhang commented on HBASE-17914: --- Any other concerns? Thanks. > Create a new reader instead of cloning a new StoreFile when compaction > -- > > Key: HBASE-17914 > URL: https://issues.apache.org/jira/browse/HBASE-17914 > Project: HBase > Issue Type: Sub-task > Components: Compaction, regionserver >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17914.patch, HBASE-17914-v1.patch, > HBASE-17914-v1.patch, HBASE-17914-v2.patch, HBASE-17914-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[
https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15971380#comment-15971380
]
Hadoop QA commented on HBASE-17914:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 37 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
0s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 28s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
35s {color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 28s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 28s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 1m 28s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
32s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 0s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
34s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 105m 37s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 50s
{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 158m 1s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.snapshot.TestExportSnapshot |
| | hadoop.hbase.snapshot.TestMobExportSnapshot |
| | hadoop.hbase.snapshot.TestMobSecureExportSnapshot |
| | hadoop.hbase.snapshot.TestSecureExportSnapshot |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12863648/HBASE-17914-v3.patch |
| JIRA Issue | HBASE-17914 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile scalac scaladoc |
| uname | Linux 23f3cec4ff09 3.13.0-107-generic #154-Ubuntu SMP Tue D
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[ https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15970900#comment-15970900 ] Duo Zhang commented on HBASE-17914: --- OK the problem is in the spark module... Let me fix. > Create a new reader instead of cloning a new StoreFile when compaction > -- > > Key: HBASE-17914 > URL: https://issues.apache.org/jira/browse/HBASE-17914 > Project: HBase > Issue Type: Sub-task > Components: Compaction, regionserver >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17914.patch, HBASE-17914-v1.patch, > HBASE-17914-v1.patch, HBASE-17914-v2.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[
https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15970890#comment-15970890
]
Hadoop QA commented on HBASE-17914:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 36 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
46s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 38s
{color} | {color:red} The patch causes 39 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m 1s
{color} | {color:red} The patch causes 39 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 7m 24s
{color} | {color:red} The patch causes 39 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 9m 47s
{color} | {color:red} The patch causes 39 errors with Hadoop v2.6.4. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 12m 11s
{color} | {color:red} The patch causes 39 errors with Hadoop v2.6.5. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 14m 36s
{color} | {color:red} The patch causes 39 errors with Hadoop v2.7.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 16m 59s
{color} | {color:red} The patch causes 39 errors with Hadoop v2.7.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 19m 23s
{color} | {color:red} The patch causes 39 errors with Hadoop v2.7.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 21m 45s
{color} | {color:red} The patch causes 39 errors with Hadoop v3.0.0-alpha2.
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
54s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 26s
{color} | {color:red} hbase-server generated 3 new + 0 unchanged - 0 fixed = 3
total (was 0) {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 104m 48s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 139m 46s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.snapshot.TestExportSnapshot |
| | hadoop.hbase.snapshot.TestMobExportSnapshot |
| | hadoop.hbase.snapshot.TestMobSecureExportSnapshot |
| | hadoop.hbase.snapshot.TestSecureExportSnapshot |
\\
\\
|| S
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[ https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15970362#comment-15970362 ] Duo Zhang commented on HBASE-17914: --- The failed UTs are known to be flaky. So any concerns? Thanks. > Create a new reader instead of cloning a new StoreFile when compaction > -- > > Key: HBASE-17914 > URL: https://issues.apache.org/jira/browse/HBASE-17914 > Project: HBase > Issue Type: Sub-task > Components: Compaction, regionserver >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17914.patch, HBASE-17914-v1.patch, > HBASE-17914-v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[
https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15970315#comment-15970315
]
Hadoop QA commented on HBASE-17914:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 38s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 16 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 16s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 1s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
54m 43s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 108m 8s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 188m 29s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.snapshot.TestMobExportSnapshot |
| | hadoop.hbase.snapshot.TestMobSecureExportSnapshot |
| Timed out junit tests |
org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestExportSnapshot |
| | org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient |
| | org.apache.hadoop.hbase.client.TestSnapshotFromClient |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12863579/HBASE-17914-v1.patch |
| JIRA Issue | HBASE-17914 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 7c8db456b440 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 0cd4cec |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/6449/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/6449/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6449/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Co
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[ https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15970244#comment-15970244 ] Duo Zhang commented on HBASE-17914: --- When the StoreFile is closed. Either by compaction or reassign. > Create a new reader instead of cloning a new StoreFile when compaction > -- > > Key: HBASE-17914 > URL: https://issues.apache.org/jira/browse/HBASE-17914 > Project: HBase > Issue Type: Sub-task > Components: Compaction, regionserver >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17914.patch, HBASE-17914-v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[
https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15970116#comment-15970116
]
Ted Yu commented on HBASE-17914:
{code}
77// indicate that whether this StoreFileReader is shared, i.e., used
for pread. If not, we will
78// close the internal reader when readCompleted is called.
{code}
Who closes the reader when shared is true ?
> Create a new reader instead of cloning a new StoreFile when compaction
> --
>
> Key: HBASE-17914
> URL: https://issues.apache.org/jira/browse/HBASE-17914
> Project: HBase
> Issue Type: Sub-task
> Components: Compaction, regionserver
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0
>
> Attachments: HBASE-17914.patch, HBASE-17914-v1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[
https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15970071#comment-15970071
]
Hadoop QA commented on HBASE-17914:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 26s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 16 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 20s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 2s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
54m 53s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 193m 10s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
25s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 274m 30s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestAsyncRegionAdminApi |
| | hadoop.hbase.util.TestHBaseFsckTwoRS |
| | hadoop.hbase.snapshot.TestMobExportSnapshot |
| | hadoop.hbase.snapshot.TestMobSecureExportSnapshot |
| Timed out junit tests | org.apache.hadoop.hbase.mapreduce.TestRowCounter |
| | org.apache.hadoop.hbase.replication.regionserver.TestWALEntryStream |
| | org.apache.hadoop.hbase.mapreduce.TestImportExport |
| | org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd |
| | org.apache.hadoop.hbase.util.TestHBaseFsckTwoRS |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12863556/HBASE-17914-v1.patch |
| JIRA Issue | HBASE-17914 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 84f086bfa039 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 0cd4cec |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/6444/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/6444/artifact/patchprocess/patch-unit-hbase-
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[ https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15969951#comment-15969951 ] Duo Zhang commented on HBASE-17914: --- The failed UTs are related. Let me check. > Create a new reader instead of cloning a new StoreFile when compaction > -- > > Key: HBASE-17914 > URL: https://issues.apache.org/jira/browse/HBASE-17914 > Project: HBase > Issue Type: Sub-task > Components: Compaction, regionserver >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17914.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[
https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15968958#comment-15968958
]
Hadoop QA commented on HBASE-17914:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 15 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
7s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 54s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 3s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 27s
{color} | {color:red} hbase-server generated 2 new + 0 unchanged - 0 fixed = 2
total (was 0) {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 105m 4s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 147m 11s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestSplitTransactionOnCluster |
| | hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFilesSplitRecovery |
| | hadoop.hbase.snapshot.TestExportSnapshot |
| | hadoop.hbase.snapshot.TestMobExportSnapshot |
| | hadoop.hbase.snapshot.TestSecureExportSnapshot |
| | hadoop.hbase.client.TestFromClientSide |
| | hadoop.hbase.client.TestFromClientSideWithCoprocessor |
| | hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery |
| | hadoop.hbase.client.TestSizeFailures |
| | hadoop.hbase.master.procedure.TestSplitTableRegionProcedure |
| Timed out junit tests |
org.apache.hadoop.hbase.master.procedure.TestModifyTableProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
|
| | org.apache.hadoop.hbase.TestAcidGuarantees |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12863446/HBASE-17914.patch |
| JIRA Issue | HBASE-17914 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux d3df3850567f 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[
https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15968845#comment-15968845
]
Hadoop QA commented on HBASE-17914:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 0m 6s {color}
| {color:red} Docker failed to build yetus/hbase:8d52d23. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12863444/HBASE-17914.patch |
| JIRA Issue | HBASE-17914 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6429/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Create a new reader instead of cloning a new StoreFile when compaction
> --
>
> Key: HBASE-17914
> URL: https://issues.apache.org/jira/browse/HBASE-17914
> Project: HBase
> Issue Type: Sub-task
> Components: Compaction, regionserver
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0
>
> Attachments: HBASE-17914.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[
https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15968831#comment-15968831
]
Hadoop QA commented on HBASE-17914:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 0m 6s {color}
| {color:red} Docker failed to build yetus/hbase:8d52d23. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12863442/HBASE-17914.patch |
| JIRA Issue | HBASE-17914 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6428/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Create a new reader instead of cloning a new StoreFile when compaction
> --
>
> Key: HBASE-17914
> URL: https://issues.apache.org/jira/browse/HBASE-17914
> Project: HBase
> Issue Type: Sub-task
> Components: Compaction, regionserver
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0
>
> Attachments: HBASE-17914.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[ https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15968829#comment-15968829 ] Duo Zhang commented on HBASE-17914: --- For the readahead thing, I introduced a flag to control it for now. Will open new issues to remove it. I think we should not use readahead in most cases. > Create a new reader instead of cloning a new StoreFile when compaction > -- > > Key: HBASE-17914 > URL: https://issues.apache.org/jira/browse/HBASE-17914 > Project: HBase > Issue Type: Sub-task > Components: Compaction, regionserver >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-17914.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
