[jira] [Comment Edited] (HBASE-17138) Backport read-path offheap (HBASE-11425) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-17138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994227#comment-15994227
]
Jingcheng Du edited comment on HBASE-17138 at 5/3/17 3:52 PM:
--
The changes in {{RegionScanner}} in master branch is neither binary compatible
nor source compatible for branch-1, and changes in {{RegionCoprocessorHost}}
might be only binary compatible. If users move to a new 1.x version which
includes the off-heap read path, they have to change the customized region
scanners and coprocessors.
Do we allow this kind of backward compatibility break in minor release?
[~busbey] [~anoop.hbase] [~Apache9]
According to the compatibility matrix in HBase
(https://hbase.apache.org/book.html#hbase.versioning), such changes are
Server-Side Limited API compatibility, but the changes in {{RegionScanner}}
break both binary and source compatibility.
was (Author: jingcheng.du):
The changes in {{RegionScanner}} in master branch is neither binary compatible
nor source compatible for branch-1, and changes in {{RegionCoprocessorHost}}
might be only binary compatible. If users move to a new 1.x version which
includes the off-heap read path, they have to change the customized region
scanners and coprocessors.
Do we allow this kind of backward compatibility break in minor release?
[~busbey] [~anoop.hbase] [~Apache9]
According to the compatibility matrix in HBase
(https://hbase.apache.org/book.html#hbase.versioning), such changes are
Server-Side Limited API compatibility, it is allowed in HBase minor releases.
> Backport read-path offheap (HBASE-11425) to branch-1
>
>
> Key: HBASE-17138
> URL: https://issues.apache.org/jira/browse/HBASE-17138
> Project: HBase
> Issue Type: Improvement
>Reporter: Yu Li
>Assignee: Yu Sun
> Attachments:
> 0001-fix-EHB-511-Resolve-client-compatibility-issue-introduced-by-offheap-change.patch,
> 0001-to-EHB-446-offheap-hfile-format-should-keep-compatible-v3.patch,
> 0001-to-EHB-456-Cell-should-be-compatible-with-branch-1.1.2.patch
>
>
> From the
> [thread|http://mail-archives.apache.org/mod_mbox/hbase-user/201611.mbox/%3CCAM7-19%2Bn7cEiY4H9iLQ3N9V0NXppOPduZwk-hhgNLEaJfiV3kA%40mail.gmail.com%3E]
> of sharing our experience and performance data of read-path offheap usage in
> Alibaba search, we could see people are positive to have HBASE-11425 in
> branch-1, so I'd like to create a JIRA and move the discussion and decision
> making here.
> Echoing some comments from the mail thread:
> Bryan:
> Is the backported patch available anywhere? If it ends up not getting
> officially backported to branch-1 due to 2.0 around the corner, some of us
> who build our own deploy may want to integrate into our builds
> Andrew:
> Yes, please, the patches will be useful to the community even if we decide
> not to backport into an official 1.x release.
> Enis:
> I don't see any reason why we cannot backport to branch-1.
> Ted:
> Opening a JIRA would be fine. This makes it easier for people to obtain the
> patch(es)
> Nick:
> From the DISCUSS thread re: EOL of 1.1, it seems we'll continue to
> support 1.x releases for some time... I would guess these will be
> maintained until 2.2 at least. Therefore, offheap patches that have seen
> production exposure seem like a reasonable candidate for backport, perhaps in
> a 1.4 or 1.5 release timeframe.
> Anoop:
> Because of some compatibility issues, we decide that this will be done in 2.0
> only.. Ya as Andy said, it would be great to share the 1.x backported
> patches.
> The following is all the jira ids we have back ported:
> HBASE-10930 Change Filters and GetClosestRowBeforeTracker to work with Cells
> (Ram)
> HBASE-13373 Squash HFileReaderV3 together with HFileReaderV2 and
> AbstractHFileReader; ditto for Scanners and BlockReader, etc.
> HBASE-13429 Remove deprecated seek/reseek methods from HFileScanner.
> HBASE-13450 - Purge RawBytescomparator from the writers and readers for
> HBASE-10800 (Ram)
> HBASE-13501 - Deprecate/Remove getComparator() in HRegionInfo.
> HBASE-12048 Remove deprecated APIs from Filter.
> HBASE-10800 - Use CellComparator instead of KVComparator (Ram)
> HBASE-13679 Change ColumnTracker and SQM to deal with Cell instead of byte[],
> int, int.
> HBASE-13642 Deprecate RegionObserver#postScannerFilterRow CP hook with
> byte[],int,int args in favor of taking Cell arg.
> HBASE-13641 Deperecate Filter#filterRowKey(byte[] buffer, int offset, int
> length) in favor of filterRowKey(Cell firstRowCell).
> HBASE-13827 Delayed scanner close in KeyValueHeap and StoreScanner.
> HBASE-13871 Change RegionScannerImpl to deal with Cell instead of byte[],
> int, int.
> HBASE-11911 Break up tests into more fine grained categories (Alex Newman)
> HBASE-12059 Create
[jira] [Comment Edited] (HBASE-17138) Backport read-path offheap (HBASE-11425) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-17138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994227#comment-15994227
]
Jingcheng Du edited comment on HBASE-17138 at 5/3/17 3:46 PM:
--
The changes in {{RegionScanner}} in master branch is neither binary compatible
nor source compatible for branch-1, and changes in {{RegionCoprocessorHost}}
might be only binary compatible. If users move to a new 1.x version which
includes the off-heap read path, they have to change the customized region
scanners and coprocessors.
Do we allow this kind of backward compatibility break in minor release?
[~busbey] [~anoop.hbase] [~Apache9]
According to the compatibility matrix in HBase
(https://hbase.apache.org/book.html#hbase.versioning), such changes are
Server-Side Limited API compatibility, it is allowed in HBase minor releases.
was (Author: jingcheng.du):
The changes in {{RegionScanner}} in master branch is only binary compatible
(not source compatible) for branch-1 and changes in {{RegionCoprocessorHost}}
might be only binary compatible too. If users move to a new 1.x version which
includes the off-heap read path, they have to change the customized region
scanners and coprocessors before recompiling.
Do we allow this kind of backward compatibility break in minor release?
[~busbey] [~anoop.hbase] [~Apache9]
According to the compatibility matrix in HBase
(https://hbase.apache.org/book.html#hbase.versioning), such changes are
Server-Side Limited API compatibility, it is allowed in HBase minor releases.
> Backport read-path offheap (HBASE-11425) to branch-1
>
>
> Key: HBASE-17138
> URL: https://issues.apache.org/jira/browse/HBASE-17138
> Project: HBase
> Issue Type: Improvement
>Reporter: Yu Li
>Assignee: Yu Sun
> Attachments:
> 0001-fix-EHB-511-Resolve-client-compatibility-issue-introduced-by-offheap-change.patch,
> 0001-to-EHB-446-offheap-hfile-format-should-keep-compatible-v3.patch,
> 0001-to-EHB-456-Cell-should-be-compatible-with-branch-1.1.2.patch
>
>
> From the
> [thread|http://mail-archives.apache.org/mod_mbox/hbase-user/201611.mbox/%3CCAM7-19%2Bn7cEiY4H9iLQ3N9V0NXppOPduZwk-hhgNLEaJfiV3kA%40mail.gmail.com%3E]
> of sharing our experience and performance data of read-path offheap usage in
> Alibaba search, we could see people are positive to have HBASE-11425 in
> branch-1, so I'd like to create a JIRA and move the discussion and decision
> making here.
> Echoing some comments from the mail thread:
> Bryan:
> Is the backported patch available anywhere? If it ends up not getting
> officially backported to branch-1 due to 2.0 around the corner, some of us
> who build our own deploy may want to integrate into our builds
> Andrew:
> Yes, please, the patches will be useful to the community even if we decide
> not to backport into an official 1.x release.
> Enis:
> I don't see any reason why we cannot backport to branch-1.
> Ted:
> Opening a JIRA would be fine. This makes it easier for people to obtain the
> patch(es)
> Nick:
> From the DISCUSS thread re: EOL of 1.1, it seems we'll continue to
> support 1.x releases for some time... I would guess these will be
> maintained until 2.2 at least. Therefore, offheap patches that have seen
> production exposure seem like a reasonable candidate for backport, perhaps in
> a 1.4 or 1.5 release timeframe.
> Anoop:
> Because of some compatibility issues, we decide that this will be done in 2.0
> only.. Ya as Andy said, it would be great to share the 1.x backported
> patches.
> The following is all the jira ids we have back ported:
> HBASE-10930 Change Filters and GetClosestRowBeforeTracker to work with Cells
> (Ram)
> HBASE-13373 Squash HFileReaderV3 together with HFileReaderV2 and
> AbstractHFileReader; ditto for Scanners and BlockReader, etc.
> HBASE-13429 Remove deprecated seek/reseek methods from HFileScanner.
> HBASE-13450 - Purge RawBytescomparator from the writers and readers for
> HBASE-10800 (Ram)
> HBASE-13501 - Deprecate/Remove getComparator() in HRegionInfo.
> HBASE-12048 Remove deprecated APIs from Filter.
> HBASE-10800 - Use CellComparator instead of KVComparator (Ram)
> HBASE-13679 Change ColumnTracker and SQM to deal with Cell instead of byte[],
> int, int.
> HBASE-13642 Deprecate RegionObserver#postScannerFilterRow CP hook with
> byte[],int,int args in favor of taking Cell arg.
> HBASE-13641 Deperecate Filter#filterRowKey(byte[] buffer, int offset, int
> length) in favor of filterRowKey(Cell firstRowCell).
> HBASE-13827 Delayed scanner close in KeyValueHeap and StoreScanner.
> HBASE-13871 Change RegionScannerImpl to deal with Cell instead of byte[],
> int, int.
> HBASE-11911 Break up tests into more fine grained categories (Alex Newman)
> HBASE-12059 Create hbase-annotations modu
[jira] [Comment Edited] (HBASE-17138) Backport read-path offheap (HBASE-11425) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-17138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994227#comment-15994227
]
Jingcheng Du edited comment on HBASE-17138 at 5/3/17 10:39 AM:
---
The changes in {{RegionScanner}} in master branch is only binary compatible
(not source compatible) for branch-1 and changes in {{RegionCoprocessorHost}}
might be only binary compatible too. If users move to a new 1.x version which
includes the off-heap read path, they have to change the customized region
scanners and coprocessors before recompiling.
Do we allow this kind of backward compatibility break in minor release?
[~busbey] [~anoop.hbase] [~Apache9]
According to the compatibility matrix in HBase
(https://hbase.apache.org/book.html#hbase.versioning), such changes are
Server-Side Limited API compatibility, it is allowed in HBase minor releases.
was (Author: jingcheng.du):
The changes in {{RegionScanner}} in master branch is only binary compatible
(not source compatible) for branch-1, and its annotations are
{{@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.COPROC)}} and
{{@InterfaceStability.Evolving}}.
According to the compatibility matrix in HBase
(https://hbase.apache.org/book.html#hbase.versioning), does it mean we allow
the changes of binary compatibility in branch-1 minor releases? [~busbey]
[~anoop.hbase] [~Apache9]
> Backport read-path offheap (HBASE-11425) to branch-1
>
>
> Key: HBASE-17138
> URL: https://issues.apache.org/jira/browse/HBASE-17138
> Project: HBase
> Issue Type: Improvement
>Reporter: Yu Li
>Assignee: Yu Sun
> Attachments:
> 0001-fix-EHB-511-Resolve-client-compatibility-issue-introduced-by-offheap-change.patch,
> 0001-to-EHB-446-offheap-hfile-format-should-keep-compatible-v3.patch,
> 0001-to-EHB-456-Cell-should-be-compatible-with-branch-1.1.2.patch
>
>
> From the
> [thread|http://mail-archives.apache.org/mod_mbox/hbase-user/201611.mbox/%3CCAM7-19%2Bn7cEiY4H9iLQ3N9V0NXppOPduZwk-hhgNLEaJfiV3kA%40mail.gmail.com%3E]
> of sharing our experience and performance data of read-path offheap usage in
> Alibaba search, we could see people are positive to have HBASE-11425 in
> branch-1, so I'd like to create a JIRA and move the discussion and decision
> making here.
> Echoing some comments from the mail thread:
> Bryan:
> Is the backported patch available anywhere? If it ends up not getting
> officially backported to branch-1 due to 2.0 around the corner, some of us
> who build our own deploy may want to integrate into our builds
> Andrew:
> Yes, please, the patches will be useful to the community even if we decide
> not to backport into an official 1.x release.
> Enis:
> I don't see any reason why we cannot backport to branch-1.
> Ted:
> Opening a JIRA would be fine. This makes it easier for people to obtain the
> patch(es)
> Nick:
> From the DISCUSS thread re: EOL of 1.1, it seems we'll continue to
> support 1.x releases for some time... I would guess these will be
> maintained until 2.2 at least. Therefore, offheap patches that have seen
> production exposure seem like a reasonable candidate for backport, perhaps in
> a 1.4 or 1.5 release timeframe.
> Anoop:
> Because of some compatibility issues, we decide that this will be done in 2.0
> only.. Ya as Andy said, it would be great to share the 1.x backported
> patches.
> The following is all the jira ids we have back ported:
> HBASE-10930 Change Filters and GetClosestRowBeforeTracker to work with Cells
> (Ram)
> HBASE-13373 Squash HFileReaderV3 together with HFileReaderV2 and
> AbstractHFileReader; ditto for Scanners and BlockReader, etc.
> HBASE-13429 Remove deprecated seek/reseek methods from HFileScanner.
> HBASE-13450 - Purge RawBytescomparator from the writers and readers for
> HBASE-10800 (Ram)
> HBASE-13501 - Deprecate/Remove getComparator() in HRegionInfo.
> HBASE-12048 Remove deprecated APIs from Filter.
> HBASE-10800 - Use CellComparator instead of KVComparator (Ram)
> HBASE-13679 Change ColumnTracker and SQM to deal with Cell instead of byte[],
> int, int.
> HBASE-13642 Deprecate RegionObserver#postScannerFilterRow CP hook with
> byte[],int,int args in favor of taking Cell arg.
> HBASE-13641 Deperecate Filter#filterRowKey(byte[] buffer, int offset, int
> length) in favor of filterRowKey(Cell firstRowCell).
> HBASE-13827 Delayed scanner close in KeyValueHeap and StoreScanner.
> HBASE-13871 Change RegionScannerImpl to deal with Cell instead of byte[],
> int, int.
> HBASE-11911 Break up tests into more fine grained categories (Alex Newman)
> HBASE-12059 Create hbase-annotations module
> HBASE-12106 Move test annotations to test artifact (Enis Soztutar)
> HBASE-13916 Create MultiByteBuffer an aggregation of ByteBuffers.
> HBASE-15679 Assertion on wr
[jira] [Comment Edited] (HBASE-17138) Backport read-path offheap (HBASE-11425) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-17138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994227#comment-15994227
]
Jingcheng Du edited comment on HBASE-17138 at 5/3/17 10:10 AM:
---
The changes in {{RegionScanner}} in master branch is only binary compatible
(not source compatible) for branch-1, and its annotations are
{{@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.COPROC)}} and
{{@InterfaceStability.Evolving}}.
According to the compatibility matrix in HBase
(https://hbase.apache.org/book.html#hbase.versioning), does it mean we allow
the changes of binary compatibility in branch-1 minor releases? [~busbey]
[~anoop.hbase] [~Apache9]
was (Author: jingcheng.du):
The changes in {{RegionScanner}} in master branch is only binary compatible for
branch-1, and its annotations are
{{@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.COPROC)}} and
{{@InterfaceStability.Evolving}}.
According to the compatibility matrix in HBase
(https://hbase.apache.org/book.html#hbase.versioning), does it mean we allow
the changes of binary compatibility in branch-1 minor releases? [~busbey]
[~anoop.hbase] [~Apache9]
> Backport read-path offheap (HBASE-11425) to branch-1
>
>
> Key: HBASE-17138
> URL: https://issues.apache.org/jira/browse/HBASE-17138
> Project: HBase
> Issue Type: Improvement
>Reporter: Yu Li
>Assignee: Yu Sun
> Attachments:
> 0001-fix-EHB-511-Resolve-client-compatibility-issue-introduced-by-offheap-change.patch,
> 0001-to-EHB-446-offheap-hfile-format-should-keep-compatible-v3.patch,
> 0001-to-EHB-456-Cell-should-be-compatible-with-branch-1.1.2.patch
>
>
> From the
> [thread|http://mail-archives.apache.org/mod_mbox/hbase-user/201611.mbox/%3CCAM7-19%2Bn7cEiY4H9iLQ3N9V0NXppOPduZwk-hhgNLEaJfiV3kA%40mail.gmail.com%3E]
> of sharing our experience and performance data of read-path offheap usage in
> Alibaba search, we could see people are positive to have HBASE-11425 in
> branch-1, so I'd like to create a JIRA and move the discussion and decision
> making here.
> Echoing some comments from the mail thread:
> Bryan:
> Is the backported patch available anywhere? If it ends up not getting
> officially backported to branch-1 due to 2.0 around the corner, some of us
> who build our own deploy may want to integrate into our builds
> Andrew:
> Yes, please, the patches will be useful to the community even if we decide
> not to backport into an official 1.x release.
> Enis:
> I don't see any reason why we cannot backport to branch-1.
> Ted:
> Opening a JIRA would be fine. This makes it easier for people to obtain the
> patch(es)
> Nick:
> From the DISCUSS thread re: EOL of 1.1, it seems we'll continue to
> support 1.x releases for some time... I would guess these will be
> maintained until 2.2 at least. Therefore, offheap patches that have seen
> production exposure seem like a reasonable candidate for backport, perhaps in
> a 1.4 or 1.5 release timeframe.
> Anoop:
> Because of some compatibility issues, we decide that this will be done in 2.0
> only.. Ya as Andy said, it would be great to share the 1.x backported
> patches.
> The following is all the jira ids we have back ported:
> HBASE-10930 Change Filters and GetClosestRowBeforeTracker to work with Cells
> (Ram)
> HBASE-13373 Squash HFileReaderV3 together with HFileReaderV2 and
> AbstractHFileReader; ditto for Scanners and BlockReader, etc.
> HBASE-13429 Remove deprecated seek/reseek methods from HFileScanner.
> HBASE-13450 - Purge RawBytescomparator from the writers and readers for
> HBASE-10800 (Ram)
> HBASE-13501 - Deprecate/Remove getComparator() in HRegionInfo.
> HBASE-12048 Remove deprecated APIs from Filter.
> HBASE-10800 - Use CellComparator instead of KVComparator (Ram)
> HBASE-13679 Change ColumnTracker and SQM to deal with Cell instead of byte[],
> int, int.
> HBASE-13642 Deprecate RegionObserver#postScannerFilterRow CP hook with
> byte[],int,int args in favor of taking Cell arg.
> HBASE-13641 Deperecate Filter#filterRowKey(byte[] buffer, int offset, int
> length) in favor of filterRowKey(Cell firstRowCell).
> HBASE-13827 Delayed scanner close in KeyValueHeap and StoreScanner.
> HBASE-13871 Change RegionScannerImpl to deal with Cell instead of byte[],
> int, int.
> HBASE-11911 Break up tests into more fine grained categories (Alex Newman)
> HBASE-12059 Create hbase-annotations module
> HBASE-12106 Move test annotations to test artifact (Enis Soztutar)
> HBASE-13916 Create MultiByteBuffer an aggregation of ByteBuffers.
> HBASE-15679 Assertion on wrong variable in
> TestReplicationThrottler#testThrottling
> HBASE-13931 Move Unsafe based operations to UnsafeAccess.
> HBASE-12345 Unsafe based ByteBuffer Comparator.
> HBASE-13998 Remove CellComparator#compareRows
[jira] [Comment Edited] (HBASE-17138) Backport read-path offheap (HBASE-11425) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-17138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994227#comment-15994227
]
Jingcheng Du edited comment on HBASE-17138 at 5/3/17 10:05 AM:
---
The changes in {{RegionScanner}} in master branch is only binary compatible for
branch-1, and its annotations are
{{@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.COPROC)}} and
{{@InterfaceStability.Evolving}}.
According to the compatibility matrix in HBase
(https://hbase.apache.org/book.html#hbase.versioning), does it mean we allow
the changes of binary compatibility in branch-1 minor releases? [~busbey]
[~anoop.hbase] [~Apache9]
was (Author: jingcheng.du):
The changes in {{RegionScanner}} is only binary compatibility, and its
annotations are
{{@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.COPROC)}} and
{{@InterfaceStability.Evolving}}.
According to the compatibility matrix in HBase
(https://hbase.apache.org/book.html#hbase.versioning), does it mean we allow
the changes of binary compatibility in branch-1 minor releases? [~busbey]
[~anoop.hbase] [~Apache9]
> Backport read-path offheap (HBASE-11425) to branch-1
>
>
> Key: HBASE-17138
> URL: https://issues.apache.org/jira/browse/HBASE-17138
> Project: HBase
> Issue Type: Improvement
>Reporter: Yu Li
>Assignee: Yu Sun
> Attachments:
> 0001-fix-EHB-511-Resolve-client-compatibility-issue-introduced-by-offheap-change.patch,
> 0001-to-EHB-446-offheap-hfile-format-should-keep-compatible-v3.patch,
> 0001-to-EHB-456-Cell-should-be-compatible-with-branch-1.1.2.patch
>
>
> From the
> [thread|http://mail-archives.apache.org/mod_mbox/hbase-user/201611.mbox/%3CCAM7-19%2Bn7cEiY4H9iLQ3N9V0NXppOPduZwk-hhgNLEaJfiV3kA%40mail.gmail.com%3E]
> of sharing our experience and performance data of read-path offheap usage in
> Alibaba search, we could see people are positive to have HBASE-11425 in
> branch-1, so I'd like to create a JIRA and move the discussion and decision
> making here.
> Echoing some comments from the mail thread:
> Bryan:
> Is the backported patch available anywhere? If it ends up not getting
> officially backported to branch-1 due to 2.0 around the corner, some of us
> who build our own deploy may want to integrate into our builds
> Andrew:
> Yes, please, the patches will be useful to the community even if we decide
> not to backport into an official 1.x release.
> Enis:
> I don't see any reason why we cannot backport to branch-1.
> Ted:
> Opening a JIRA would be fine. This makes it easier for people to obtain the
> patch(es)
> Nick:
> From the DISCUSS thread re: EOL of 1.1, it seems we'll continue to
> support 1.x releases for some time... I would guess these will be
> maintained until 2.2 at least. Therefore, offheap patches that have seen
> production exposure seem like a reasonable candidate for backport, perhaps in
> a 1.4 or 1.5 release timeframe.
> Anoop:
> Because of some compatibility issues, we decide that this will be done in 2.0
> only.. Ya as Andy said, it would be great to share the 1.x backported
> patches.
> The following is all the jira ids we have back ported:
> HBASE-10930 Change Filters and GetClosestRowBeforeTracker to work with Cells
> (Ram)
> HBASE-13373 Squash HFileReaderV3 together with HFileReaderV2 and
> AbstractHFileReader; ditto for Scanners and BlockReader, etc.
> HBASE-13429 Remove deprecated seek/reseek methods from HFileScanner.
> HBASE-13450 - Purge RawBytescomparator from the writers and readers for
> HBASE-10800 (Ram)
> HBASE-13501 - Deprecate/Remove getComparator() in HRegionInfo.
> HBASE-12048 Remove deprecated APIs from Filter.
> HBASE-10800 - Use CellComparator instead of KVComparator (Ram)
> HBASE-13679 Change ColumnTracker and SQM to deal with Cell instead of byte[],
> int, int.
> HBASE-13642 Deprecate RegionObserver#postScannerFilterRow CP hook with
> byte[],int,int args in favor of taking Cell arg.
> HBASE-13641 Deperecate Filter#filterRowKey(byte[] buffer, int offset, int
> length) in favor of filterRowKey(Cell firstRowCell).
> HBASE-13827 Delayed scanner close in KeyValueHeap and StoreScanner.
> HBASE-13871 Change RegionScannerImpl to deal with Cell instead of byte[],
> int, int.
> HBASE-11911 Break up tests into more fine grained categories (Alex Newman)
> HBASE-12059 Create hbase-annotations module
> HBASE-12106 Move test annotations to test artifact (Enis Soztutar)
> HBASE-13916 Create MultiByteBuffer an aggregation of ByteBuffers.
> HBASE-15679 Assertion on wrong variable in
> TestReplicationThrottler#testThrottling
> HBASE-13931 Move Unsafe based operations to UnsafeAccess.
> HBASE-12345 Unsafe based ByteBuffer Comparator.
> HBASE-13998 Remove CellComparator#compareRows(byte[] left, int loffset, int
> llength, byte[] r
[jira] [Comment Edited] (HBASE-17138) Backport read-path offheap (HBASE-11425) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-17138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15905595#comment-15905595 ] Tianying Chang edited comment on HBASE-17138 at 3/10/17 7:44 PM: - [~anoop.hbase] Yes, we are interested in getting this patch in our production online facint cluster which is running on 1.2.1, any help from you will be great! It seems the customized version in Alibaba was forked from 1.1 (although on top of it with many of their private patches). If so I thought the effort fro backport to 1.2 should be similar? Do you have a high level sense what else need to be done besides those 70+ patches? Thanks was (Author: tychang): [~anoopamz] Yes, we are interested in getting this patch in our production online facint cluster which is running on 1.2.1, any help from you will be great! It seems the customized version in Alibaba was forked from 1.1 (although on top of it with many of their private patches). If so I thought the effort fro backport to 1.2 should be similar? Do you have a high level sense what else need to be done besides those 70+ patches? Thanks > Backport read-path offheap (HBASE-11425) to branch-1 > > > Key: HBASE-17138 > URL: https://issues.apache.org/jira/browse/HBASE-17138 > Project: HBase > Issue Type: Improvement >Reporter: Yu Li >Assignee: Yu Sun > Attachments: > 0001-fix-EHB-511-Resolve-client-compatibility-issue-introduced-by-offheap-change.patch, > 0001-to-EHB-446-offheap-hfile-format-should-keep-compatible-v3.patch, > 0001-to-EHB-456-Cell-should-be-compatible-with-branch-1.1.2.patch > > > From the > [thread|http://mail-archives.apache.org/mod_mbox/hbase-user/201611.mbox/%3CCAM7-19%2Bn7cEiY4H9iLQ3N9V0NXppOPduZwk-hhgNLEaJfiV3kA%40mail.gmail.com%3E] > of sharing our experience and performance data of read-path offheap usage in > Alibaba search, we could see people are positive to have HBASE-11425 in > branch-1, so I'd like to create a JIRA and move the discussion and decision > making here. > Echoing some comments from the mail thread: > Bryan: > Is the backported patch available anywhere? If it ends up not getting > officially backported to branch-1 due to 2.0 around the corner, some of us > who build our own deploy may want to integrate into our builds > Andrew: > Yes, please, the patches will be useful to the community even if we decide > not to backport into an official 1.x release. > Enis: > I don't see any reason why we cannot backport to branch-1. > Ted: > Opening a JIRA would be fine. This makes it easier for people to obtain the > patch(es) > Nick: > From the DISCUSS thread re: EOL of 1.1, it seems we'll continue to > support 1.x releases for some time... I would guess these will be > maintained until 2.2 at least. Therefore, offheap patches that have seen > production exposure seem like a reasonable candidate for backport, perhaps in > a 1.4 or 1.5 release timeframe. > Anoop: > Because of some compatibility issues, we decide that this will be done in 2.0 > only.. Ya as Andy said, it would be great to share the 1.x backported > patches. > The following is all the jira ids we have back ported: > HBASE-10930 Change Filters and GetClosestRowBeforeTracker to work with Cells > (Ram) > HBASE-13373 Squash HFileReaderV3 together with HFileReaderV2 and > AbstractHFileReader; ditto for Scanners and BlockReader, etc. > HBASE-13429 Remove deprecated seek/reseek methods from HFileScanner. > HBASE-13450 - Purge RawBytescomparator from the writers and readers for > HBASE-10800 (Ram) > HBASE-13501 - Deprecate/Remove getComparator() in HRegionInfo. > HBASE-12048 Remove deprecated APIs from Filter. > HBASE-10800 - Use CellComparator instead of KVComparator (Ram) > HBASE-13679 Change ColumnTracker and SQM to deal with Cell instead of byte[], > int, int. > HBASE-13642 Deprecate RegionObserver#postScannerFilterRow CP hook with > byte[],int,int args in favor of taking Cell arg. > HBASE-13641 Deperecate Filter#filterRowKey(byte[] buffer, int offset, int > length) in favor of filterRowKey(Cell firstRowCell). > HBASE-13827 Delayed scanner close in KeyValueHeap and StoreScanner. > HBASE-13871 Change RegionScannerImpl to deal with Cell instead of byte[], > int, int. > HBASE-11911 Break up tests into more fine grained categories (Alex Newman) > HBASE-12059 Create hbase-annotations module > HBASE-12106 Move test annotations to test artifact (Enis Soztutar) > HBASE-13916 Create MultiByteBuffer an aggregation of ByteBuffers. > HBASE-15679 Assertion on wrong variable in > TestReplicationThrottler#testThrottling > HBASE-13931 Move Unsafe based operations to UnsafeAccess. > HBASE-12345 Unsafe based ByteBuffer Comparator. > HBASE-13998 Remove CellComparator#compareRows(byte[] left, int loffset, int
[jira] [Comment Edited] (HBASE-17138) Backport read-path offheap (HBASE-11425) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-17138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15731546#comment-15731546 ] Anoop Sam John edited comment on HBASE-17138 at 12/8/16 8:46 AM: - With all these list and analysis, lets start a vote in dev@. Then we can decide based on the voting pattern there. Ya a new branch based on branch-1 might be needed with smaller patches getting in and then finally a merge back to branch-1 was (Author: anoop.hbase): With all these list and analysis, lets start a vote in dev@. Then we can decide based on the voting pattern there. > Backport read-path offheap (HBASE-11425) to branch-1 > > > Key: HBASE-17138 > URL: https://issues.apache.org/jira/browse/HBASE-17138 > Project: HBase > Issue Type: Improvement >Reporter: Yu Li >Assignee: Yu Sun > Attachments: > 0001-fix-EHB-511-Resolve-client-compatibility-issue-introduced-by-offheap-change.patch, > 0001-to-EHB-446-offheap-hfile-format-should-keep-compatible-v3.patch, > 0001-to-EHB-456-Cell-should-be-compatible-with-branch-1.1.2.patch > > > From the > [thread|http://mail-archives.apache.org/mod_mbox/hbase-user/201611.mbox/%3CCAM7-19%2Bn7cEiY4H9iLQ3N9V0NXppOPduZwk-hhgNLEaJfiV3kA%40mail.gmail.com%3E] > of sharing our experience and performance data of read-path offheap usage in > Alibaba search, we could see people are positive to have HBASE-11425 in > branch-1, so I'd like to create a JIRA and move the discussion and decision > making here. > Echoing some comments from the mail thread: > Bryan: > Is the backported patch available anywhere? If it ends up not getting > officially backported to branch-1 due to 2.0 around the corner, some of us > who build our own deploy may want to integrate into our builds > Andrew: > Yes, please, the patches will be useful to the community even if we decide > not to backport into an official 1.x release. > Enis: > I don't see any reason why we cannot backport to branch-1. > Ted: > Opening a JIRA would be fine. This makes it easier for people to obtain the > patch(es) > Nick: > From the DISCUSS thread re: EOL of 1.1, it seems we'll continue to > support 1.x releases for some time... I would guess these will be > maintained until 2.2 at least. Therefore, offheap patches that have seen > production exposure seem like a reasonable candidate for backport, perhaps in > a 1.4 or 1.5 release timeframe. > Anoop: > Because of some compatibility issues, we decide that this will be done in 2.0 > only.. Ya as Andy said, it would be great to share the 1.x backported > patches. > The following is all the jira ids we have back ported: > HBASE-10930 Change Filters and GetClosestRowBeforeTracker to work with Cells > (Ram) > HBASE-13373 Squash HFileReaderV3 together with HFileReaderV2 and > AbstractHFileReader; ditto for Scanners and BlockReader, etc. > HBASE-13429 Remove deprecated seek/reseek methods from HFileScanner. > HBASE-13450 - Purge RawBytescomparator from the writers and readers for > HBASE-10800 (Ram) > HBASE-13501 - Deprecate/Remove getComparator() in HRegionInfo. > HBASE-12048 Remove deprecated APIs from Filter. > HBASE-10800 - Use CellComparator instead of KVComparator (Ram) > HBASE-13679 Change ColumnTracker and SQM to deal with Cell instead of byte[], > int, int. > HBASE-13642 Deprecate RegionObserver#postScannerFilterRow CP hook with > byte[],int,int args in favor of taking Cell arg. > HBASE-13641 Deperecate Filter#filterRowKey(byte[] buffer, int offset, int > length) in favor of filterRowKey(Cell firstRowCell). > HBASE-13827 Delayed scanner close in KeyValueHeap and StoreScanner. > HBASE-13871 Change RegionScannerImpl to deal with Cell instead of byte[], > int, int. > HBASE-11911 Break up tests into more fine grained categories (Alex Newman) > HBASE-12059 Create hbase-annotations module > HBASE-12106 Move test annotations to test artifact (Enis Soztutar) > HBASE-13916 Create MultiByteBuffer an aggregation of ByteBuffers. > HBASE-15679 Assertion on wrong variable in > TestReplicationThrottler#testThrottling > HBASE-13931 Move Unsafe based operations to UnsafeAccess. > HBASE-12345 Unsafe based ByteBuffer Comparator. > HBASE-13998 Remove CellComparator#compareRows(byte[] left, int loffset, int > llength, byte[] right, int roffset, int rlength). > HBASE-13998 Remove CellComparator#compareRows()- Addendum to fix javadoc warn > HBASE-13579 Avoid isCellTTLExpired() for NO-TAG cases (partially backport > this patch) > HBASE-13448 New Cell implementation with cached component offsets/lengths. > HBASE-13387 Add ByteBufferedCell an extension to Cell. > HBASE-13387 Add ByteBufferedCell an extension to Cell - addendum. > HBASE-12650 Move ServerName to hbase-common module (partially backport this > patch) > HBASE-12296 Fil
