[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993583#comment-13993583
]
Ted Yu commented on HBASE-10965:
w.r.t. complexity, patch v7 creates FilterUtil class for utility methods and
adds new unit test. Apart from those, the patch is small.
bq. all implementors of Filter need to do the same.
There're many custom Filters in use today. It is hard to know which filters
override filterRow() but don't override hasFilterRow() - without autodetection
proposed in this JIRA.
Hence the following code from HRegion cannot be safely removed without
autodetection of hasFilterRow() :
{code}
return filter != null (!filter.hasFilterRow())
filter.filterRow();
{code}
One limitation is that we cannot optimize FilterList#filterRow() through short
circuit when FilterList#hasFilterRow() turns false. HBASE-11093 is blocked due
to this reason.
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Components: Filters
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt,
10965-v6.txt, 10965-v7.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
Downside to purely depending on hasFilterRow() telling us whether custom
filter overrides filterRow(List) or filterRow() is that the check below may
be rendered ineffective:
{code}
if (nextKv == KV_LIMIT) {
if (this.filter != null filter.hasFilterRow()) {
throw new IncompatibleFilterException(
Filter whose hasFilterRow() returns true is incompatible
with scan with limit!);
}
{code}
When user forgets to override hasFilterRow(), the above check becomes not
useful.
Another limitation is that we cannot optimize FilterList#filterRow() through
short circuit when FilterList#hasFilterRow() turns false.
See
https://issues.apache.org/jira/browse/HBASE-11093?focusedCommentId=13985149page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13985149
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow(). For FilterBase-derived classes, if
filterRow() is implemented and not inherited from FilterBase, it is
equivalent to having hasFilterRow() return true.
With precise detection of presence of Filter#filterRow(), the following code
from HRegion is no longer needed while backward compatibility is kept.
{code}
return filter != null (!filter.hasFilterRow())
filter.filterRow();
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13996838#comment-13996838
]
Ted Yu commented on HBASE-10965:
From Filter.java in 0.96 :
{code}
* Interface for row and column filters directly applied within the
regionserver.
...
* When implementing your own filters, consider inheriting {@link FilterBase}
to help
* you reduce boilerplate.
{code}
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Components: Filters
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt,
10965-v6.txt, 10965-v7.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
Downside to purely depending on hasFilterRow() telling us whether custom
filter overrides filterRow(List) or filterRow() is that the check below may
be rendered ineffective:
{code}
if (nextKv == KV_LIMIT) {
if (this.filter != null filter.hasFilterRow()) {
throw new IncompatibleFilterException(
Filter whose hasFilterRow() returns true is incompatible
with scan with limit!);
}
{code}
When user forgets to override hasFilterRow(), the above check becomes not
useful.
Another limitation is that we cannot optimize FilterList#filterRow() through
short circuit when FilterList#hasFilterRow() turns false.
See
https://issues.apache.org/jira/browse/HBASE-11093?focusedCommentId=13985149page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13985149
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow(). For FilterBase-derived classes, if
filterRow() is implemented and not inherited from FilterBase, it is
equivalent to having hasFilterRow() return true.
With precise detection of presence of Filter#filterRow(), the following code
from HRegion is no longer needed while backward compatibility is kept.
{code}
return filter != null (!filter.hasFilterRow())
filter.filterRow();
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13996049#comment-13996049
]
ramkrishna.s.vasudevan commented on HBASE-10965:
{code}
+ Class? extends Object declaringClass = m.getDeclaringClass();
+ Class? extends Object superCls = declaringClass.getSuperclass();
+ if (declaringClass.equals(clazz) superCls.equals(filterCls)) {
+// filter class directly overrides Filter
+return filter.hasFilterRow();
+ }
{code}
If the filter is just implementing directly the 'Filter' class.
'declaringClass.getSuperclass();' would return null? Am not sure. Can you
verify this once? If so 'null' check may be needed.
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Components: Filters
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt,
10965-v6.txt, 10965-v7.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
Downside to purely depending on hasFilterRow() telling us whether custom
filter overrides filterRow(List) or filterRow() is that the check below may
be rendered ineffective:
{code}
if (nextKv == KV_LIMIT) {
if (this.filter != null filter.hasFilterRow()) {
throw new IncompatibleFilterException(
Filter whose hasFilterRow() returns true is incompatible
with scan with limit!);
}
{code}
When user forgets to override hasFilterRow(), the above check becomes not
useful.
Another limitation is that we cannot optimize FilterList#filterRow() through
short circuit when FilterList#hasFilterRow() turns false.
See
https://issues.apache.org/jira/browse/HBASE-11093?focusedCommentId=13985149page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13985149
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow(). For FilterBase-derived classes, if
filterRow() is implemented and not inherited from FilterBase, it is
equivalent to having hasFilterRow() return true.
With precise detection of presence of Filter#filterRow(), the following code
from HRegion is no longer needed while backward compatibility is kept.
{code}
return filter != null (!filter.hasFilterRow())
filter.filterRow();
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13996106#comment-13996106
]
Lars Hofhansl commented on HBASE-10965:
---
-0 on this change. Just seems to add complexity for little gain.
If we could remove hasFilterRow completely from the interface (which we can do
in 1.0, I think) that would be a different story.
Now we have the worst of both worlds. Filter still has hasFilterRow() and on
top of that we have the reflection stuff.
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Components: Filters
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt,
10965-v6.txt, 10965-v7.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
Downside to purely depending on hasFilterRow() telling us whether custom
filter overrides filterRow(List) or filterRow() is that the check below may
be rendered ineffective:
{code}
if (nextKv == KV_LIMIT) {
if (this.filter != null filter.hasFilterRow()) {
throw new IncompatibleFilterException(
Filter whose hasFilterRow() returns true is incompatible
with scan with limit!);
}
{code}
When user forgets to override hasFilterRow(), the above check becomes not
useful.
Another limitation is that we cannot optimize FilterList#filterRow() through
short circuit when FilterList#hasFilterRow() turns false.
See
https://issues.apache.org/jira/browse/HBASE-11093?focusedCommentId=13985149page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13985149
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow(). For FilterBase-derived classes, if
filterRow() is implemented and not inherited from FilterBase, it is
equivalent to having hasFilterRow() return true.
With precise detection of presence of Filter#filterRow(), the following code
from HRegion is no longer needed while backward compatibility is kept.
{code}
return filter != null (!filter.hasFilterRow())
filter.filterRow();
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13996265#comment-13996265
]
Anoop Sam John commented on HBASE-10965:
My concern also same. We have to rely on the boolean return value as well as we
have auto detect. Code is more complex now. I initially thought we can remove
hasFilterRow () But as long as we have Filter and FilterBase 2 abstract super
classes we can not remove the api.
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Components: Filters
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt,
10965-v6.txt, 10965-v7.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
Downside to purely depending on hasFilterRow() telling us whether custom
filter overrides filterRow(List) or filterRow() is that the check below may
be rendered ineffective:
{code}
if (nextKv == KV_LIMIT) {
if (this.filter != null filter.hasFilterRow()) {
throw new IncompatibleFilterException(
Filter whose hasFilterRow() returns true is incompatible
with scan with limit!);
}
{code}
When user forgets to override hasFilterRow(), the above check becomes not
useful.
Another limitation is that we cannot optimize FilterList#filterRow() through
short circuit when FilterList#hasFilterRow() turns false.
See
https://issues.apache.org/jira/browse/HBASE-11093?focusedCommentId=13985149page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13985149
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow(). For FilterBase-derived classes, if
filterRow() is implemented and not inherited from FilterBase, it is
equivalent to having hasFilterRow() return true.
With precise detection of presence of Filter#filterRow(), the following code
from HRegion is no longer needed while backward compatibility is kept.
{code}
return filter != null (!filter.hasFilterRow())
filter.filterRow();
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13996819#comment-13996819
]
Ted Yu commented on HBASE-10965:
In 0.94, Filter is an interface.
This means user filters coming from 0.94 would always override FilterBase.
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Components: Filters
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt,
10965-v6.txt, 10965-v7.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
Downside to purely depending on hasFilterRow() telling us whether custom
filter overrides filterRow(List) or filterRow() is that the check below may
be rendered ineffective:
{code}
if (nextKv == KV_LIMIT) {
if (this.filter != null filter.hasFilterRow()) {
throw new IncompatibleFilterException(
Filter whose hasFilterRow() returns true is incompatible
with scan with limit!);
}
{code}
When user forgets to override hasFilterRow(), the above check becomes not
useful.
Another limitation is that we cannot optimize FilterList#filterRow() through
short circuit when FilterList#hasFilterRow() turns false.
See
https://issues.apache.org/jira/browse/HBASE-11093?focusedCommentId=13985149page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13985149
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow(). For FilterBase-derived classes, if
filterRow() is implemented and not inherited from FilterBase, it is
equivalent to having hasFilterRow() return true.
With precise detection of presence of Filter#filterRow(), the following code
from HRegion is no longer needed while backward compatibility is kept.
{code}
return filter != null (!filter.hasFilterRow())
filter.filterRow();
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13989175#comment-13989175
]
Hadoop QA commented on HBASE-10965:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12643213/10965-v7.txt
against trunk revision .
ATTACHMENT ID: 12643213
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9457//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9457//console
This message is automatically generated.
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Components: Filters
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt,
10965-v6.txt, 10965-v7.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
Downside to purely depending on hasFilterRow() telling us whether custom
filter overrides filterRow(List) or filterRow() is that the check below may
be rendered ineffective:
{code}
if (nextKv == KV_LIMIT) {
if (this.filter != null filter.hasFilterRow()) {
throw new IncompatibleFilterException(
Filter whose hasFilterRow() returns true is incompatible
with scan with limit!);
}
{code}
When user forgets to override hasFilterRow(), the above check becomes not
useful.
Another limitation is that we cannot optimize FilterList#filterRow() through
short circuit when FilterList#hasFilterRow() turns false.
See
https://issues.apache.org/jira/browse/HBASE-11093?focusedCommentId=13985149page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13985149
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow(). If filterRow() is implemented and
not inherited from FilterBase, it is equivalent to having hasFilterRow()
return true.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988837#comment-13988837
]
Ted Yu commented on HBASE-10965:
I ran test suite on Linux. There were 3 test failures:
{code}
Failed tests:
testQuarantineMissingHFile(org.apache.hadoop.hbase.util.TestHBaseFsck):
expected:2 but was:1
Tests in error:
testFlushCommitsWithAbort(org.apache.hadoop.hbase.client.TestMultiParallel):
test timed out after 30 milliseconds
testMasterRestartWhenTableInEnabling(org.apache.hadoop.hbase.master.TestAssignmentManager)
{code}
The above tests don't use filter.
They passed on re-run.
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Components: Filters
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt,
10965-v6.txt, 10965-v7.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
Downside to purely depending on hasFilterRow() telling us whether custom
filter overrides filterRow(List) or filterRow() is that the check below may
be rendered ineffective:
{code}
if (nextKv == KV_LIMIT) {
if (this.filter != null filter.hasFilterRow()) {
throw new IncompatibleFilterException(
Filter whose hasFilterRow() returns true is incompatible
with scan with limit!);
}
{code}
When user forgets to override hasFilterRow(), the above check becomes not
useful.
Another limitation is that we cannot optimize FilterList#filterRow() through
short circuit when FilterList#hasFilterRow() turns false.
See
https://issues.apache.org/jira/browse/HBASE-11093?focusedCommentId=13985149page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13985149
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow(). If filterRow() is implemented and
not inherited from FilterBase, it is equivalent to having hasFilterRow()
return true.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988406#comment-13988406
]
Ted Yu commented on HBASE-10965:
From https://builds.apache.org/job/PreCommit-HBASE-Build/9449/consoleFull :
{code}
HBASE-10965 patch is being downloaded at Fri May 2 20:53:24 UTC 2014 from
http://issues.apache.org/jira/secure/attachment/12643103/10965-v6.txt
...
Results :
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] --- maven-surefire-plugin:2.12-TRUNK-HBASE-2:test
(secondPartTestsExecution) @ hbase-shell ---
[INFO] Surefire report directory:
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-shell/target/surefire-reports
[INFO] Using configured provider
org.apache.maven.surefire.junitcore.JUnitCoreProvider
...
FATAL: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected
termination of the channel
hudson.remoting.RequestAbortedException:
hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected
termination of the channel
at
hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:41)
at
hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:34)
{code}
Basically hbase-server tests passed but the build got aborted.
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Components: Filters
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt,
10965-v6.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
Downside to purely depending on hasFilterRow() telling us whether custom
filter overrides filterRow(List) or filterRow() is that the check below may
be rendered ineffective:
{code}
if (nextKv == KV_LIMIT) {
if (this.filter != null filter.hasFilterRow()) {
throw new IncompatibleFilterException(
Filter whose hasFilterRow() returns true is incompatible
with scan with limit!);
}
{code}
When user forgets to override hasFilterRow(), the above check becomes not
useful.
Another limitation is that we cannot optimize FilterList#filterRow() through
short circuit when FilterList#hasFilterRow() turns false.
See
https://issues.apache.org/jira/browse/HBASE-11093?focusedCommentId=13985149page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13985149
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow(). If filterRow() is implemented and
not inherited from FilterBase, it is equivalent to having hasFilterRow()
return true.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985161#comment-13985161
]
Ted Yu commented on HBASE-10965:
Toward the tail of HBASE-11093, Anoop makes a case that as long as the
following code is present in HRegion, change in HBASE-11093 w.r.t.
FilterList#filterRow() cannot be applied:
{code}
private boolean filterRow() throws IOException {
// when hasFilterRow returns true, filter.filterRow() will be called
automatically inside
// filterRowCells(ListCell kvs) so we skip that scenario here.
return filter != null (!filter.hasFilterRow())
filter.filterRow();
}
{code}
See
https://issues.apache.org/jira/browse/HBASE-11093?focusedCommentId=13985149page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13985149
I think goal of this JIRA can be achieved without removing hasFilterRow().
New method for autodetecting presence of hasFilterRow() can be added to
FilterBase. We can rely on this new method in place where hasFilterRow() is
currently called in HRegion.
Post 1.0 release, we can remove hasFilterRow().
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Components: Filters
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
Downside to purely depending on hasFilterRow() telling us whether custom
filter overrides filterRow(List) or filterRow() is that the check below may
be rendered ineffective:
{code}
if (nextKv == KV_LIMIT) {
if (this.filter != null filter.hasFilterRow()) {
throw new IncompatibleFilterException(
Filter whose hasFilterRow() returns true is incompatible
with scan with limit!);
}
{code}
When user forgets to override hasFilterRow(), the above check becomes not
useful.
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow(). If filterRow() is implemented and
not inherited from FilterBase, it is equivalent to having hasFilterRow()
return true.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13971541#comment-13971541
]
stack commented on HBASE-10965:
---
bq. Accommodating brokenly implemented filters is already in place.
So the argument is that because we do something whack once, it is ok to
continue along the whack path?
Traditionally, compensations like these compounded makes the rig's operation
fickle.
And this example you cite seems categorically different from your patch in that
it is the internals dictating how execution flows rather than compensation for
what seems like a failure to implement the Interface as described (and that it
calls filterRow though hasFilterRow is false is a bug).
bq. But hasFilterRow() is not defined. There're other files in Phoenix which
exhibit the same pattern.
Eh... file a bug with pheonix that they've failed to implement filters properly.
bq. The contract is clearly published but not all people follow it.
The javadoc is not decisive. That could be tightened up. If folks don't read
the contract, then it makes it tough. We should fail fast though rather than
try make guess at what was intended.
If this issue had examination of why we have these two tied methods at all I'd
have some sympathy for the proposal but as is it looks to me to be making a bad
situation worse.
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
Downside to purely depending on hasFilterRow() telling us whether custom
filter overrides filterRow(List) or filterRow() is that the check below may
be rendered ineffective:
{code}
if (nextKv == KV_LIMIT) {
if (this.filter != null filter.hasFilterRow()) {
throw new IncompatibleFilterException(
Filter whose hasFilterRow() returns true is incompatible
with scan with limit!);
}
{code}
When user forgets to override hasFilterRow(), the above check becomes not
useful.
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow(). If filterRow() is implemented and
not inherited from FilterBase, it is equivalent to having hasFilterRow()
return true.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13971563#comment-13971563
]
Ted Yu commented on HBASE-10965:
bq. file a bug with pheonix that they've failed to implement filters properly.
PHOENIX-910 was logged two weeks ago.
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
Downside to purely depending on hasFilterRow() telling us whether custom
filter overrides filterRow(List) or filterRow() is that the check below may
be rendered ineffective:
{code}
if (nextKv == KV_LIMIT) {
if (this.filter != null filter.hasFilterRow()) {
throw new IncompatibleFilterException(
Filter whose hasFilterRow() returns true is incompatible
with scan with limit!);
}
{code}
When user forgets to override hasFilterRow(), the above check becomes not
useful.
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow(). If filterRow() is implemented and
not inherited from FilterBase, it is equivalent to having hasFilterRow()
return true.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[ https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969266#comment-13969266 ] Anoop Sam John commented on HBASE-10965: In initial time of 94 , I remember that Filter#hasFilterRow() was associated with Filter#filterRow(ListKV) We used this boolean API in 2 places 1. To check whether Scan is with a Filter, which is having row filtering, and with a batch value set. This is not correct way and we have to fail then 2. The place before we call Filter#filterRow(ListKV) in HRegion Later we made it like hasFilterRow() to return true when filterRow() is also implemented and so same is used in code where we call filterRow() also. Actually we need to use this boolean API only in case 1. When one uses a filter and a Scan batch wrongly, we will fail the scan op 1st step itself. So there is no need to check this boolean API value before calling the actual Filter methods. The impl in FilterBase do nothing and no problem in calling that I believe. So with this Jira, if there is consensus to remove even the hasFilterRow() method, we can simplify the things. Just in place where we check the Scan filter and batch to see whether we can proceed with the open scanner, just do this auto detect of method impls and act. In other places of HRegion scan flow, no need to rely on any of these boolean values of hasFilterRow() returns. Make some sense? When we auto detect presence of methods take care of the FL case. ( FL inside FL) which I was saying above. Automate detection of presence of Filter#filterRow() Key: HBASE-10965 URL: https://issues.apache.org/jira/browse/HBASE-10965 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt There is potential inconsistency between the return value of Filter#hasFilterRow() and presence of Filter#filterRow(). Filters may override Filter#filterRow() while leaving return value of Filter#hasFilterRow() being false (inherited from FilterBase). This JIRA aims to remove the inconsistency by automatically detecting the presence of overridden Filter#filterRow(). If filterRow() is implemented and not inherited from FilterBase, it is equivalent to having hasFilterRow() return true. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969457#comment-13969457
]
ramkrishna.s.vasudevan commented on HBASE-10965:
bq.This will be problematic When a FL contains another FL, the inner one will
always say hasFilterRow as true.
Good catch.
We could deprecate hasFilterRow() if there is a consensus in removing it.
Later can remove it.
You mean to use the hasFilterrow() only in openScanner and not use in every
next() calls? Ideally if the
{code}
if ((isEmptyRow || ret == FilterWrapper.FilterRowRetCode.EXCLUDE) ||
filterRow()) {
{code}
filterRow() is getting removed here then hasfilterRow() is called to check the
stopRow alone. In that case moving it to the openScanner makes sense instead
of doing the reflection way of using it.
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow(). If filterRow() is implemented and
not inherited from FilterBase, it is equivalent to having hasFilterRow()
return true.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969760#comment-13969760
]
Ted Yu commented on HBASE-10965:
bq. You would change how we run filters to accommodate brokenly implemented
filters?
Accommodating brokenly implemented filters is already in place. Please take a
look at the following method in HRegion#RegionScannerImpl:
{code}
private boolean filterRow() throws IOException {
// when hasFilterRow returns true, filter.filterRow() will be called
automatically inside
// filterRowCells(ListCell kvs) so we skip that scenario here.
return filter != null (!filter.hasFilterRow())
filter.filterRow();
}
{code}
When filter.hasFilterRow() returns false, filter.filterRow() is still called.
bq. Are folks implementing filters w/o reading the Interface
Here is one example:
phoenix-core/src/main/java/org/apache/phoenix/filter/MultiKeyValueComparisonFilter.java
{code}
@Override
public boolean filterRow() {
{code}
But hasFilterRow() is not defined. There're other files in Phoenix which
exhibit the same pattern.
bq. is it not clear on what needs overriding
From hbase-client/src/main/java/org/apache/hadoop/hbase/filter/Filter.java
(since 0.96):
{code}
/**
* Primarily used to check for conflicts with scans(such as scans that do not
read a full row at a
* time).
*
* @return True if this filter actively uses filterRow(List) or filterRow().
*/
abstract public boolean hasFilterRow();
{code}
The contract is clearly published but not all people follow it.
If a custom filter overrides filterRow(List) or filterRow(), it is easy to use
reflection to detect. Having hasFilterRow() method leaves room for
inconsistency.
If we can reach consensus on whether hasFilterRow() method should be deprecated
first and then removed, I will address Lars' and Anoop's review comments in the
next patch.
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow(). If filterRow() is implemented and
not inherited from FilterBase, it is equivalent to having hasFilterRow()
return true.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969773#comment-13969773
]
Ted Yu commented on HBASE-10965:
Another downside to purely depending on hasFilterRow() telling us whether
custom filter overrides filterRow(List) or filterRow() is that the check below
may be rendered ineffective:
{code}
if (nextKv == KV_LIMIT) {
if (this.filter != null filter.hasFilterRow()) {
throw new IncompatibleFilterException(
Filter whose hasFilterRow() returns true is incompatible with
scan with limit!);
}
{code}
When user forgets to override hasFilterRow(), the above check becomes not
useful.
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow(). If filterRow() is implemented and
not inherited from FilterBase, it is equivalent to having hasFilterRow()
return true.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13968349#comment-13968349
]
Anoop Sam John commented on HBASE-10965:
{code}
+if (hasFilterRow == null) {
+ for (Filter filter : filters) {
+if (FilterWrapper.isFilterMethodImplemented(filter, filterRow)) {
+ this.hasFilterRow = true;
+ return true;
+}
}
+ this.hasFilterRow = false;
}
{code}
This will be problematic When a FL contains another FL, the inner one will
always say hasFilterRow as true.
So here as per V4 patch we will auto detect presence of filterRow() overridden
alone?
FilterWrapper#hasFilterRow() just checks presence of filterRow() ? ! When
Filter extending
filterRowCells(ListCell kvs) or filterRow(ListKeyValue kvs) hasFilter()
have to return true (And we need fail in HRegion a scan with filter and batch
being set)
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow(). If filterRow() is implemented and
not inherited from FilterBase, it is equivalent to having hasFilterRow()
return true.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[ https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13967425#comment-13967425 ] ramkrishna.s.vasudevan commented on HBASE-10965: Will check this later so will get back on this tomorrow or monday. Automate detection of presence of Filter#filterRow() Key: HBASE-10965 URL: https://issues.apache.org/jira/browse/HBASE-10965 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Attachments: 10965-v1.txt, 10965-v2.txt There is potential inconsistency between the return value of Filter#hasFilterRow() and presence of Filter#filterRow(). Filters may override Filter#filterRow() while leaving return value of Filter#hasFilterRow() being false (inherited from FilterBase). This JIRA aims to remove the inconsistency by automatically detecting the presence of overridden Filter#filterRow(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13967579#comment-13967579
]
Hadoop QA commented on HBASE-10965:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12639934/10965-v3.txt
against trunk revision .
ATTACHMENT ID: 12639934
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9271//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9271//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9271//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9271//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9271//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9271//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9271//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9271//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9271//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9271//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9271//console
This message is automatically generated.
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow(). If filterRow() is implemented and
not inherited from FilterBase, it is equivalent to having hasFilterRow()
return true.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[ https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13967677#comment-13967677 ] stack commented on HBASE-10965: --- bq. There is potential inconsistency between the return value of Filter#hasFilterRow() and presence of Filter#filterRow(). Filters may override Filter#filterRow() while leaving return value of Filter#hasFilterRow() being false (inherited from FilterBase). The particular filter implementation is broke then? You would change how we run filters to accommodate brokenly implemented filters? bq. This JIRA aims to remove the inconsistency by automatically detecting the presence of overridden Filter#filterRow(). If filterRow() is implemented and not inherited from FilterBase, it is equivalent to having hasFilterRow() return true. Why do we need this? Who has this problem? Are folks implementing filters w/o reading the Interface (is it not clear on what needs overriding -- if not, shouldn't we fix that?) or who is implementing filters w/o test that proves their filter does the right thing? Is the filter API broke? Should we fix that rather than add what looks like a band-aid, one that may 'surprise' implementors? Automate detection of presence of Filter#filterRow() Key: HBASE-10965 URL: https://issues.apache.org/jira/browse/HBASE-10965 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt There is potential inconsistency between the return value of Filter#hasFilterRow() and presence of Filter#filterRow(). Filters may override Filter#filterRow() while leaving return value of Filter#hasFilterRow() being false (inherited from FilterBase). This JIRA aims to remove the inconsistency by automatically detecting the presence of overridden Filter#filterRow(). If filterRow() is implemented and not inherited from FilterBase, it is equivalent to having hasFilterRow() return true. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13967692#comment-13967692
]
Hadoop QA commented on HBASE-10965:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12639959/10965-v4.txt
against trunk revision .
ATTACHMENT ID: 12639959
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9272//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9272//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9272//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9272//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9272//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9272//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9272//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9272//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9272//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9272//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9272//console
This message is automatically generated.
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow(). If filterRow() is implemented and
not inherited from FilterBase, it is equivalent to having hasFilterRow()
return true.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13967693#comment-13967693
]
Lars Hofhansl commented on HBASE-10965:
---
Is this right?
{code}
- if ((isEmptyRow || ret == FilterWrapper.FilterRowRetCode.EXCLUDE) ||
filterRow()) {
+ if ((isEmptyRow || ret == FilterWrapper.FilterRowRetCode.EXCLUDE)) {
{code}
And this? (why not continue to call hasFilterRow(), which we just fixed in the
patch to do right thing, and most important cache the outcode)
{code}
-if (this.hasFilter() this.filter.hasFilterRow()) {
+if (this.hasFilter()
FilterWrapper.isFilterMethodImplemented(this.filter, filterRow)) {
{code}
The last point raises a bigger discussion: If we autodetect this, it should be
removed from the filter API completely and moved into the framework. I.e.
there's no need for Filter to even have hasFilterRow() anymore.
Lastly, it would certainly be nice to autodetect this. Is it worth effort,
though?
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt, 10965-v3.txt, 10965-v4.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow(). If filterRow() is implemented and
not inherited from FilterBase, it is equivalent to having hasFilterRow()
return true.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[ https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13967201#comment-13967201 ] Ted Yu commented on HBASE-10965: For Phoenix, I did the following: apply patch to 0.98 branch mvn clean package install -DskipTests in Phoenix workspace, change hbase-hadoop2.version to 0.98.2-SNAPSHOT Use command 'nohup ~/apache-maven-3.1.1/bin/mvn clean install -Dhadoop.profile=2 h2.out ' to run unit tests and they passed. Automate detection of presence of Filter#filterRow() Key: HBASE-10965 URL: https://issues.apache.org/jira/browse/HBASE-10965 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Attachments: 10965-v1.txt There is potential inconsistency between the return value of Filter#hasFilterRow() and presence of Filter#filterRow(). Filters may override Filter#filterRow() while leaving return value of Filter#hasFilterRow() being false (inherited from FilterBase). This JIRA aims to remove the inconsistency by automatically detecting the presence of overridden Filter#filterRow(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[ https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13967306#comment-13967306 ] Ted Yu commented on HBASE-10965: TestMultiParallel fails on QA bot. But it doesn't use any filter and it passes locally on Mac. Automate detection of presence of Filter#filterRow() Key: HBASE-10965 URL: https://issues.apache.org/jira/browse/HBASE-10965 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Attachments: 10965-v1.txt There is potential inconsistency between the return value of Filter#hasFilterRow() and presence of Filter#filterRow(). Filters may override Filter#filterRow() while leaving return value of Filter#hasFilterRow() being false (inherited from FilterBase). This JIRA aims to remove the inconsistency by automatically detecting the presence of overridden Filter#filterRow(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13967315#comment-13967315
]
Hadoop QA commented on HBASE-10965:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12639884/10965-v1.txt
against trunk revision .
ATTACHMENT ID: 12639884
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestMultiParallel
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9263//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9263//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9263//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9263//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9263//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9263//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9263//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9263//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9263//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9263//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9263//console
This message is automatically generated.
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow().
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13967380#comment-13967380
]
Anoop Sam John commented on HBASE-10965:
We need similar change here also.
HRegion#nextInternal(ListCell results, int limit)
{code}
if (this.filter != null filter.hasFilterRow()) {
throw new IncompatibleFilterException(
Filter whose hasFilterRow() returns true is incompatible with scan with
limit!);
}
{code}
Also we rely on this hasFilterRow() in other places too. There also some
changes needed?
Why is the change in FilterList? IMO in FilterList we have to decide based on
individual filters within it. We can not return true directly.
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow().
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[ https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13967387#comment-13967387 ] Ted Yu commented on HBASE-10965: The filter referenced in above code is of FilterWrapper where hasFilterRow() uses the return value from isFilterMethodImplemented(). bq. Also we rely on this hasFilterRow() in other places too. I searched all references to hasFilterRow() and made corresponding changes. Filter#hasFilterRow() can now be deprecated. bq. Why is the change in FilterList This is to make hasFilterRow() consistent with the filterRow() method. Prior to this change, filterRow() iterates through filters and calls filterRow(). So the change is consistent with the description of Filter#hasFilterRow(). Automate detection of presence of Filter#filterRow() Key: HBASE-10965 URL: https://issues.apache.org/jira/browse/HBASE-10965 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Attachments: 10965-v1.txt, 10965-v2.txt There is potential inconsistency between the return value of Filter#hasFilterRow() and presence of Filter#filterRow(). Filters may override Filter#filterRow() while leaving return value of Filter#hasFilterRow() being false (inherited from FilterBase). This JIRA aims to remove the inconsistency by automatically detecting the presence of overridden Filter#filterRow(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10965) Automate detection of presence of Filter#filterRow()
[
https://issues.apache.org/jira/browse/HBASE-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13967398#comment-13967398
]
Hadoop QA commented on HBASE-10965:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12639910/10965-v2.txt
against trunk revision .
ATTACHMENT ID: 12639910
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9268//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9268//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9268//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9268//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9268//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9268//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9268//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9268//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9268//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9268//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9268//console
This message is automatically generated.
Automate detection of presence of Filter#filterRow()
Key: HBASE-10965
URL: https://issues.apache.org/jira/browse/HBASE-10965
Project: HBase
Issue Type: Task
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10965-v1.txt, 10965-v2.txt
There is potential inconsistency between the return value of
Filter#hasFilterRow() and presence of Filter#filterRow().
Filters may override Filter#filterRow() while leaving return value of
Filter#hasFilterRow() being false (inherited from FilterBase).
This JIRA aims to remove the inconsistency by automatically detecting the
presence of overridden Filter#filterRow().
--
This message was sent by Atlassian JIRA
(v6.2#6252)
